Stop Starting With the Dashboard

This is a guest post written by Spark Compliance’s Heather Smith.

Why effective compliance monitoring starts with the decision leadership needs to make – not the metrics you already have.

Imagine your General Counsel comes to you with what sounds like a straightforward question: “Our investigations are taking longer. Do we need another investigator?” You pull up the dashboard and see that average case closure time has increased from 55 days to 72 days. You have a metric, you can quantify the change, and you can show the trend. But can you actually answer the question?

Not yet.

Maybe investigator caseloads have grown or cases are sitting too long before assignment. But maybe a handful of unusually complex investigations are driving the increase. Or investigators may be finishing their work on time while cases remain open because corrective actions are delayed.

The same 72-day average could point to very different problems and very different decisions. Another investigator might be exactly what you need. Or it might solve absolutely nothing.

This is one of the challenges Britini Jennings and I recently explored during our From Data to Decisions session at SCCE’s 25th Annual Compliance & Ethics Institute. In my experience, most compliance teams aren’t short on data. We have investigation data, hotline data, third-party data, training data, audit findings, risk assessments and increasingly sophisticated dashboards.

The harder question is whether that data actually helps us answer the questions leadership is asking.

When we’re designing or improving compliance monitoring, it’s natural to start with what we already have. What data do our systems capture? What reports can we generate? What KPIs are other organizations using? What can we put on a dashboard? The problem is that we can end up measuring what’s readily available instead of what’s actually useful. A better starting point is the decision. What is leadership likely to need to know about our material compliance risks? For example:

• Are our controls working?
• Is the risk increasing?
• Do we need to intervene?
• Where should we deploy limited resources?

This doesn’t mean waiting for leadership to ask the question. Effective monitoring should be proactive. We may not know exactly what the CEO or Audit Committee will ask six months from now, but we know the kinds of decisions they’re going to have to make. Our monitoring should give us enough visibility into our material risks that we’re ready when those questions come.

Consider a company that added 150 distributors over the past year. The CEO wants to know whether the company’s third-party controls are keeping up with that growth. Compliance pulls the dashboard and reports a 98% due diligence completion rate.

At first glance, 98% sounds reassuring. But what if the incomplete 2% includes your highest-risk distributors? What if distributors are being activated before approval, high-risk reviews are backing up, overrides are increasing, or payments are being made before due diligence is complete?

There’s nothing inherently wrong with the 98% completion rate. It just doesn’t answer the question the CEO is really asking: Is rapid growth putting pressure on our controls?

That’s an important distinction. We can have a perfectly accurate metric and still not have the information we need to make a decision.

Map the Risk Before You Choose the Indicator

Once we start with the leadership decision, the next question is: What information would actually help us answer it?

That’s where risk mapping comes in.

Using the distributor example, think about the underlying process. The business identifies a distributor, due diligence begins, Compliance reviews higher-risk parties, approval occurs, the distributor is activated and eventually the distributor is paid.

Before deciding which pieces of that process should become KPIs, ask where the process could break down and what you would expect to see if it did.

If growth were overwhelming the process, high-risk reviews might begin piling up. If the business were circumventing the control, distributors might be activated before approval or overrides might increase. Further downstream, payments might occur before due diligence is complete.

Those are our candidate indicators. And I say candidate because just because we can measure something doesn’t mean we should monitor it. At this point, we’re looking for evidence that could give us meaningful visibility into the risk.

One question keeps this exercise focused: If this risk were happening, what would we expect to see?

The Same Process Does Not Always Require the Same Monitoring

Investigations are a good example. In one location, the bottleneck may be assigning new allegations. In another, growing investigator caseloads may be slowing the work. In a third, investigations may be completed efficiently while corrective-action delays keep cases open.

All three may report the same headline metric – average investigation closure time – while dealing with very different problems. The monitoring needed to understand those problems should be different too.

That’s why I’m cautious about simply copying another organization’s dashboard or adopting a standard list of “best practice” KPIs.

Your monitoring should follow your risk, not somebody else’s list of metrics.

A Different Way to Start

If you’re evaluating your monitoring program, choose one material compliance risk and resist the urge to open the dashboard. Instead, work through four questions:

• What decisions is leadership likely to need to make about this risk?
• Where could the risk materialize in the underlying process?
• If something were beginning to break down, what would we expect to see?
• What evidence would give us visibility into those failure points?

Then compare the candidate indicators you’ve identified against the metrics you’re already collecting. You may find that you’re already monitoring exactly what you need. You may also find that you’ve been carefully measuring something for years simply because the data was available.

That’s really the shift we’re trying to make with Data to Decisions. We don’t need more data for the sake of having more data. We need enough visibility into our material risks to be able to tell leadership: Here’s what we’re seeing. Here’s what it means. And here’s what we think you should do about it.

So when the General Counsel asks whether increasing investigation closure time means the company needs another investigator, the 72-day average may be where the conversation begins. It shouldn’t be where the analysis ends.

The real question isn’t “What can we measure?” It’s “What do we need to know?”

Next in the Series

Once we’ve identified potential indicators, the next question is which ones are actually worth monitoring. More isn’t necessarily better. In Part 2, “300 Hotline Reports Is a Fact. It Isn’t an Answer,” I’ll look at how we move from a long list of possible metrics to the smallest useful set of indicators that helps leadership

From Data to Decisions

This is Part 1 of a six-part series based on the monitoring framework Britini Jennings and I recently presented at SCCE’s 25th Annual Compliance & Ethics Institute.

Part 1: Stop Starting With the Dashboard
Part 2: 300 Hotline Reports Is a Fact. It Isn’t an Answer.
Part 3: If You Can’t Define It, You Can’t Monitor It
Part 4: When Does a Metric Become a Signal?
Part 5: A 9 Tells You Where to Look – Not What to Do
Part 6: From Signal to Action: What Should Leadership Actually Do?

About the author
Heather Smith is a Compliance Advisory Services Consultant with Spark Compliance Consulting, a Diligent brand. A former Chief Compliance Officer, corporate counsel and Navy JAG officer, she advises organizations on designing and strengthening effective compliance programs.