Malcolm Sparrow tells NRCoP Conference how to be a problem-centric regulator

Harvard’s Professor Malcolm Sparrow once summarised the concept of problem-based regulation with a simple mantra – ‘pick important problems: fix them’ – but he says turning that simple notion into real-world programs that have positive outcomes is far harder.

The legend of regulatory theory and practice was keynote speaker at the recent annual conference of the ANZSOG-auspiced National Regulators Community of Practice (NRCoP), explaining the value of problem-based regulation and the difficulties that arise in implementing it.

“Even after more than two decades of work on the concept of problem-based work it still remains very difficult, and fragile even once you’ve implemented it,” he said.

He said that problem-based regulation was distinct in theory and practice from more traditional program or process-based approaches.

“The program-centric approach rests on developing an idea of what we will do in general to deal with a broad class of harms. We’ll run an inspection schedule, we’ll have a complaints line—a range of processes and functions you set up and operate.

“The alternate method is to look more closely at the general class of harms and understand that it is not one thing at all, there’s tens or hundreds of different varieties of harms. They don’t necessarily behave the same way and they are not all necessarily treated adequately by your routine programs.”

“The problem-centric approach emerges as an opportunity to focus carefully on one or another of these highly-defined concentrations and hopefully produce a tailor-made intervention. The solutions are usually innovative – not because you set out to be innovative – but because everything you’ve ever done before about this problem clearly hadn’t fixed it, so chances are something new and different is needed.”

He gave the example of a regulatory agency addressing the fact that hospitals constituted a high-risk industry by targeting thousands of inspections on that one industry sector.

“That’s not problem solving, it’s risk-based targeting based on analysis, but it’s not problem solving. But if, for example, you move down from the level of the hospital to one of the specific issues that makes a hospital environment dangerous—the issue of back-strain injuries for nurses—this is now looking more like an actionable project. It’s something that you could organise around and be inventive about, update equipment and practices and probably get a significant reduction in a year or two, assuming you did it right.”

Why ‘picking important problems’ is harder than it sounds

Professor Sparrow said he had come up with the ‘Pick important problems: fix them’ line in response to a request to summarise his program in a short sentence and had regretted it ever since.

“I’ve been regretting it ever since because it causes a lot of anger and frustration. It looks so obviously common-sense and people think there is nothing to it. They are offended and say, ‘everything we do is about this, you are insulting us to suggest otherwise’.

“But this is a very specific operational method. If you say you do it already, then let me ask you some simple mechanical questions? Who picks the problems? How often do they pick? Do they pick from a list? How is the list formulated and who puts the list together? What is the set of criteria for selecting problems from the list, and where are those criteria written down? What are your in-house systems for allocating people, time, money, support to each problem you choose to work on in this way?”

“When I explain the simple mechanics of what you’d have to be doing, they come around and say: ‘Oh, I see, we could never do that in my organisation’. They are aware of all kinds of constraints that make this awkward given the way the organisation is set up and the way it normally operates.”

He said that many regulatory organisations were somewhere along the journey of learning how to do problem-based work, some having done some ad-hoc projects but having no formal apparatus for managing such work, and others working hard on formalising the structures necessary to support it and make it possible.

Which problems should regulators focus on?

Professor Sparrow said that it was important to be highly selective and spend time identifying and targeting the highest priority problems so as not to exceed the organisation’s capacity.

“We are looking for risks, problems, patterns of behaviour out there in the world which we have a clear imperative to reduce, to prevent, to eliminate, to mitigate, depending on the nature of the risk,” he said.

“That makes problem solving an operational method for working on the world and not a managerial method for working on the organisation.”

“When nominations come in you shouldn’t feel obliged to launch all of them. If you do you will bury yourself. If a problem passes all five filters (see diagram above) and you’ve got the resources and this qualifies as a current priority, then you are ready to launch one. And in respect of the projects you don’t choose, you need to learn very early how to be diplomatic and say ‘No’ to people in a way that keeps them engaged.

How regulators can develop problem-based solutions

Professor Sparrow outlined a six-step process for developing problem-based solutions, which he developed based on surveys of fifteen professions.

He said that following a series of discrete steps allowed for managerial clarity about what stage a project team has reached and helps to justify investment of resources and the imposition of suitable time pressures. This particular recipe also helps to avoid the ‘leap to action’ syndrome that could lead to hasty and poorly thought-out responses.

He said that nominating and precisely defining problems were best seen as two separate steps because the nominators may not be on the final project team, so the nomination process is actually part of the background managerial system, rather than part of the protocol to be followed by a project team once established.

It was vital that Step Three – determining how to measure impact – was done before solutions were developed, let alone implemented.

“This is an incredibly important discipline that almost no one will naturally want to follow,” he said.

“If you act first and then turn to metrics as an afterthought, it is too late, you are flying blind. If you develop solutions first, people will be tempted to measure the implementation of the plan, not whether the plan had an effect on the risk that we wanted to address. “Once you have the metrics in place, and benchmarked, you can search through a range of possible solutions without needing to change the metrics or what you think should happen to them if the project succeeds.”

“A lot of agencies stumble in their efforts to solve problems because they fail to determine how to measure impact, so they can’t tell if they are succeeding or failing, and then things eventually just peter out.”

He said that knowing when to close projects and move on to other priorities was a vital part of problem-based regulation and that organisations should aim to close projects at the same rate they opened them, on average, otherwise they drown under a mass of unfinished work with no tangible results to show for it.

On reasons to close a project, he said “Either the project has succeeded enough to make the problem less urgent, or it has failed so completely and is intractable, so we need to shift resources to something more promising, or a crisis comes along and everything is suspended at least on a temporary basis.”

“Closing projects goes against the public service norm of ‘if you are doing a good job, you get more money and more people for the same thing.’ You need to be able to get used to the idea that when you succeed on a risk-based project you are done. If risks have been reduced by 90 per cent, then this one will no longer be the priority anymore, and we revert to ordinary baseline-level attention and monitoring.”

Building problem-based regulation into your work

Professor Sparrow said that the issues around integrating problem-based work with existing functional and process work, and with crisis response work that agencies faced from time-to-time, was extremely complicated and that there was no clear organisational theory providing guidance for regulators on these puzzles.

He said agencies needed to keep focusing on metrics and time deadlines and to ensure that analytical units were linked to operational ones.

“Roughly 20 per cent of your effort on any problem-oriented work will be analytical, and if you do a substantial amount of this you will soon discover you don’t have enough analysts. You will want more analysts and versatile analysts, and a system that makes them available to any project team,” he said.

But analysis needed to be tied to action otherwise: “you end up doing risk admiration – or risk classification, we might even do colour-coded heat maps, but if it never leads to action it is of no value to the public.”

Professor Sparrow said that while problem-based regulation was difficult, it was sometimes the only way to address problems that were not being adequately resolved by existing structures or processes.

“People who have engaged in this work tell me they have never done such intellectually challenging work, but also that they have never done such rewarding work,” he said.

“When we find a solution that is surgical and tailored, we may pop that problem like a balloon. When there’s a 50% or 80% reduction in a particular harm that we can measure, then the team can claim credit for a specific outcome; and that is a very rare luxury.”