Government Transformed 2.0 Episode Five: What is the institutional imperative – and why does it matter for evaluation in government (and everything else)?

Government Transformed

In the final episode of the Government Transformed 2.0 series, Richard Johnstone, Nicholas Gruen, and David Halpern discuss the critical imp

Key takeaways

• Nicholas Gruen defines the institutional imperative as the tendency for institutions to focus on self-preservation and process rather than maximizing the utility of capital.
• David Halpern notes that much government spending lacks rigorous testing, meaning the effectiveness of trillions in expenditures remains largely unknown.

Main topics

• Institutional Imperative
• Government Evaluation

Notable quotes

The institutional imperative had become the preoccupying force.

Conclusion

True government transformation requires institutionalizing curiosity a

Transcript preview

Speaker 1 (0:04) Welcome to Government Transform 2.0, a podcast series by Global Government Forum. My name is Richard Johnston. I'm the Executive Editor at Gold Government Forum. And in this podcast series, we're looking at the story of transformation and what lies ahead with Australian economist, Nicholas Gruen. Our conversations in this series reflect on the work of the Government 2.0 Task Force in Australia, which was chaired by Nicholas and which in 2010 set out a landmark vision for how the work of government could be transformed by technology. Throughout this series, we've looked at how much progress has been made in achieving that vision. unpicking the story of government transformation over the last decade and a half. This is the final episode in the series and will address evaluation in government. Speaker 1 (0:50) In this episode, Nicholas and I are again joined by David Halpern, who you may recall from episode two, and who is the former chief executive of the UK government's behavioural insights team, and now director of the Downing-Backcock Institute. Together in this episode, we discuss what works and what doesn't in government evaluation, what Nicholas calls the institutional imperative and how it affects government, and the need for evaluative thinking rather than just evaluative processes in the public sector. We also touch on how to actually achieve evidence-based policy through independent evaluations and the potential for bottom-up meritocratic processes to improve government services. To kick off the conversation, I asked Nicholas to define what the institutional imperative is and how he thinks it impacts the work of government. Speaker 2 (1:39) So the term institutional imperative I've pinched from Warren Buffett. And Warren Buffett talks about a very different institutional imperative because he's talking about the imperatives in business and particularly business administration of large companies. And what he says is that business managers find that they are driven to expand the business. And his argument is that as an investor, and a business manager, his role is not to expand a business, but to use capital as productively as possible. And those two things are not the same. In government, government is harder than business because it doesn't have a bottom line of profit. And so we have all the... mechanisms of central planning to try to understand what we're doing and to try to make sure that we're doing good things. Now, the example that I often use is that I was consulting to an organization that was trying to turn around what it did from paying benefits to veterans to trying to And that was kind of all that it did. And what they had been tasked to do is to start administering all of their services and the benefits to optimize the well-being of veterans, of veterans of Australia's military forces. They asked me to consult and I said, and then they said, would I write a report? And I said, no, I wouldn't write a report. I would participate in meetings if that was all right with them as a critical friend. And in one meeting, I remember I said nothing at all, but I was quite well regarded by these people. And I said nothing at all for an hour. And towards the end, they turned to me and they said, well, what did you, what do you think? How do you think this has gone? And I said, well. how many times did people mention the veterans' well-being framework? And they all said, you know, 20 times, 50 times, whatever. And how many times did someone mention the well-being of a veteran? And the answer was not at all. Now, that doesn't mean that those people didn't care about veterans, but what had happened is the institutional imperative had become the preoccupying force. And that was essentially crowding out. problem-solving, critical thinking about how a system can improve the well-being of veterans. It was getting itself completely preoccupied with what this framework was going to look like. And it wasn't framing those questions in terms of how do we maximize the benefits to veterans. It was framing it in terms of how do we get it to follow from the academic literature and so on and so forth. Speaker 1 (4:50) That's a really nice encapsulation of the problem that can exist if evaluation is done poorly. But to perhaps bring David in there, why do you think evaluation is important in government, David? And how can it be done well in a way that reflects the end objective of the service rather than the process of the evaluation? Speaker 3 (5:09) Well, because we're spending trillions and trillions of dollars and we don't know if it's making things any better. We can't normally even make the claim we know half of it works, we just don't know which half. For most areas, we haven't tested them. We haven't tested a million variations about what's going on or what is possible. And so to use that version of, are we eating our capital as productively as possible? Are we using our public servants, our expenditure in ways that are actually improving educational outcomes, health outcomes, etc.? It's normally presumed that's happening, but actually it may not be happening at all. The famous example, because what most people will know, is in medicine, where we now take it for granted that before a pill is widely used, someone has tested it normally in a randomized controlled trial to see does it in fact lead to improvement in the condition. There are also very famous examples now, not enough actually, but where things which were widely done for a while, when they were finally evaluated, were shown to be not only not effective, sometimes harmful. Famous examples of being scared straight, you know, scaring kids into don't re-offend, you know, whatever. But also medical examples such as treating people with steroids after they have a head injury, which turns out to be actually more likely for you to die rather than to improve you. So when you've got those examples, it should put a shudder through your heart. We think all these things we're doing with great good intent, are they actually having a good impact, which we hope. And the truth is, we just often don't know. It's a sort of dirty secret of government, I often think. Speaker 1 (6:41) And before we get into exploring how to improve that, are there examples where you think evaluation has worked? Are there examples you can point to when people talk to you about this and say, yeah, this did show a public policy intervention that worked and made a big difference? Speaker 3 (6:56) Absolutely. So there are high profile examples and sometimes requiring kids to stay at school longer. We can use a so-called discontinuity design being shown to actually. very big benefits. Sometimes they're very micro examples. I love a recent one driven by a police officer down in Kent in Britain who tried out during COVID whether recording of domestic violence could be done online instead of on person. And it turned out, of course, it was much faster to report the offence online, but it also led to significantly higher conviction rates. higher satisfaction rates and cost less. And it was developed and tested by an officer on the ground. I think one example also, sometimes government's role with respect to regulation is should we intervene in certain markets? A very powerful example in Britain a few years ago was testing whether people who took payday loans actually ended up better off or not. Can you use what's called a discontinuity design to compare those who just got them or just didn't? It was... incredibly conclusive and it showed that six months on the people who got the payday loans were much worse off and essentially set the case for saying we need to regulate much much harder essentially get rid of a lot of those types of providers so it can make a very very big difference um and i said what's it's what we so rarely use this astonishingly powerful tool Speaker 1 (8:16) nicholas the institutional imperative that you discussed at the start we'll come on to that in a minute but that kind of evaluation that david's setting out there There is value in that understanding of what interventions work and what don't, would you say? Speaker 2 (8:31) Yes, but I have a somewhat different view. I think the examples David's given are excellent examples. But my concern with evaluation was always about evaluative thinking. The way great companies become great. You won't find too many evaluations going on inside companies. You will find a great deal of evaluative thinking. So there's lots of A-B testing. Whenever they're doing anything, they're trying to understand what the impact of them doing things is. So to take David's example of schooling, more schooling is better. What he's said there, he's welcome to correct me if he thinks I'm misrepresenting this, but what he said is that on average, governments getting people to stay in school longer is a good idea and I don't have any quarrel with that at all. And it's important that we go and get that information and we have it and we use it in policy. But I would also be hoping a good system would, firstly, that won't be true for 100 % of the population. And so a good system will be working out where more schooling is better, where it's not better, why starting to hypothesize as to why, where we don't have more schooling, what might make it better or what alternatives might exist. So it's a large. problem-solving, forward-moving organization. And that's what makes a great company. And it's that kind of evaluative thinking that I think is ultimately what we want. And 15 years ago, I started speaking a lot about evaluation. And the result has been, I mean, not so much the result of me, but I was one of many voices saying we should do more evaluation. And I've rather... I say that rather less these days because what's happened is that, you know, I became sort of pin-up boy for the Australian Evaluation Society. I spoke at their annual dinner and so on, and now they're doing very well. And evaluators get good gigs doing evaluations. But I don't see a lot of evidence that government organizations, government agencies are... that there's a lot of evaluative thinking going on in government agencies. And that's the holy grail for me. That's when we could really start to move the dial. Speaker 3 (11:19) By the way, I think evaluation can sound dull and mechanical and whatever. I mean, at its best, what it is, is it's bolted together with a... a kind of really creative process where you're trying to