Critic Says Call for Evidence-Based Policy Fueled by Naivety

Print Share on LinkedIn More


June 11, 2012; Source: The Guardian

British blogger David Walker recently responded to a proposal by Geoff Mulgan, the director of the U.K.’s National Endowment for Science, Technology and the Arts (NESTA). Mulgan thinks that the evidence-based work of the national Institute for Health and Clinical Excellence (NICE), in which pharmaceutical and other treatment protocols are subjected to rigorous testing and approval, can be replicated more broadly in all of social policy.

Mulgan’s proposal seems like it is a bit carried away with the compelling message of the evidence-based acolytes. Walker suggests that the Mulgan/NICE proposal doesn’t fit with many proposed social policies (he calls the government’s proposals to reform health care “the institutionalization of anarchy,” for example). Making NICE work in the U.S. seems just as preposterous, particularly when the policymaking body to be convinced is the gridlocked Congress.

Walker makes clear why the drug testing of NICE might not have the same effect when applied to social programs: “Drugs have effects: they prevent infection, they control conditions. Social policies are usually much less specific.” Walker adds, “Much social policy is about reshaping institutions and changing governance, and these phenomena are much harder to capture let alone measure than testing a cancer drug or even a discrete intervention such as a smoking cessation programme.”

{loadmodule mod_banners,Newswire Subscription Plea}

Walker is basically calling the evidence-basers naïve about real life politics and government decision-making. He reminds his readers, “Ideology cuts a swath across public management. And partisanship more often wants confirmation than challenge. To believe policy can be more evidence based, you need a strong dash of piety and naivety.”

That doesn’t matter to the movement’s true believers. The longshoreman philosopher Eric Hoffer devised the concept of the “true believer,” someone who holds a “fanatical faith” in an idea, someone “without wonder or hesitation.” Despite the obvious impracticalities of Mulgan’s NICE-like proposal for social policy, Britain’s New Philanthropy Capital quickly convened an invitation-only discussion of the idea. An NPC report on the gathering noted, “There was palpable enthusiasm in the room for the idea, with the debate focusing less on whether there should be one and more on what its role would be and to what extent the context was different from that of healthcare.”

Expect the true believers to try something similar here. Is this where the evidence-based Moonies are heading? We’re not talking about those who know how to use outcome measures in a rational way—that is, without devoting ever more money to the search for evidence and less and less to the real work of creating that evidence. We’re talking about the acolytes who have turned the search for the holy grail of evidence into a cult. —Rick Cohen

  • Geoff Mulgan

    It’s always a good idea to do a minute or two’s research before writing commentary. If Rick Cohen had done so he would have seen that I was not proposing to replicate NICE (whcih anyway doesn”t look at policy), and that I’m not a fetishist about evidence. Having worked in the centre of governments around the world over many years, I’m also not naive about the interaction of politics and evidence. But I, like many others, think that there is a lot to be gained from making evidence more visible, more usable and more used – and that’s the guiding spirit of the ‘Alliance for Useful Evidence’. Policymakers and NGOs shouldnt slavishly follow evidence – not least because in so many fields the evidence is so partial. But they don’t have any excuse for being unaware of what is known. I’d urge readers to look at what David Walker actually wrote – which was a lot more nuanced than this report of it – and at the details of our proposals.

  • Ivan Avery Frey

    John Ralston Saul wrote a book on this. It’s called the “Voltaire’s Bastards: The Dictatorship of Reason in the West”


  • Joe Diament

    I’m a well educated and reasonably bright guy, but don’t really understand your rant. Just what is it about being or doing things according to the best scientific evidence that you don’t like? Clearly if we only do what current evidence shows and don;t seek new evidence we would be closing off possibilities that have yet to be developed, so there must be science (social & hard) to continue the evolution of evidence, but to argue against a general policy of using evidence in the realm of social policy seems odd. Maybe you’re confusing policy and politics. As evidenced by the current Congress and many politicians, they clearly don;t want to let truth (evidence) get in the way of their story (ideology).

  • Robert

    I found this to be an awful article that paints with an awfully broad brush. I [I]think[/I] I understand what prompts it, and if I’m right, it doesn’t necessarily come from a terrible place. As a career non-profiteer, I’ve seen the frustration about evidence and I’ve seen the light on focusing all of our efforts on evidence based practice and policy.

    Many non-profits suffer from being in the difficult place of measuring outcomes. For example, prevention outcomes are hard to quantify because it’s difficult to measure something that doesn’t happen (especially if it’s something that people are unwilling to admit in the first place). In cases like that, skilled researchers have found a solution in antecedent or consequential measures. But it takes effort.

    That’s where so many other non-profits turn defensive. In the absence of easy and meaningful measures (or when subjected to funder requirements to measure something meaningless–the equivalent of busy work) we often get frustrated over what seems like a distraction from our mission.

    But the fact is, we cannot afford to continue investing our precious resources and limited energy in things that feel good but have no impact. I know hundreds upon hundreds of people who mean the very best in their work to end poverty, reduce violence, develop our communities or promote health, yet they have no evidence that what they do works.

    Should we teach people coming out of prison better penmanship so they have increased self-esteem or should we give cognitive behavioral therapy to reduce their recidivism rate? Both have been funded, but guess which one actually works? We also shouldn’t be surprised that the feel-good answer to the above question actually makes people worse.

    How about awareness raising? It doesn’t hurt to print a bunch of bumper stickers, buy billboards and produce PSA’s, right? RIGHT? It turns out, a lot of those PSA’s are so poorly targeted and produced they actually perpetuate the thing they seek to end. People (young ones in particular) are incredibly savvy and it’s not as easy as “getting the message out there.”

    Now that doesn’t mean that evidence based practice and policy is the holy grail because, well, there are some things we don’t have an evidence base for [I]yet[/I]. That’s the reason for pilot projects and test sites. But there’s no excuse for going all-in on interventions or programs without having a well-thought plan for research and measurement.

    Even worse, there’s no excuse for promoting obstinance to non-profits on the issue. It is reckless and likely does more to harm the people, communities, animals and issues we care about than any funding cut could ever do. It’s an easy way to get cheap applause from lazy organizations but no way to change the world for the better.