Evidence | Alex's Archives

Tag Archives | Evidence

Reinhart and Rogoff: replication and responsibility

admin / April 18, 2013 / Economics / Permalink

… the actions of economists today bear on the life chances of the world’s population far more substantially than do the actions of the members of most other professions.

George DeMartino

Replication is an activity that doesn’t attract enough attention, enough credit, or enough effort in the social sciences. But it is an activity that is getting a lot of attention at this precise moment. This has come courtesy of the exposure of both flaws and contestable methodological choices in Reinhart and Rogoff’s landmark study of public debt and economic growth.

The economic blogosphere has exploded with debate over the issue. But, just in case you’re not following it, here are the key points. Reinhart and Rogoff followed up their major historical work looking at debt and economic growth This time is different with a paper called Growth in a time of debt published in the American Economic Review in 2010. Their key result is that levels of public debt in excess of 90% of GDP are associated with lower rates of economic growth. Indeed, the mean annual growth rate they report, once debt crosses the 90% threshold, is negative.

This body of work is highly influential.

A quick search on Google Scholar will tell you that the NBER version of Growth in a time of debt has been cited 450 times, while This time is different has been cited over 2000 times since 2009. That is a lot of citations for social science publications: you’re doing pretty well once citations for a piece get into double figures within four years.

In a paper published this week, Thomas Herndon, Michael Ash, and Robert Pollin (HAP) of the University of Massachusetts, Amherst identify problems with the Reinhart and Rogoff result. Continue Reading →

2 Comments

Evidence or otherwise on Housing Benefit reform

admin / March 31, 2013 / Politics, Welfare State / Permalink

The mainstream media seem finally to have cottoned on to the fact that our welfare system is to undergo substantial change tomorrow. I mentioned a couple of months ago that the changes around the so-called bedroom tax were, belatedly, attracting broader media interest. And the media are connecting the deathly dry changes to the regulations to real life stories of hardship. They’ve also started to join up the dots to realise that it could well turn out that April is, indeed, the cruellest month.

Some of us have been banging on about the potentially negative implications of these changes for some months, if not years. It is good that they are now achieving some serious public profile. But it is a bit late to head off the chaos that could well follow their implementation.

What precisely will follow the raft of changes during April is a bit of a moot point. Will the prognostications of catastrophe be correct? Or will the Government’s much more sanguine view be borne out? Clearly, it is an issue of great significance.

It emerged as a key area of contention in the report on the impact of housing benefit reform published by the Public Accounts Committee last week. Continue Reading →

3 Comments

Research and the policy process

admin / February 13, 2013 / Politics / Permalink

[This text accompanies a presentation made to a SW Crucible event on 13^th February]

In this post I offer a perspective on academic research and the Whitehall policy process. It draws on interactions of various types in and around Whitehall going back to the start of the Major government. That includes having been involved in quite a few research projects and evaluations contracted by government departments and quangos.

One point to make at the outset is that different Whitehall departments have very different cultures, including their orientation towards research – they differ in where research sits in relation to other considerations as an input into policy thinking. And of course practice within Departments evolves over time. So I’m not sure there are many truly general lessons to be drawn.

So this is just one perspective, albeit a perspective that is also informed by the literature on research utilization. There are no doubt others with different experiences and different insights to share.

I want to make some comments under six headings. Continue Reading →

2 Comments

Appraising health reform

admin / February 25, 2012 / Economics, Politics / Permalink

Blogging is a fantastic medium for providing a brief statement of your views. Or for building an argument involving a small number of points. Or, perhaps, for giving a high level summary of a more complex argument.

But it’s not a great medium through which to appraise complex arguments or carefully weigh the evidence. Where a blog draws explicitly on evidence it tends to draw on one or two studies to illustrate its point. For some purposes that works just fine. But for others it can be misleading. It can give the impression that there is a robust evidence base to back up the points being made, whereas in fact the evidence is being cherry-picked. The weight of evidence may lie with the other side of the argument. Or, more likely, the evidence does not offer many simple messages.

A couple of weeks ago I made this point in the comment thread on a blog post about competition and choice in public service reform. It wasn’t entirely warmly received by some other commenters. Continue Reading →

0 Comments

Is nudging enough?

shodanalexm / July 31, 2011 / Economics, Politics / Permalink

Rene Kinzett, one of my fellow contributors over at Dale & Co, posted Nudging or Nannying last weekend. The argument was perhaps a little cryptic, but the point was that relying on the subtlety of trying to “nudge” behaviour in the right direction is not an adequate policy response to certain types of problem. His example of treating rickets among young women who for cultural reasons do not have enough exposure to sunlight is an interesting one. He referred to minimum alcohol pricing, a policy being introduced in Scotland, and banning smoking in cars carrying children, a policy being introduced in Wales, as more conventional regulatory policy that will help those on the edges of society, when nudges are judged inadequate.

A couple of commenters took Kinzett to task for this argument. One made the point that nudge theory is being preferred for developing policy at the moment because it has been shown to be more effective than traditional regulatory approaches. It was also arguing that no one is suggesting that nudge can be used in isolation and should be combined with other mechanisms to deliver better social outcomes.

It seems to me that these comments miss the mark in at least a couple of ways. Continue Reading →

0 Comments

Policy and evidence – the homelessness episode part II

shodanalexm / July 9, 2011 / Housing, Politics / Permalink

Last weekend the Observer ran with the story on welfare reform and homelessness. A senior civil servant at CLG had written to the Prime Minister warning that the Government’s proposed welfare reforms could result in – among other negative consequences – 40,000 additional homeless households (as I discussed here). This raised questions about a Government willing to ignore its own evidence and the accuracy or otherwise of Ministerial statements to Parliament. Subsequently Grant Shapps has dismissed the 40,000 figure because it was based upon “out of date” information and didn’t relate to current government policy. He also announced a £20m fund for integrating homelessness prevention services, rolling out a model that has worked in London to the rest of the country.

A passage in yesterday’s blog by the Guardian HousingNetwork Editor caught my eye: Continue Reading →

0 Comments

Dispatching rogue landlords

shodanalexm / July 4, 2011 / Housing / Permalink

Tonight’s C4 Dispatches programme provided some very clear evidence regarding poor standards of accommodation and management in the private rented sector. It is linked to the Shelter campaign to Evict Rogue Landlords. While the individual underhand practices deployed by landlords are very unpleasant, the impact of the programme will be mitigated by the problem that all research in this sector faces – that it is hard to quantify the scale of the problem. If one problem is that no one prosecutes rogue landlords, for example, then the statistics appear to show that unlawful behaviour by landlords isn’t a big problem. The logic is faulty – absence of evidence isn’t evidence of absence – but convenient for those who have no wish to act. The majority of private tenants are happy with their current landlord. But that tells us nothing about how many tenancies they’ve moved on from because of poor treatment by a landlord.

Grant Shapps was interviewed briefly in the programme. His contribution had two key elements. First, he argued that the national registration scheme for private landlords proposed by the previous Labour government ran the risk of becoming a bureaucratic exercise and so was dropped. In fact, his argument here was a little less than clear. But the net result is that this type of regulatory scheme appears to be off the agenda. Second, he argued that there are lots of local authority powers and regulations already in existence to deal with problem private landlords.

This second point is correct but almost entirely irrelevant. Continue Reading →

0 Comments

Think Tanks and the policy process: Right, wrong and possibly both at the same time

shodanalexm / May 3, 2011 / Politics / Permalink

I’m currently halfway through The Conservative Party and Social Policy, edited by Hugh Bochel. The contributors chart recent developments in the policy agenda of the dominant Coalition partner. The book does a good job of conveying the protean nature of Conservative thought. Of course, one of the dangers of such an enterprise is that when assessing whether Cameron’s Conservativism represents continuity or change – traditional, neo-liberal or progressive – you’re driven to the conclusion that it’s too early to say. And that is precisely what the book does.

As well as having plenty to say on the substance of policy, the book also has something to say about the policy process, although that isn’t really central to its purpose. Continue Reading →

2 Comments

The ethics of the case for public sector reform

shodanalexm / February 24, 2011 / Politics, Welfare State / Permalink

[Originally posted on Liberal Democrat Voice, 24/02/11]

David Cameron’s article on public service reform in the Telegraph was the opening shot in what could be a significant battle both within the Coalition and across the House. The case presented raises at least three important ethical issues.

First, the way in which evidence is being used to justify these proposals is deeply suspect. Mr Cameron states that publicly providing bureaucratic and target-driven services might be worth supporting if they delivered quality services: “but the evidence shows otherwise. Whether it’s cancer survival rate, school results or crime, for too long we’ve been slipping against comparable countries“. These are very partial readings of the data. These claims in relation to both health and education have already been challenged and debunked a number of times over recent weeks. Yet they continue to be advanced as a justification for change. Continue Reading →

0 Comments