The mainstream media seem finally to have cottoned on to the fact that our welfare system is to undergo substantial change tomorrow. I mentioned a couple of months ago that the changes around the so-called bedroom tax were, belatedly, attracting broader media interest. And the media are connecting the deathly dry changes to the regulations to real life stories of hardship. They’ve also started to join up the dots to realise that it could well turn out that April is, indeed, the cruellest month.

Some of us have been banging on about the potentially negative implications of these changes for some months, if not years. It is good that they are now achieving some serious public profile. But it is a bit late to head off the chaos that could well follow their implementation.

What precisely will follow the raft of changes during April is a bit of a moot point. Will the prognostications of catastrophe be correct? Or will the Government’s much more sanguine view be borne out? Clearly, it is an issue of great significance.

It emerged as a key area of contention in the report on the impact of housing benefit reform published by the Public Accounts Committee last week.

The PAC report provides a good overview of some of the potential risks associated with these policy reforms. Some of the most telling points the committee makes relate to evidence and policy. When the DWP produced Impact Assessments on the reforms to housing benefit, back in 2011, they declined to make a comprehensive assessment of the impacts. This was on the basis of a two limbed argument. First, we don’t know the behavioural response of households affected by the changes. Second, the raft of changes being made to the welfare system will have complex cross-effects. So there is no way to assess the overall impact in advance.

This is the position that the DWP has maintained until at least the end of 2012. It gives them one sort of defence in the face of probing. They can claim they just don’t know what’s going to happen when the policy is implemented. And nor do you.

In one sense this is true. But it is a rather unsatisfactory defence. The idea that it is not possible to undertake any sort of scenario modelling is barely credible. Curiosity about the potential impact of these changes, and attempting to mitigate the risks of negative effects as far as possible, would, you might have thought, been central to responsible policy making. Apparently not.

Instead the Department is claiming that it is adopting a ‘wait and see’ approach. It will observe how the policy evolves following implementation and respond if negative consequences become apparent. It could not, however, give the PAC any very concrete details on what negative consequences would lead to a change of course, what the thresholds for acting might be, or how and how quickly it was able to respond. Again, you might have assumed that thinking these things through in detail would be central to responsible policy making. Apparently not.

Of course this position stands in stark contrast to the sorts of debates we have over the macroeconomy, where analysts within and without government don’t decline to make forecasts on the basis that it is a bit difficult. From another perspective you would have thought that it would make it difficult to make overarching claims about how much money the policy changes would save the government. But, strangely enough, it doesn’t stop that happening.

The most interesting aspect of the PAC report is probably not the report itself but the transcripts of the oral evidence sessions. The joust between the PAC and the DWP civil servants is particularly intriguing. It is a bit of an unfair fight. The Committee are not experts in this field – as is quite evident from some of the questions that are asked – whereas the Permanent Secretary of the Department for Work and Pensions is in full-on Sir Humphrey mode. The Committee makes some valiant attempts to pin him down, and manages to extract some admissions from him. But I think the Permanent Secretary gets the better of the encounter overall.

There are some notable omissions from the discussion. The oral evidence sessions were just before Christmas. It was only after Christmas that the public consciousness was penetrated by the point that two thirds of those affected by the so-called bedroom tax have a family member with a disability and that many of the rooms targeted by the policy are not “spare” in any conventional sense. It appears that, unfortunately, the Committee did not probe the Permanent Secretary on this issue.

Throughout the joust there were claims and counter-claims regarding evidence of policy effects. Committee members were able to provide examples of individual households negatively affected by the policy. The Permanent Secretary was able to dismiss these as anecdotes, and therefore inadmissible. He repeatedly returned to the idea that others should be providing more systematic – that is, non-anecdotal – evidence of problems before it was sensible for the DWP to take the issue seriously. That strikes me as an interesting approach to laying the burden of proof. I rather think the obligation should be on the DWP to check systematically whether and where problems were already emerging. And, of course, the Permanent Secretary isn’t averse to letting his evidence-based policymaking halo slip occasionally. He introduces the odd anecdote to counter a point made by a member of the Committee. It goes without saying that these aren’t referred to as anecdotes. Rather they are styled “random” examples. The implication being, of course, that there are plenty more where that came from. Whether or not there are plenty more goes unexplored.

One of the most interesting passages illustrating the way in which evidence is invoked in the debate relates to whether statutory homelessness is rising as a result of reforms already implemented. The issue under consideration is what we can read into the statistics that are available. The Permanent Secretary takes an interesting approach:

Q86 Chair: So is Shelter’s figure that homelessness is up a third since the election wrong?

Permanent Secretary: Homeless acceptances at the point of the election were around 10,000, and they are now around 13,000 or 14,000, so up—

Q87 Chair: So it is about a third.

Permanent Secretary: Okay, but before you look too smug—

Chair: No, I’m not getting smug; I just want to get underneath this.

Permanent Secretary: The main reasons why those have gone up are to do with the fact that relatives and friends are no longer willing to provide accommodation, or there has been a relationship breakdown with a partner. When you look at the people who actually say that the reason they are homeless is the end of a shorthold tenancy, they have gone up by 1,000 since the election—from 1,700 to 2,800. For the 1,000 people in that situation, that is bad news, but in the grand scheme of things, where 775,000 people are affected by the LHA contraction, I would be perfectly happy to argue that actually this is a reasonable set of reforms. We need to be very careful that local authorities continue to manage this outcome well, but the evidence is they are managing it well.

The point is clear. Only those who are homeless as a result of the end of a shorthold tenancy should be attributed to welfare policy change. We might say that looks plausible.

But hang on. One of the other changes the Coalition has brought in, and has been implementing since 2012, is to crank up the non-dependent deductions associated with housing benefit. This will, for example, make it financially more difficult for adult children to stay in the family home. If that is happening on any scale how would that be recorded in the statistics? It would most likely turn up under “Relatives and friends are no longer willing to provide accommodation”.

Should all such cases be attributed to this policy change? Certainly not. There are going to be other reasons to find oneself in this situation, including a more general rise in the level of financial and interpersonal tensions as a result of households finding themselves in more difficult financial circumstances. We don’t really know the detail of what is going on. But to claim that none of these cases can be attributed to the Government’s welfare reforms is even less plausible.

This is the sort of detailed change in the rules that the Committee probably isn’t well equipped to probe.

Later in the session there is a similar, but slightly less clear cut, exchange about turnover in the rented sector. Committee members were concerned that the level of churn in the private rented sector and households having to move out of inner London boroughs because rents locally are no longer affordable.

At one point in the discussion the Permanent Secretary offered the following:

Let me just bring you back to housing benefit, then. Private rented in Hackney, March 2011, before we started the reforms: 10,070 claimants; August 2012, the latest data: 10,360. That is a 2.8% increase, so it doesn’t feel to me as if some seismic event has occurred in Hackney.

Let’s not dispute the statistics. But what do they mean? Can focussing on a single borough tell us anything about the idea that people are being priced out of inner London? Clearly not. You’d need to look at the picture more holistically and look at mobility across the city. We know already that rates of new claimants are increasing more in outer London than inner London. That is suggestive of outward mobility. The Hackney statistics are commensurate with that bigger picture if we assume that there are a lot of additional new claimants in the borough, as a result of current economic challenges.

Is that the best way to explain those Hackney statistics? I’m not claiming that it is. I don’t know. But I’m not sure the DWP know either, unless they’ve got well calibrated stock-flow model that they’re not sharing with us.

I don’t think we need to labour the point. We are in highly contested territory here. And the limited evidence available is open to a number of different interpretations. It is perhaps not surprising that the DWP reach for the interpretation that is most sympathetic to the Government’s policy agenda. But it isn’t the only interpretation possible.

Evidence-based policy may be an improvement on policy made with no reference to evidence.

But evidence can be deployed in all sorts of ways in policy discussions, and not all of them are perhaps what they seem.

Image: © Dreaming Andy – Fotolia.com