The relationship between politicians and civil servants is back in the spotlight. Janan Ganesh in the FT, for example, has argued that civil servants need to be brought to heel more effectively by their political masters. Cries of ‘politicisation’, whenever the prized independence of the civil service is threatened, should be recognized for what they are – an attempt by civil servants to maintain their capacity to undermine democracy by frustrating the will of elected governments.

Giles Wilkes, in contrast, argues that the machinations of a Mandarinate intent on frustrating the will of the people are much less of an issue than this suggests. There are many reasons why Ministers’ pet ideas do not translate into policy. He argues that the independent testing of the feasibility and desirability of proposals delivers better policy. It is rushed policy, that civil servants fail to question or refine sufficiently, that often provides the most egregious examples of policy failure:

… ministers and their advisers frequently do not understand the implications of their policy spasms. Such spasms often stem from a pitifully thin evidence base, and are only subject to scrutiny from a generally tame bunch of close commentator-friends, who will naturally be told, and repeat back to their readers, that the policy idea is sheer, radical genius.

No it isn’t.

I find myself with greater sympathy for the view Giles sets out. I should, I guess, state for the record that I have spent a bit of time in and around Whitehall, including some time formally on the books of the civil service. So that may mean I’m biased. But I’ve always engaged with Whitehall from a distance as a result of the other roles I’ve held simultaneously. No chance of going native.

My sympathy for the Wilkes view arises more from a recognition of the fact that good policy making is hard. Rushed policy is rarely good policy. And good policymaking seems in short supply at the moment. This Coalition continues to provide us with a stream of bad policy to reflect on.

The poster boys for bad policymaking are the DWP. You only have to list the initiatives that have been attempted by Iain Duncan Smith’s mob – WCA, PIP, Univeral Jobmatch, Universal Credit, the Work Programme, the Troubled Families programme, and more – to see a procession of ill-thought through initiatives. In some cases they may not be wrong in principle, but they’ve surely turned out to be wrong in practice. Part of the problem here is that DWP lost a chunk of its institutional capacity early on in the Coalition period and then had to rerecruit a lot of new staff. It is precisely where politicians are able to push ahead without effective challenge that we end up with a bunch of bodged initiatives that have between them caused huge amounts of unnecessary suffering and wasted vast amounts of taxpayers’ money.

It is not possible to review the failings of each of these policies. But I am reminded of the recent NAO report on the Work Programme. While this document attracted some media attention a few weeks ago, it was relatively low profile and its lessons for policymaker justify further attention.

The report is intriguing inasmuch as it hints at a characteristic DWP pathology under IDS – the desire to rely on its own idiosyncratic perspective to represent the world: “The Department has not cleared the findings in this report on the grounds that they do not reflect the Department’s view of the relevant facts”.

But it is more intriguing in the key lessons learnt it identifies. Basically, the report says is that the Work Programme was rushed into implementation, it was over-ambitious, and the contracts that it relies upon as part of the payment by results scheme were not very well designed or monitored. As a consequence, among other things, the government was not as well insulated against poor performance as anticipated and resources were not directed towards the most vulnerable service users as was desirable. Overall, the level of performance in job placement under the Work Programme has now achieved levels similar to those of previous programmes, after a poor start. It may well have managed to achieve this level of performance on a lower budget. But the report doesn’t comment on whether the revenue savings outweigh, or are likely ever to outweigh, the costs of transition to the new scheme.

The issue I have with these findings is not that I disagree with them. Rather it is with the notion that they should be designated as “lessons learnt”. Because we knew pretty much all this stuff before the Work Programme started. Indeed, plenty of commentators warned about the risks and problems associated with the approach before the policy hurried to implementation. So, in that sense, while it may be the case that Ministers and civil servants in DWP have engaged in some learning by doing, I’m not sure the sum of human knowledge on these topics has been significantly enhanced. If policymakers had spent a bit more time on the initial design, considering in more detail some of the lessons from the vast literature on contracting, performance monitoring and the behavioural consequences of incentivisation then some of the problems they’ve subsequently learned about for themselves may have been avoided.

It strikes me that the Work Programme is one of those areas in which a greater focus on learning at the design stage, more realism about appropriate implementation timescales, and greater organisational capacity to think through implementation issues and project manage substantial organisational change would have increased the chances of success considerably.

That would tend to suggest that greater scope for the civil service to manage the delivery process at a speed driven by the chances of implementation success, rather than political expediency, would be desirable. On the other hand, the Work Programme experience seems to me to hint at a question mark over the whole system. It is an issue that has been noted many times previously. That is, Westminster and Whitehall places great emphasis upon clever people coming up with ideas and spending time making policy, but the grubby business of implementation is not really accorded the same status. People who understand how difficult it is to effect large scale organisational change successfully are either in short supply or their views are not weighted sufficiently heavily in the decision-making process.

So, for me, part of the story is not so much Ministers versus Civil Servants but “thinkers” versus “doers” – and the need for doers to bring an air of reality to proposals for policy change.