[Originally posted at The Conversation, 22/08/13]
With independent journalism increasingly under threat, will academics be the next set of critical voices to be targeted? A report calling for research and evidence to have a reduced role in public policy, issued yesterday by a right-wing think-tank, suggests this process is already under way.
These criticisms come after successive governments have sought to encourage academics to leave their ivory towers and influence the wider world. Right now, many academics around the country are knee-deep in final preparations for the next round of research funding. This time around we’ve the additional task of demonstrating how our research has had an impact on the world beyond the academy. In many fields, including my own, that impact is on policy.
At the same time, the government is in the process of establishing a network of “what works” evidence centres for social policy. This government’s commitment to evidence-based policy is perhaps less secure than its precedessor’s, but the idea that research evidence should play a role in policy making remains.
Since the turn of the millennium this greater focus upon evidence has been welcomed by many as signalling a move towards better quality policy, although there has been a lively debate in the academic literature over the feasibility and desirability of evidence-based policy making.
The intervention by the free-market Institute of Economic Affairs (IEA) is rather more sceptical. Their report, written by Jamie Whyte, is titled “Quack Policy: Abusing science in the cause of paternalism”. The document examines four policy areas with a view to demonstrating that scientific input into policy should be given less – or no – credence because it fails to meet the standards of science.
One can only imagine that the fact the four areas chosen – minimum alcohol pricing, passive smoking, climate change, happiness research – line up pretty closely with well-established libertarian obsessions is entirely coincidental because, as the foreword assures us, the IEA has no corporate view.
As so often with IEA publications, Quack Policy contains some perfectly reasonable points in among the more problematic arguments. In this case it has some sensible general points to make about the correct way to do a cost-benefit analysis, scientific certainty and robustness, credulity, and the dangers of lay audiences accepting arguments from authority.
But there is much that is highly questionable about the argument overall. One of Whyte’s concerns, for instance, is “expertise slippage”: the danger of researchers well-qualified in one discipline offering relatively untutored and ill-informed observations in another.
That is of course a real danger. And yet it is equally a danger for Quack Policy. Whyte is a philosopher who knows some welfare economics. But that hardly gives him the resources to dismiss the evidence base across a wide range of research areas, examined relatively superficially.
It is easy to pick further holes in the argument. Few of Whyte’s own counter arguments are evidence-based in the way he demands of “science”, so presumably they should be dismissed. The report seems to switch position in mid-argument on whether preferences are fixed or endogenous. The microeconomics deployed either ignores or rejects most of what we have learned from behavioural economics about market or decision-making imperfections, even though some of these core concepts are now well-established empirically.
Whyte sees pre-twentieth century physics as the template for dividing the world into science and non-science, and finds almost everything else falls short. But in the process he seems to be working with a relatively outmoded understanding of how science operates in practice and how knowledge accumulates.
Discrediting the alternative
More important than the content of this publication is its purpose. And we might want to think about that in its broader context.
We have a government that has intentionally opened up policy in a range of areas to detailed influence from industry bodies, while seeking to maintain minimal transparency. It is also seeking to constrain access to legal mechanisms for reviewing its decisions.
We are currently exploring the aftermath of the Miranda incident, which looks very like something geared to have a chilling effect on investigative journalism. It poses searching questions about the freedom of the press and the accountability of the surveillance state. There is more than a suggestion that concerted efforts to insulate political and economic elites from criticism are underway.
In this context, Quack Policy might be interpreted as the beginning of an attempt to discredit another alternative voice in society: ignore what emerges from the academy because most of it isn’t real science just special pleading by self-aggrandising and paternalist busybodies – or “list utilitarians”, as Whyte describes them.
I suspect the IEA wouldn’t see it that way. Whyte argues:
Someone who has gone through the intellectual process required to get a PhD in … some of the more politicised branches of sociology … will probably have reduced his chances of uttering truths: he will have corrupted his mind.
My comments would probably be dismissed as being of the same stripe. That would be entirely consistent: both with his thesis and mine.
Alex Marsh has in the past received funding to undertake research and provide expert advice from the Department for Communities and Local Government, and its predecessors, and from the Law Commission.
This article was originally published at The Conversation.
Read the original article.
Image: © julien tromeur – Fotolia.com
Jamie Whyte here. You claim that I succumb to the very fallacy of “expertise slippage” that I complain about. Please tell me where I ask readers to accept my scientific opinion on account of my philosophical expertise? If you cannot, please explain why you said I did.
Thanks for your comment/question. Sorry that I’ve not responded sooner, I’ve been out and about today.
The post at The Conversation was working to a tight word limit so there was no space to elaborate on any of the points. If I get time I might well write another post giving a fuller account of my view.
In terms of the point about expertise slippage, your criticisms of the various literatures are not restricted to philosophical concerns (depending, of course, on where you want to place the limits of philosophy). Some of your criticisms are directed at the current state of an empirical literature. The relevant literatures are substantial, but you typically refer to only one or two studies. It is no doubt challenging to anything but that, given the breadth of subject coverage in your report. But it doesn’t really constitute a robust assessment of the state of each debate.
To take the example of passive smoking, you primarily discuss two pieces of academic work – one from 20 years ago. I presume that a substantial literature has accumulated since then and to arrive at an informed assessment of the state of empirical research in the field would be a substantial task.
Whether or not you ask them to do so, the unwary reader of your chapter might well gain the impression that there is a general conspiracy within the academic community to suppress the findings of tobacco-funded research, and that the case against passive smoking has not been made out. Yet it would be incautious to form such an impression on the basis of such brief acquaintance with the debates.
Some of your criticisms are not empirically based but assertions – for example the argument that scientists are self-aggrandizing would seem to be a translation of straight down the line rational choice theory into this context. But the argument is a priori. You don’t offer systematic evidence in its favour. I can’t offer systematic evidence against it. But I can say that I know plenty of scientists that don’t fit the description. I would also say that the economics literature is increasingly rediscovering the richer range of motivations that the classical economists worked with, moving beyond the simplistic view of homo economicus that have been the flawed foundations of so many late twentieth century models. That is a welcome development. And it suggests to me that, from an analytical perspective, now would be an odd time to go big on applying the home economicus model to academia.
As I say, I might get to the position where I can write a fuller response to your report. There is plenty in it I didn’t disagree with, alongside plenty that I did.
I’ll do this in bits. First, I do not ask readers to accept what is in the book on the basis of my authority. I provide arguments. The point about “expertise slippage” in the chapter on lay people’s deference to scientists. Specifically, it concerns the correct identification of experts. Often, for example, medical doctors are presented as authorities on matters that primarily concern welfare economics, in a context where deference is being asked for. I do not repeat that mistake in my book, because I do not seek deference.
On the superficiality of my analysis … In the Alcohol chapter I use the Sheffield University report on the matter which Alcohol Concern, the BMA and the government take as the basis for their proposed policy. No more thorough analysis has been done. If I find major flaws in its reasoning, is that not a satisfactory way of proceeding in a book such as mine? Where does my critique fail? Where would attending to other research show my error? You really need to say if you are going to make these claims.
Similarly, please say how I misrepresented the passive smoking science or any of the other research discussed in the book, and how correcting this misrepresentation would undermine my argument. Since I ACCEPT the findings that say passive smoking increases the chance of cancer by 33%, it is hard to see how my argument could be undermined by a better understanding of the science?
The claim that scientists are self-aggrandizing (not a term I use) is also in the chapter on deference by laymen. It is of no relevance to debates about the four policies discussed. I am there explaining why laymen should often discount the degree of confidence announced by scientists. It is true that I provided no evidence that people are generally inclined to act in their own interests or that scientists are just like the rest of us in this regard. But do you honestly doubt that I easily could? Would it not have been the most absurd pedantry to have bored readers with?
in your original blog post and in your response you say that my arguments failed to take account of the advances made by behavioural economics over recent years. But again you do not say why. How is my critique of any of the four policies covered in the book undermined by behavioural economics? And why do you say I sometimes say preferences are fixed (which neither I nor anyone else has ever said) and sometimes that they are endogenous? (It is strange, BTW, to contrast fixed and endogenous.)
Finally, the strangest thing about your blog post was the idea that my book is an attempt to undermine the independence of academics. How is criticizing shoddy thinking an attempt to undermine academic independence?
It’s perhaps not helpful to be conducting this conversation in different places. Nor, I suspect, are we going to agree on several points.
I have tried to address some of your points elsewhere.
I wasn’t saying your argument was superficial, I was saying your engagement with the literature was relatively superficial. A broader engagement wouldn’t necessarily have undermined your argument, it may have strengthened it.
I also think your argument has considerable rhetorical sophistication. And that that is doing more work than you imply. But that is difficult to demonstrate without undertaking a discourse analysis.
I agree that the Sheffield work on minimum alcohol pricing isn’t a great basis for assessing the wisdom of the policy, because it is partial. And I agree that it is important to look at the economics of the issue alongside health and other considerations. But the economic considerations may not outweigh the other considerations.
I don’t think it would be pedantry to provide systematic evidence in support of a key assertion.
I didn’t say that your book was an attempt to undermine the independence of academics. I said, read in the broader context, its argument could be interpreted as an attempt to discredit them. They can be as independent as they like, but they shouldn’t be listened to. I’m not saying that is your intention. As I have written elsewhere, that is a use to which a representation of your argument could be put.
“Could be interpreted”. Well yes … but that is hardly relevant since anything could be interpreted in any way at all.
And, yes, I am saying that scientists should not be listened to, JUST ON ACCOUNT OF BEING SCIENTISTS. Their credibility is earned not by their employment at a university or wearing a white lab coat but by their standing in a field of science with a track-record of successful prediction. And, ultimately, even that is quite irrelevant if someone — anyone — can show that on the matter in question, the scientist’s thinking has gone wrong. There is, ultimately, no place for authority in science.
The idea that people act in their own interest is an assumption I do indeed make (and, by interests, I only mean that they act according to their preferences, which need not be derived from commercial desires or desires of any other particular kind.) But that “key assumption” has surely not been undermined even by behavioural economics. Do I really need to provide evidence of it? Or that scientists, being people, are inclined to all the vanities that the rest of us are. It is precisely because scientists agree with me about this that science is conducted by repeatable experiment rather than on a system of taking scientists’ word for things. Again, are you seriously denying any of thi?.
Hello again Jamie
Thanks for your further comments.
I don’t think I agree with you on the “could be interpreted” point. We are not entirely free to interpret texts however we wish, in an Humpty Dumpty Through the Looking Glass kind of way.
For example, your book from its title – “Quack policy: abusing science in the cause of paternalism” – onward cannot credibly be interpreted as a ringing endorsement of evidence-based policy making as it has been practised so far.
Whether it is arguing that EBPM is even possible is a bit more ambiguous.
I read your argument as saying that if the evidence produced were to meet your criteria for acceptability then it would be admissible as one input into the policy process, but that if it falls short of that standard then it shouldn’t have influence over policy.
I think it would be possible to read your argument differently. If a reader placed the emphasis slightly differently then they might conclude that you are saying that the evidence can never be sufficiently compelling to deliver the certainty that justifies action; self-interested scientists produce evidence that furthers their own interests and their view of the world, which is different and less legitimate than the view of ‘the public’. So scientists have no specific insight or valid claim to an input into policy. That is an interpretation of your book that is plausible – even if it isn’t the one you intended – and it is an interpretation which might serve a purpose.
I agree that arguments from authority are almost always bad arguments. They seem particularly prevalent in economics, which is one of its great weaknesses (Keynes/Hayek/Friedman said X, Y, Z so it must be a good thing etc). However, outside of physics and astronomy I’m not sure a track record of successful prediction looms quite as large in assessments of expertise as you would wish it to, although I haven’t studied the matter systematically. If you are an evolutionary biologist or an earthquake scientist you can say all sorts of useful things about how the world has worked/works without spending a lot of time making very specific point predictions about what is going to happen when.
I think the concept of self-interest is one that, generally speaking, is used rather vaguely. It has been interpreted narrowly and broadly in economics. It has been narrowed to a focus on own consumption in order to allow for mathematical tractability. And it has then been widened when it became apparent that trying to explain everything in terms of satisfaction of preferences for own-consumption wasn’t really adequate. So we get others’ utility/consumption/wellbeing or a range of other factors entering utility functions. That is a move that retains the integrity of the formal theoretical framework, while largely emptying it of predictive content.
I don’t deny “self-interest” is a factor in motivation, although it depends what is meant by that. But I do deny that giving self-interest theoretical primacy in explaining motivation is justified.
That is why I said that contemporary economics is slowly rediscovering something that classical economists were perfectly comfortable with. Bentham argued that there were, what, nine sources of utility: pleasures of sense, pleasures of wealth, pleasures of skill, self-recommendation, a good name, power, piety, benevolence, and malevolence. Only the first couple fit with how conventional neoclassical microeconomics has understood motivation. And I’m perfectly comfortable with the idea that people might act primarily out of benevolence or piety or the pleasure of skill or whatever, rather than self-interest. Quite what the distribution of motivations is among scientists is, and where self-interest sits on their mental horizons, is an empirical question.
OK Alex, I give up. I keep asking you to get more specific about where the arguments I make about the policies I discuss (official exemplars of evidence-based policy produced by highly regarded practitioners) go wrong. And you keep broadening the conversation in ways that do not show where my arguments went wrong.