David Pollock of the British Humanist Association (BHA) complains about my (and other people’s?) ‘unforgiving onslaught of abuse against an unassuming opinion poll‘. He has since been joined by a Jemma Hooper (also of the BHA), who thinks ‘professionals should applaud quantitative data’ (more of which later); and Caspar Melville at New Humanist magazine, who seems to be blaming Julian for all this [quite right too!]).
So right now I must confess to feeling a lot like Denis Healey did after he had been set upon by Geoffrey Howe.
I’ll deal with some of the points of my critics in turn.
David says this:
even without any pretensions to being serious sociological research (on a budget of £5,000?), the poll is surely indicative.
No David, it is not indicative. I don’t think you quite understand the abject hopelessness of your poll. If an A-level Sociology student had come up with this piece of research then they would have failed (hopefully). It is interesting that nobody has cared to respond to my substantive points. Here are a few more.
David says this:
religious people who reject both the option of saying that ‘religious beliefs are needed for a complete understanding of the universe’ and the option of saying that ‘People need religious teachings in order to understand what is right and wrong’ have at most a pretty attenuated sort of religion…
1. The poll does not give people the option to reject the view that each statement represents. That is not how it was set up. (Which is part of my complaint.) This is what the instructions say: ‘I’d like you to tell me on balance which one in each pair most closely matches your view. You might find that the statements overlap a little, however please tell me which one you feel most closely matches your view. (If you had to choose just one of the statements which one best matches your view?)’. And also: ‘Where respondents were unsure, interviewers were allowed to select “Neither” or “Don’t know”, but these options were not presented to respondents and they were encouraged to choose a statement from each set if they could.’
So nobody is rejecting anything substantive when they respond to this poll (and David, you ought to retract that statement – it isn’t fair to those who responded to this poll). People are presented with a forced choice, and then they are encouraged to answer one way or the other even though it is conceded that there might be overlap in the statements (too right there is). I still find it staggering that anybody would think that this works as a methodology. Did the BHA bother to check with a social scientist? It took me about five seconds to see that this poll was hopeless (I read about it in New Humanist magazine). And I’m a very mediocre sociologist.
2. Notwithstanding the methodological disaster, it is conceptually up the creek as well. Christians in the UK are not particularly anti-science. Most of them believe in evolution (at least, they did when I last checked). So it is entirely plausible that people whose religious belief is not attenuated – not that it is obvious that an attenuated religious belief equates to humanism anyway, but never mind about that – will choose the statement ‘Scientific and other evidence provides the best way to understand the universe’ rather than ‘Religious beliefs are needed for a complete understanding of the universe’ when they are forced to choose (there’s that irritating methodological point again). Also it is possible that this is what religious people would say even if they weren’t forced to choose. It is entirely plausible that they might think that science explains what happens in the universe; religion what happens outside of it.
The statement about religious teachings, and right and wrong is also a conceptual mess. I’m an atheist, and I might think that people need religious teachings in order to understand what is right and wrong. Sociologists in the structural functionalist tradition, for example, think something like this (not quite like this, but not far off). Not all of them are religious. Also religious people might not think that religious teachings are necessary for an understanding of right and wrong. It is easy to imagine that some religious people will think that God has granted human beings an innate ability to understand the difference between the two.
So even if the poll wasn’t a methodological disaster, it’s a conceptual disaster.
It doesn’t show what is being claimed for it.
3. Even if there were no methodological or conceptual problems, it still doesn’t suggest the conclusion that the BHA was so desperate to find. It is possible to elicit opinions on all kinds of things by the means of questionnaires. It does not follow that people have these opinions. If I go out into the street and ask people about their eschatological views, then I’ll get eschatological views, even from people who had never given this stuff a thought before. The fact that people say that they think science is the best way to understand the universe doesn’t mean that they actually think it is the best way to understand the universe. They may have no idea about what constitutes science. They may think homeopathy is science, for example. Perhaps they are amongst the one-third of British people who think that the sun goes around the earth.
Okay, so what else?
Well, David offers an instrumental reason for conducting the poll. He thinks that it will (might?) help improve the standing of the BHA at meetings with ministers. Oh dear. So the idea here is to commission the world’s worst poll in the hope that nobody will notice? David suggests that I am scornful, etc., and I guess that is kind of true. But really I am trying very hard. The poll is a disaster. If it improves the BHA’s standing, then I really am giving up all hope for the triumph of reason.
I might continue this in another post. But just let me deal with Jemma Hooper.
Jemma, whether people should applaud quantitative data depends (partly) on whether the data is any good. This poll’s data is hopeless. Therefore, it should not be applauded. Do you think that we should applaud the quantitative data that predicted a win for Thomas Dewey in 1948 US Presidential Election? It’s a famous polling error. Truman, having won, appeared on the news holding a copy of the Chicago Tribune, which had printed “Dewey Beats Truman” on its front page on the basis of polling data.
Moreover, even if quantitative data is good, not everybody thinks it is useful. This is why some people are committed to using qualitative research methods. This is very basic sociology. The kind of thing which people learn about at GCSE level. (I’m not being rude here – it is the kind of thing people learn about in GCSE Sociology.)
The most disappointing thing about this whole affair is that the BHA, an organisation presumably committed to reason, proper enquiry, etc., has been so cavalier in the way that it approaches social research. They ought to be ashamed of themselves. They should stop defending the indefensible (when you’re in a hole, stop digging). And they should apologise to their members for wasting £5000.00 on a worthless poll.
Edit: Ophelia Benson has blogged on the BHA’s ‘we’re a campaigning organisation, cannot be overly concerned with academic pedantry’ defence.