Ming: the choice of illiterate, old amateurs?

10 February 2006

The counter-spin from Camp Campbell: Sir Menzies’ supporters argue that the online poll – commissioned by John Stevens, a former Tory MEP who defected to the Lib Dems and is a friend of Mr Huhne – was bound to be weighted to younger, professional, internet-literate members, precisely the kind of people attracted to the Eastleigh…

Written by

James Graham

musings

The counter-spin from Camp Campbell:

Sir Menzies’ supporters argue that the online poll – commissioned by John Stevens, a former Tory MEP who defected to the Lib Dems and is a friend of Mr Huhne – was bound to be weighted to younger, professional, internet-literate members, precisely the kind of people attracted to the Eastleigh MP’s campaign.

Pah! Who wants those sort of votes anyway?

It should be pointed out of course that the Tories have even fewer “younger, professional, internet-literate members” and yet YouGov predicted Cameron’s victory by 1%.

Comments

14 responses to “Ming: the choice of illiterate, old amateurs?”

10 February 2006

Martin Tod

No idea where that quote came from…

Speaking as someone who’s spent a few million quid of other people’s money on market research in the past (possibly even more than John Stevens), the issues that struck me were:

* The sample size is far too small to call a three-way AV election like the LD leadership election – about half the size of the final poll YouGov ran for the Tories. Statistically, the error rate is going to be considerably larger than 1%. The election is far closer than the Tory election (but that’s been true from day 1).

* The sample isn’t properly controlled. YouGov can’t know whether it reflects our membership because none of the candidates have membership lists. There’s no sign that they corrected the results for our demographic profile (which I believe has been published somewhere) – and given the very small pool they were recruiting from, this would have been very difficult. I’m surprised that they didn’t show the demographic, especially regional, breakdown of their sample in their report. If there were the wrong proportion from London or Scotland, or Hampshire, with that base size, the poll could be completely skewed.

* The most important problem is that the voting intention question came after a whole series of ‘push polling’ type questions – with ‘do you agree’ statements attacking Simon and Ming – but with no serious concern raised about Chris. If it was a serious attempt to understand voting intention, it would have put the voting intention question first – and certainly not have asked for reaction to whole series of statements first.

So the candidates may be closer than they were at the start of the campaign. But, based on this poll, it’s not safe to call the order either way. You can’t even guarantee that Chris will beat Simon.

And, after all, as Chris Rennard would say, the only votes that count are the ones in the ballot box.

Martin

Reply
10 February 2006

Gareth

er, ‘push polling’ Martin?

Good set of questions I thought – and among those eliciting interest. How on earth can you call questions about tax and Trident ‘attacks’ on Simon and Ming?

Yes, it’s close. Yes, it’s difficult to call. Yes, YouGov got the Tory leadership result right to within 1 percent 😉

Reply
10 February 2006

Martin Tod

Gareth,

I didn’t… asking

Do you agree or disagree with the following statements?
â€˜Simon Hughes is too unreliable to lead the Liberal Democratsâ€™

before asking voting intention is a farcical piece of questionnaire design and means that you can’t take the question as an accurate predictor of the result.

As far as YouGov’s accuracy is concerned, aside from the YouGov survey which got the answer right for the Tories having twice the sample size and a properly designed questionnaire, I’m surprised you’re putting so much credence behind a company whose most recent voting intention survey has the Liberal Democrats on 13%.

Martin

Reply
10 February 2006

Martin Tod

Or to put it more elegantly:

1. Do you agree or disagree with the following statements?
“YouGov is the company which said the Liberal Democrats vote had collapsed to 13% only a few days before we won the Dunfermline by-election.”

2. Which polling company do you think you can trust most to accurately survey the Liberal Democrats?

Martin

Reply
10 February 2006

James

YouGov themselves admit to a margin of error of +/-5% which is why I said yesterday I suspected it was still predicting a Ming win. I would also agree that not asking the voting intention questions first was unusual (although it has to be said that people vote because of their opinion on issues, not the other way around).

But the “push polling” allegation is nonsense, as it asked about the main concerns of all three candidates “Chris Huhne has too little experience of being a Westminster MP to be leader of the Liberal Democrats” came out agree 38%, disagree 52% (and the Ming being too old question was agree 36%, disagree 52%).

As someone who took part in the legendary Camp Campbell YouGov poll, I can say that similar questions were asked in that one as well. Yet the results were deemed too disastrous to publish.

Reply
10 February 2006

Gareth

Martin…

what about the question ‘Chris Huhne is too inexperienced to lead the Liberal Democrats’ which came before the Simon question….?

Reply
10 February 2006

James

I have to say (and this isn’t a personal dig at Martin), but the irony of the last few days hearing Lib Dem activists moaning about “ramping” hasn’t been lost on me. 🙂

Reply
10 February 2006

Angus J Huck

You Gov got Cameron’s margin of victory right because Cameron’s support was even throughout every section of the Tory Party. The sample simply didn’t matter.

In a poll of Lib Dem members, sampling is everything. Constiuency A may have 600 members and be adjacent to Constituency B withy 100.

Many of the MPs actively backing Ming have large memberships. I wonder how many of them were polled?

I still predict that Chris Huhne will come third. Why? Because he is the least known of the three. On past experience, candidates who are little known do badly.

I strongly suspect that the poll sample was made up of middle-class metropolitan activists.

At the beginning of the 1987 General Election, there was a poll putting the SDP/Liberal Alliance on 30% (we got 23% on the day). No other poll put us anywhere near this. So how did it happen? The poll was conducted exclusively in the village of Crockenhill, Kent (which has a Lib Dem councillor – Colin Dibsdall).

How much did John Stevens pay for the poll? Did he buy the “cut-price” version (small, poorly targeted sample – from a list supplied by Chris Huhne)?

By the way. A famous byelection victory was achieved with a last minute bogus opinion poll (won’t say which one). A stroke of genius, or ghastly skullduggery, however one wishes to view it.

Reply
10 February 2006

James

That just isn’t the way YouGov work. They poll people who come to their website and sign up as a Lib Dem member. You seem to think this was some kind of ICM-style telephone survey.

The proof will be in the pudding, but if you’re basing your assumption that the YouGov figures are wrong on the basis of a completely different way to how they actually poll, I suspect you may be in for a surprise.

Reply
10 February 2006

Andy Mayer

My personal plea would be that Liberal Democrats stop using that old chesnut that “the polling doesn’t agree with our canvassing returning/what I’m hearing on the doorstep”, and not just in this context. If canvassing ever produces results that are accurate it is more by chance than by design, it is afterall the oldest push polling technique

“Hello I’m calling from the Liberal Democrats on behalf of top bloke Willy Smith and I was wondering if you would agree with the proposition that Mr. Smith is smashing”… “You would?, and your husband as well? oh lovely would you like a poster?”

Reply
10 February 2006

James

Andy, learn to canvass properly. Seriously. If you’re push polling, you’re doing it wrong.

The party does voter ID rather than attempt to persuade people at the doorstep. This is because we need accurate data first and foremost.

Reply
10 February 2006

Andy Mayer

Seriously James, the moment you introduce yourself as a Liberal Democrat, which is in every VoterID/canvassing script I’ve ever you are biasing the outcome of the conversation. The rest of the introductory spiel is also usually slightly weighted towards hoping the repsondent will tick the definate or probable box. The people doing canvassing are not impartial and also tend to overstate the outcome. Most scripts also have set-piece statements to persuade people to change their minds

That’s fine, the balance between ‘good enough’ VoterID with persuasion techniques in the script is the point of canvassing. My point though was that this isn’t the same as doing a representative poll. Are you suggesting that it is?

Reply
11 February 2006

James

If you apply the right formula to it, and recognise that what you are analysing is swing, not absolute figures, then yes, a good canvass can be surprisingly accurate.

Reply
11 February 2006

Martin Tod

Andy,

I wouldn’t underestimate canvassing. That’s why we put the results through the Richmond formula. Personally I’ve always been surprised how accurate they are (once correctly adjusted).

Gareth,

Earlier questions in the survey (not included in the final report) had deliberately ramped up Chris’s experience outside Parliament – effectively weakening the ‘not enough experience’ criticism compared to the ‘issues’ raised about the other candidates (which the earlier part of the questionnaire had, if anything, reinforced). And the point was not that Simon was particularly picked upon, but that putting any kind of criticism up before asking voting intention is (to put it in technical terms as someone who has spent an 8 figure sum on market research over the last couple of years) er… total crap methodologically. It becomes a concept test, not a preference measure.

Martin

Reply