A rough guide to reading Leadership polls

The latest leadership poll in Britain signals good news for the Conservatives, and bad news for the present Government. But how good, and how significant are the results? A simple three-step process is suggested which will help readers to take a more informed view of what such polling results might mean.

According to The Guardian newspaper, the conservatives are the strongest they have been in the polls for 12 years. The BBC examined the poll data and concluded that support for Labour under Gordon Brown could drop to 29%, while the Tories led by David Cameron would attract 42%.

Good news indeed for David. The article, by Julian Glover continues the regular monthly polls by the Guardian conducted by polling experts ICM. I tried to assess the significance of the results, and quickly hit several complications. The BBC news was particularly unhelpful. It plucked out a few elements of the Guardian poll, but in a way that left me searching for pen and paper to make sense of the information.

An hour, and a few sheets of crumpled notepaper later, and I had arrived at some interesting conclusions. I realized that it was not the first time I had been forced to work out things in this way from newspaper reports of polling results.

Here is a rough and ready guide that might help anyone who is not already familiar with the terrible beauty of statistical analysis. It is based on not much more than a respect for numbers (numeracy).

How to read opinion polls

Step 1 Stick as closely as possible to the data and decide what the numbers are telling you. You may have to re-organize the data for this.
Step 2 See what conclusions are being drawn in the news story
Step 3 Ask what gaps are there between the data and the conclusions.

The three-step process applied

In practice, news stories tend to rush you on to step 2, then perhaps provide some help with Step 1, and avoid much mention of Step 3. The BBC report illustrates the point:

Support for Labour under Gordon Brown could drop to 29%, while the Tories led by David Cameron would attract 42%, an opinion poll suggests. With Mr Brown expected to take over as PM, the ICM/Guardian phone poll asked 1,000 adults at the weekend which party – with a named leader – they preferred. The same question a month ago suggested Labour under Brown would gain 31% and Conservatives under Cameron 39%. The Lib Dems under Sir Menzies Campbell dropped to 17% from 19% a month ago. When asked about voting intentions – regardless of leaders – the poll suggests 40% of respondents supported the Conservatives, up three points on January. Support for Labour was static on 31%, and the Liberal Democrats lost 4 points to drop to 19%.

All clear? Not unless you can think in more dimensions than I can. It’s actually clearer if you draw very crude graphs. Then you see he complications arising because the pollsters have been measuring voting intentions in two ways: mentioning, and not mentioning the leaders of the parties.

Even without graphs, if you put the data into a table you will see that the data reveals a swing to the conservatives (39% to 42% with mention of David Cameron, 37% to 40% without mention).

In rather similar way there is a swing away from the Liberal Democrats (19% to 17% with mention of Ming Campbell, 23% to 19% without mention). The labour figures are harder to interpret. They indicate a swing away only when Gordon is mentioned (31% to 29%, static at 31% without mention of Gordon).

This gives us the basis of our Step one. The data says there is a slight shift to the conservatives, a slight switch away from the Lib Dems, a slight switch away from labour if Gordon Brown is introduced into the questioning.

Step 2: The conclusions drawn are that the conservatives are the strongest they have been in the polls for 12 years (The Guardian claim), and that support for Labour under Gordon Brown could drop to 29%, while the Tories led by David Cameron would attract 42% (BBC interpretation of the Guardian poll).

Step 3: Well, actually there are various assumptions which are glossed over in the claims in Step 2. Sticking strictly to the data, we cannot project what support will be for the parties, with or without David, Gordon and Ming built in.

Nor can we speculate what difference their presence or absence is likely to make on voting day. These are among the real-life complications which make back-projection for twelve years inadequate for projection one or two years ahead.

I’m inclined to see what happens when we have a few more months of data. (Plea to The Guardian / ICM: please can you keep the ‘with and without’ questions to help us work out what is happening, using our three-step system).