Poll-watching has been part of the fun of the Presidential campaign. When the noise is stripped out, the statistical reality is not in accord with the stories being spun
Polls are fun. Swings, even those within the ‘corridor of (statistical) uncertainty’ of 2%-3% for a specific poll, have regularly been used to present a news stories.
There have been plenty of stories built on statistical blips. But when the entire set of results are presented together, the dominant undulations are mostly noise. The polls are rather like regular visits to a fortune teller, who tells a story derived from yarrow sticks, tea leaves or Tarot cards.
The BBC has been providing an excellent comparative summary of four polls. These come from four different organizations, each with variations in methodology. I will rely on visual inspection only (which is enough for spotting the broad level of noise and the most significant real effects statistically).
The four polls
The four polls presented in the BBC summaries have been from Gallop, Rasmussen, Washington Post and Ipsos. These were selected from a more extensive compilation of results from the pollster organisation.
The trends revealed from the four BBC polls lead to several conclusions. Across the period of polling any one poll is mostly showing a lot of noise (swings within the corridor of uncertainty of say 2% for one trend line). The blips just even out over periods of several months. You may wish to interpret it as day on day shifts in voting intentions. But the results are also consistent with repeated confirmation of a ‘null hypothesis’ of no significant difference found. This is further confirmed if any claimed swing is not detected uniformly across the polls.
This sort of inspection shows that the polls are prone to ‘false positives’ – results that show a significant swing over some time period, for one poll, but not for the others. It also suggests that among the false positives were blips associated with Hillary Clinton pulling out of the race, Obama declaring himself Democratic candidate, and arguably the recent conventions. THis way you can just about detect a slight and temporary ‘Palin Bounce’ for the McCain campaign followed by the subsequent drift downwards.
Inspection along the time-scale of the polls revealed almost identical poll percentages for Obama and McCain towards the start (Feb 2008) and recently (Sept 2008). The base-line shows round 50% for Obama, 44% for McCain.
There have been two ‘stand-out’ periods in which McCain has been shedding a few percentage points. One has been over the period of the financial crisis of the last month (Sept-Oct 2008). That, unfortunately for McCain, is significant for several reasons. First, the most recent data are always treated as the most newsworthy and important (the well-known immediacy effect in decision theory). Secondly, the election is advancing rapidly, so that the effect is taken even more seriously.
The polls now all say Obama. The averages for the popular vote have stabilized, and are interpreted as a narrow win for Obama.
Looking State by State
Attention has turned to evaluations are based on probabilities of the candidates winning each State. This is a far more sensible way of using statistics, as the victory does not go to the winner of the popular vote, but to the winner of delegates of the Electoral college. The State by State assessment has more sensitivity towards the range of probabilities of each State staying the same as last time, or switching the affiliation of the nominated members of the electoral college.
On these assessments, Obama is more clearly in the lead, and explains why the commentators are writing as if the result is more clear-cut.
One week to go
With less than a week to go, some of the theories have come and gone. McCain’s run-in seems to have been in military terms a courageous scramble. Obama’s a dignified avoidance of appearing too much of a winner, but still appearing a winner.
One day to go
Commentators are talking as if the polls suggest Obama is a near certainty. McCain claims a last-gasp gain in support enough to give his supporters continued hope. There is even more of a narrowing of concentration by reporters around the one issue ‘how will the voters vote’ and a decoupling of opinion from contextual factors. By that I mean that the economic back drop, for example, has hardly had a mention in comparison with the vivid impact of the latest Joe the Pumber encounter. Bewitched, bothered, and bewildered.
All the polls, all the ‘objective’ analysis point to only one winner. So why are so few commentators (including me) refusing to say there is no hope left for John McCain? Maybe Obama’s recent rallying cry to his supporters offers hope for Senator McCain as well:
Obama the university lecturer embarks on a little treatise on what hope actually means – “that thing deep down inside of us that insists, despite all the evidence to the contrary, there are better times ahead of us”. It is a line he has been polishing now for days, if not weeks. And his audiences always get it, and love it.
To myspace-polls for the image and their encouragement to turn us all into pollitical pollsters.