Obama McCain: Stripping the Noise out of the Polls

myspace-polls.com

Poll-watching has been part of the fun of the Presidential campaign. When the noise is stripped out, the statistical reality is not in accord with the stories being spun

Polls are fun. Swings, even those within the ‘corridor of (statistical) uncertainty’ of 2%-3% for a specific poll, have regularly been used to present a news stories.

There have been plenty of stories built on statistical blips. But when the entire set of results are presented together, the dominant undulations are mostly noise. The polls are rather like regular visits to a fortune teller, who tells a story derived from yarrow sticks, tea leaves or Tarot cards.

The BBC has been providing an excellent comparative summary of four polls. These come from four different organizations, each with variations in methodology. I will rely on visual inspection only (which is enough for spotting the broad level of noise and the most significant real effects statistically).

The four polls

The four polls presented in the BBC summaries have been from Gallop, Rasmussen, Washington Post and Ipsos. These were selected from a more extensive compilation of results from the pollster organisation.

The trends revealed from the four BBC polls lead to several conclusions. Across the period of polling any one poll is mostly showing a lot of noise (swings within the corridor of uncertainty of say 2% for one trend line). The blips just even out over periods of several months. You may wish to interpret it as day on day shifts in voting intentions. But the results are also consistent with repeated confirmation of a ‘null hypothesis’ of no significant difference found. This is further confirmed if any claimed swing is not detected uniformly across the polls.

This sort of inspection shows that the polls are prone to ‘false positives’ – results that show a significant swing over some time period, for one poll, but not for the others. It also suggests that among the false positives were blips associated with Hillary Clinton pulling out of the race, Obama declaring himself Democratic candidate, and arguably the recent conventions. THis way you can just about detect a slight and temporary ‘Palin Bounce’ for the McCain campaign followed by the subsequent drift downwards.

Inspection along the time-scale of the polls revealed almost identical poll percentages for Obama and McCain towards the start (Feb 2008) and recently (Sept 2008). The base-line shows round 50% for Obama, 44% for McCain.

There have been two ‘stand-out’ periods in which McCain has been shedding a few percentage points. One has been over the period of the financial crisis of the last month (Sept-Oct 2008). That, unfortunately for McCain, is significant for several reasons. First, the most recent data are always treated as the most newsworthy and important (the well-known immediacy effect in decision theory). Secondly, the election is advancing rapidly, so that the effect is taken even more seriously.

The polls now all say Obama. The averages for the popular vote have stabilized, and are interpreted as a narrow win for Obama.

Looking State by State

Attention has turned to evaluations are based on probabilities of the candidates winning each State. This is a far more sensible way of using statistics, as the victory does not go to the winner of the popular vote, but to the winner of delegates of the Electoral college. The State by State assessment has more sensitivity towards the range of probabilities of each State staying the same as last time, or switching the affiliation of the nominated members of the electoral college.

On these assessments, Obama is more clearly in the lead, and explains why the commentators are writing as if the result is more clear-cut.

One week to go

With less than a week to go, some of the theories have come and gone. McCain’s run-in seems to have been in military terms a courageous scramble. Obama’s a dignified avoidance of appearing too much of a winner, but still appearing a winner.

One day to go

Commentators are talking as if the polls suggest Obama is a near certainty. McCain claims a last-gasp gain in support enough to give his supporters continued hope. There is even more of a narrowing of concentration by reporters around the one issue ‘how will the voters vote’ and a decoupling of opinion from contextual factors. By that I mean that the economic back drop, for example, has hardly had a mention in comparison with the vivid impact of the latest Joe the Pumber encounter. Bewitched, bothered, and bewildered.

All the polls, all the ‘objective’ analysis point to only one winner. So why are so few commentators (including me) refusing to say there is no hope left for John McCain? Maybe Obama’s recent rallying cry to his supporters offers hope for Senator McCain as well:

Obama the university lecturer embarks on a little treatise on what hope actually means – “that thing deep down inside of us that insists, despite all the evidence to the contrary, there are better times ahead of us”. It is a line he has been polishing now for days, if not weeks. And his audiences always get it, and love it.

Acknowledgement

To myspace-polls for the image and their encouragement to turn us all into pollitical pollsters.

Obama McCain Round Three and the Myth of the Mousetrap

A better mousetrap?

The third Presidential debate provided little further evidence that might change the minds of American voters. A recent study of the myth of the mousetrap suggests how the candidates might be better able to get their ideas accepted

The candidates have had even more media exposure (if such a thing is possible) because reporting of the twists and turns of the Bush regime in its attempts to deal with the financial crisis of the last few weeks.

For all his charisma, Obama sticks to offering a low-key and generally reassuring style of debate. He did not go for broke with an appeal to the anxieties of voters. That was a relatively easy call for his advisors and Obama. He is, after all is moving ahead rather nicely in the polls.

In contrast, McCain is struggling in the polls. Observers suggest that he has decided that there is no option but to stick to a personalised attack on Obama. Arguably, each candidate had strong reasons to stick to their earlier strategies.

The leadership dilemma

The debates offer a case example of the dilemmas facing a leader. Stick or twist? If in a strong position why change? If in a weak position it would be nice to change in a way that addresses the perceived weaknesses. But how to find that new strategy, and how to put it into action?

The Myth of the Better Mousetrap

It just so happens that a book came in for review shortly before the third presidential debate. Anne Miller in The Myth of the Mousetrap: how to get your ideas adopted (and change the world) writes about the challenge for a creative person to get ideas accepted. The book primarily focuses on technological ideas and acceptance seeking processes. I commend it to technologists and inventors, but I want to locate my remarks around its relevance in the context of the Obama-Bush campaign

The dangers of the ‘he who is not with us’ approach

Miller goes back into history to show how scientific pioneers have to find ways of overcoming a comprehension gap. She quotes one scientist who many years ago (in 1944) observed the tendency of someone committed to a new idea to be over-zealous:

Zealous believers commonly follow the motto ‘He that is not for us is against us’ [and that] it does not help the cause to accuse all its critics of a state of mind that is as unworthy as fascism’.

She comments that the phrase ‘ with us or against us’ has ‘a worrying echo’ of George Bush in building his ‘coalition against evil’ , adding that
‘being combative may make your supporters feel good, but it does nothing to encourage people who are teetering on the edge of being interested in your ideas’. Her proposition is that increasingly, influence derives from efforts to involve people which will also harness their creativity.

If Miller is right, we begin to understand the dilemma facing McCain. It is less of a problem for Obama, who seems to have been more successful in involving and enlisting an army of youthful supporters.

Bush, and not thinking about the elephant

Another illustration comes from the success of the Bush campaign of 2000. Gore did not win the case by pointing out that Bush tax cuts would mainly advantage the top 1% of voters. Miller (p105) cites George Lackoff’s analysis that Bush had succeeded because ‘people do not vote with their economic self-interest, they vote with their identity and their values [such as] ..strong defense or family values’ .

I assume Lackoff’s analysis has been noted by strategists on the left and right alike. In any case, values are being repeatedly signalled by both candidates. Yet, this time around, there is an elephant (or a gorilla) that can’t be ignored. And it’s not too far away from the point made by Clinton. It’s the economy stupid. Which then gets dressed up in value-laden language. Last night, [October 15th 2008] John McCain personalised it with extensive references to Joe the plumber’ (a real person).

But there are further complications. Appeals to injustice or real and present danger have immediate emotional impact, also trigger feelings of guilt, inadequacy and anger. Such manifestations have been a feature of recent McCain events, less so with Obama’s.

Miller refers to a favoured notion of mine about change from the behavioural theorist Ed Schein. He suggests that people’s attitudes are ‘unfrozen’ by triggering acknowledgement of dissatisfaction with the status-quo coupled with vision off a better state and a simple credible action. If I have a concern about Schein’s model it is in its simplistic application in which someone goes around whipping up dissatisfaction, anxieties, and feelings of inadequacy. This holds for someone arguing for a new product idea and for a change of leadership. It is likely to be more effective in the latter case, than in the political arena, where the actions may just trigger anger against a common enemy.

So McCain is toast?

‘Barring the unexpected’, I would say yes. The possible sources of a turn-round seem to be dwindling. The initial boost from the impact of Sarah Palin is fading. Chances to score in the three televised exchanges have come and gone.

But these are exceptional times. If there is a change it will be a radical disruption of all the factors that have been pushing the polls in favour of Obama.

It might just be worth planning for the implications for a Democratic win.

Acknowledgement

To Gilleport for the image of a better mousetrap

Leaders in the news: Winners and losers

Howard Schultz Starbucks

In times of crisis, some leaders step forward, others are deemed to have failed. There have been examples of each, as the global financial crisis enters a new critical stage

Fred the Shred takes the fall

Pressure mounted on Sir Fred Goodwin to resign as chief executive of Royal Bank of Scotland (RBS) as the bank seeks to tap the Government’s £500 billion rescue fund. The Government is reluctant to a deal with RBS’s participation unless he relinquishes his role. Although he clung on tenaciously this has been a very bad week for Sir Fred. Another former city hero exits ignominiously.

Sir Fred Goodwin 0

Gordon Brown is no dead cat

The deeper the crisis, the more polls seem to swing towards Prime Minister Gordon Brown. David Cameron and George Osborne grudgingly offer support the government. Are we seeing a ‘dead cat bounce’, or is there life in the political career of Gordon Brown? He appears more relaxed in the last two weeks than he has been since taking over from Tony Blair as Prime Minister.

Brown 1, Cameron 0, Osborne 0

Boris Forces Resignation of Sir Ian Blair in Leadership Battle

The resignation of Sir Ian Blair [October 2nd 2008] develops into a political story. The BBC traced his turbulent career. Boris Johnson, incoming Mayor of London, is proving a hands-on leader willing to act forcefully. Sir Ian, under pressure on operational and personal fronts, was called into a ‘meeting without coffee’ by the Mayor before tending his resignation.

Boris 1 Blair 0

Obama and McCain Round 2

The second televised debate between the two candidates [October 7th 2008, Nashville, Tennessee] is as stage-managed as the first.
A key negative moment was was reported widely as

Jabbing his finger and spitting out “that one” instead of naming Barack Obama, John McCain showed an angry side

Polls suggest that Barack Obama is moving ahead.

Obama 1 McCain 0

Dick Fuld faces the music

Dick Fuld, controversial CEO of Lehmans has had a very bad few weeks. When ‘invited’ to testify before a hostile congressional committee following the crash of his company, he demonstrates his robust leadership style, denying wrong-doing or ethical weakness. He ticks the boxes for the callous Wall Street fat cat. Fuld very much the loser here.

Dick Fuld 0

Darling’s drastic rescue bid of the banks and maybe Gordon Brown

As The Times sees it
Chancellor Alistair Darling [October 8th 2008] launched a drastic rescue of Britain’s high street banks [to avoid] a cataclysmic failure of confidence by announcing a part-nationalisation plan with £50 billion of taxpayers’ money. Alistair Darling, like Gordon Brown has had a better week.

Alistair Darling 1

Starbucks, Schultz and the running taps

Howard Schultz, returned to the chief executive role at Starbucks earlier this year, faced with serious loss of froth in the business. Poor figures and closures continue. This week [October 8th 2008] the ‘running taps’ story threatens to sully the firm’s good environmental reputation.

Starbucks 0, Howard Schultz 0

And in the long run?

Not all these cats are dead. And, as we know, cats have seven lives.

Leaders we deserve: John McCain gets personal

Ad Hominem Attack

John McCain began his presidential campaign with a reputation as a free spirit and gadfly of the Republican party machine. At first his campaign was characterised by his willingness to challenge conventional wisdom. As the election approaches, he is relying on a more negative strategy.

In emotionally-charged public meetings, anger of John McCain supporters is being channelled against a common enemy. Watching one of the rallies on a TV clip, I also had an emotional reaction It left me feeling that in a close election, the ideals of the founders of the USA become overtaken by deeper atavistic fears.

In Britain today, as in America, there is a popular fury over the present financial chaos, and fears for the future. The British public too is finding enemies to turn their anger against. Bizarrely, the conservatives are out-doing the government in attacks on the avarice of city fat-cats, and the need to punish them and withhold future bonuses. Meanwhile, the Prime Minister, recently regarded as toast, has gained ground in opinion polls.

The Ugly Campaign

In the USA, the presidential campaign is turning ugly [October 8th -9th 2008]. BBC viewers witnessed the heightened emotional state of Joh McCain supporters in one public meeting. We saw clips including a pro-life demonstrator who shouted above the general din that Obama was a baby murderer. This may have been media distortion and simplification, although it conforms to a report in the New York Times.

Senator John McCain joined in the attacks on Thursday on Senator Barack Obama for his ties to the 1960s radical William Ayers, telling an angry, raucous crowd in Wisconsin that “we need to know the full extent of the relationship” to judge whether Mr. Obama “is telling the truth to the American people or not.”

..what has been most striking about the last 48 hours on the campaign trail is the increasingly hostile atmosphere at Mr. McCain’s rallies, where voters furiously booed any mention of Mr. Obama and lashed out at the Democrats, Wall Street and the news media.
“I’m really mad!” shouted a man in the audience in Waukesha, where Mr. McCain and his running mate, Gov. Sarah Palin of Alaska, were conducting a town-hall-style meeting. “And what’s going to surprise you, it’s not the economy. It’s the socialists taking over our country.”

McCain the maverick

Throughout the campaign, McCain supporters have emphasised his maverick tendency as a positive characteristic. The point was illustrated earlier by American columnist David Brooks. (I have synopticised the quite brilliantly-written piece).

On Tuesdays, Senate Republicans hold a weekly policy lunch. The party leaders often hand out a Message of the Week that the senators are supposed to repeat at every opportunity. John McCain generally spends the lunches ridiculing the whole proceeding. This sort of behavior has been part of McCain’s long-running rebellion against the stupidity of modern partisanship. In a thousand ways, he has tried to preserve some sense of self-respect in a sea of pandering pomposity.

When McCain set out to win the presidency he would venture forth in his bus, going places other Republicans don’t go, saying things politicians don’t say. But McCain hasn’t been able to run the campaign he had envisioned. Instead, he and his staff have been given an education by events.

McCain started his general-election campaign in poverty-stricken areas of the South and Midwest. He went through towns where most Republicans fear to tread and said things most wouldn’t say. It didn’t work. The poverty tour got very little coverage on the network news. McCain and his advisers realized the only way they could get TV attention was by talking about the subject that interested reporters most: Barack Obama.

McCain and his advisers have been compelled to adjust to the hostile environment around them, running a much more conventional race, of the kind McCain himself used to ridicule. The man who lampooned the Message of the Week is now relentlessly on message. Both McCain and Obama had visions of upending the system. Maybe in office, one of them will still be able to do that. But at least on the campaign trail, the system is winning.

The leaders we deserve

The deterioration of political campaigns into emotionally-charged rallies can not be good for the ideals of democracy. American voters are facing a critical decision in critical times.

There can be no ‘right answer’. The result is, for all the complex interactions and distortions, deliberate or unintended, the will of the electorate.

The charismatic Obama has a wider appeal internationally, if we can believe what is being written by commentators of all hues. If anything, that may be strengthening the beliefs of the committed. ‘Nobody loves us and we don’t care’ as one group of disgruntled football supporters has been known to chant ‘.

The republicans are far from beaten.

Acknowledgements

To J.G. for alerting Leaders we deserve to the David Brooks article.
To anyone claiming image rights to the thumbnail which I grabbed from the net and can’t yet locate the original. I will be pleased to acknowledge its provenance.

Obama McCain Round One: Blink and You’ve Got It

The long-awaited debate between the Presidential candidates takes place. In a well-rehearsed encounter, the moves practiced on the campaign trail are brought into the living room

Asleep on my watch, I missed the debate as it happened. A few hours later the first reports were available, as well as full video transcripts.

The BBC provided a neat ‘as it happened’ commentary which interlaced commentary from BBC North America editor Justin Webb with tweets from interacting viewer plus clips from blogs. The format works quite well for sports transmissions, and has been effectively transplanted. A nice innovation. It also included a 5-minute clip on the exchanges over foreign policy (the focus of the first of the debates)

The clip is more than enough to confirm some expectations. That the two candidates are operating according to stereotype. Mr Obama is prone to the extended answer; Mr McCain appears more drilled and perhaps less flexible. Neither (in the clip) appears to be far from a well- rehearsed performance.

Some of the tweeters indicate that they had heard it all before. This point was reinforced by Justin Webb.

This is a huge disappointment – set piece memorised stuff from both candidates

The combatants were sticking mainly to the polished moves thoroughly practiced in dozens of speeches during the campaign. By sticking to their comfort zones, they reveal ingrained stylistic aspects of their performances. Mr McCain may have successfully taken advice not to say ‘my friends’; Mr Obama to be less professorial. But the leakage was there for the body language enthusiasts. Did John blink too many times? (I haven’t counted). Was Barack tempted into verbosity? Perhaps.

Just rhetoric?

So is what we see anything more than a display of rhetoric? You could see it that way. Political performance art where the audience reactions could not have greater consequences since the battles in the Coliseum of ancient Rome when gladiators received the thumbs up or down from the Emperor.

Footnote:

Google presented me with a snippet that looked as if John McCain was a clear winner. Here is the entire line (obviously extracted from a longer statement).

1957 In the nearest thing to a presidential debate so far, at a Californian megachurch, McCain is widely perceived to have bested Obama with a straight …

My reaction was that [Google states that] McCain is widely perceived to have bested Obama.

The snippet came from the BBC report above. If you go back to the original you find that it refers to Mr McCain besting Mr Obama in the answer he gave to a specific question. Arguably, it was the only thing in the entire post that could have indicated that McCain was a clear winner. This was confirmed in other reports. Curious. Is there a republican bugging Google’s search engines to find headlines that support Senator McCain? It is early morning still. My conspiracy brain-cells are over active.

Setting aside obviously partisan views, initial reports suggest that neither candidate was ‘bested’ in this battle. It’s still all to fight for.

Bush wages war on financial terrorism but fails to rally the troops

President Bush launches one more campaign in the war against terror. But this time it is against the terror that threatens global capitalism

The aging generalissimo is preparing to step down. Perhaps he will hand over to a dynamic young leader. Perhaps to one even older than himself. But step down he will, in a matter of months.

While he might have wished for respite, he has to confront one last crisis in his last days as President. The United States faces the most severe financial crisis since the 1930s.

His plan is to fight the financial war with a $700 billion attack force. But he has to win support of his financial generals who are not convinced.

When he addressed the nation he looked tired and had lost the jaunty air which has been a feature of his press conferences.

Peering at events from the UK, BBC’s celebrity journalist Robert Peston suggests that Bush is trying to bully the troops into line.

Political impact

The political ramifications of this battle are becoming clearer. The presidential aspirants have been dragged into the crisis before they would have chosen. Senator McCain has made the more assertive move, claiming that it calls for a temporary cease-fire in the Presidential campaign. He suggests a face-to-face debate scheduled for Friday [September 26th 2008] should be cancelled. Senator Obama gives a more nuanced response (a ‘yes and’). Sure, the financial crisis is vitally important and urgent. But a wannabe President has to deal with more than one thing at a time.

That exchange struck me as significant. McCain made a plausible move. Obama’s response worked better for me.

So what?

So what? I don’t get to vote in November.

There will be plenty more mud flying around, but more seems likely to stick onto the Republican candidate and his high octane vice-Presidential candidate around their grasp of financial affairs. These hits may be harder to brush off.

It just looks, for the first time in months, that the odds are swinging back to Obama, and that vulnerabilities in the McCain campaign will do his prospects real damage.

Obama: Change comes to Washington

Barack Obama adds a creative twist to his message of change. In his acceptance speech for the Presidential nomination he insists that America will change. But change will not come from Washington, he insists, it will come to Washington

It was a speech deliberately echoing the “I have a dream” speech of Martin Luther King. King’s dream was of the community of races within America.

Obama also echoed John F. Kennedy, who insisted that Americans ask not what their country can do for them, but what they can do for their country.

But these echoes from the past reinforced Obama’s message for the future. In a creative leap, he turned the more simplistic change message on its head. Yes, America would change. But not because a new leader and administration in Washington would change America, but because America would change Washington. ‘Change comes to Washington’ he insisted.

This was the Obama combining policy with personality. The Obama avoiding what he has demonstrated in the past, avoiding burning bright intensifying the charisma of the leader. If anything, he seemed to be deliberately holding back for much of the speech lest the message was lost in the dazzle of full-wattage Obama.

There was plenty of yes and about the speech. Senator McCain? We are all patriots. That should not be an issue. [I remembered briefly the call by David Cameron, the newly appointed leader of the Conservatives to avoid Punch and Judy politics. Would the fine words last longer on this campaign than they did in the UK?].

But the dream was an old dream fulfilled rather than one freshly imagined

It is that promise that has always set this country apart – that through hard work and sacrifice, each of us can pursue our individual dreams but still come together as one American family, to ensure that the next generation can pursue their dreams as well.

The context of change was spelled out

Unlike John McCain, I will stop giving tax breaks to corporations that ship our jobs overseas, and I will start giving them to companies that create good jobs right here in America.
I will eliminate capital gains taxes for the small businesses and the start-ups that will create the high-wage, high-tech jobs of tomorrow.
I will cut taxes – cut taxes – for 95% of all working families. Because in an economy like this, the last thing we should do is raise taxes on the middle-class.

And for the sake of our economy, our security, and the future of our planet, I will set a clear goal as President: in ten years, we will finally end our dependence on oil from the Middle East.

[On Foreign Policy] As Commander-in-Chief, I will never hesitate to defend this nation, but I will only send our troops into harm’s way with a clear mission and a sacred commitment to give them the equipment they need in battle and the care and benefits they deserve when they come home. I will end this war in Iraq responsibly, and finish the fight against al Qaeda and the Taleban in Afghanistan. But I will also renew the tough, direct diplomacy that can prevent Iran from obtaining nuclear weapons

Did it work?

I don’t know. As an outsider, recently I have struggled to understand the impact of the politics of Scotland’s governing party as it seeks independence from England. What chance do I have of assessing the impact of Obama’s performance on his prospects of election as President?

One thing is clear. Barack Obama is an exceptionally creative leader. He is offering a clear choice for change, by invitation rather than exhortation. His message is that change comes from the people: encouraged but not dictated by its leaders. It is still a message requiring the audacity of hope for its full-hearted acceptance. And it is an invitation that captures the principle that we create the leaders we deserve.

Postscript

This post deliberately avoided replicating views of other observers. As I listened in cosy darkness, I did not pick up the context and visual impact of the speech. As Google listed around 1000 news reports on Obama this morning, there are plenty of reports available to chose from.

Senator McCain Can Go All the Way

Senator John McCain can go all the way to The White House. Politics is the art of the possible. McCain is looking more and more a possible winner

Events are lining up favourably for Senator John McCain. It works for him that the Democrats still have two candidates. It works for him that he can campaign without having to go back on the themes which have brought him to prominence over the last four years. Even the looming problems in Latin America seem likely to work in his favour.

America had anticipated some resolution in the race for the Presidency this week. It was expected that the Democrats would whittle down their candidates to one who would be strong favourite not just for the nomination, but to become the next Pesident of the United States.

So what happens? Events continue to defy prediction. Hillary Clinton wins in Rhode Island. It couldn’t be more symbolic, but a win is a win. It is enough to keep her in the race.

Meanwhile Ecuador and Columbia draw up troops on their common border. President Chavez of Columbia and Correa of Columbia both seem in confrontational mood.
According to The Statesman, Chavez accuses Correa of initiating a cross-border incursion of tanks and troops.

Denouncing Colombia’s slaying of rebels in a cross-border raid into Ecuador, President Chavez said Sunday [March 2nd 2008] that Venezuela will respond militarily if Colombia violates its border. He ordered Rafael Correa ‘s Embassy in Bogota closed. Ecuadorean President Rafael Correa also ordered troops to the Colombian border, withdrew his government’s ambassador from Bogota and ordered Colombia’s top diplomat expelled.

These events seem unlikely to have had much influence on the immediate outcome of the Primary elections this week. But they could well have more of an influence in the months ahead.

Suddenly, the excitement for many Americans generated by Barack Obama’s campaign may seem not quite as potent. His promise of radical change not so all-consuming and satisfactory. Likewise, the experience of Hillary Clinton appears less a convertible asset in a Presidential campaign against a warrior hero whose potential popularity is increasingly drawing comparison that of the much-loved Ronald Reagan.

All in all, the road to the White House is opening up for McCain.

That’s not to say that the battle is over. It has hardly begun. But however charged with difficulties, it is not going to be simple matter of the Democratic Party picking the next President from two likely winners.