Oct 28 2008

Statistics Is Not Simple Math, And Opinion Polls Are Statistically Dodgy

Published by at 8:36 pm under 2008 Elections,All General Discussions

When NASA or the DoD need to track a satellite we use multiple parallel models to account for gravity, solar pressure, the height and drag of the atmosphere (what little there is at orbital altitudes) and a variety of other models. We update the measurements in all these factors and run statistical models, weighting newer data over older data, all the while checking for the random outlier measurement. Even with all this we can only predict solid orbits for about 7 days before we have to update them in many cases.

And that is with a well characterized and physics bases system to model. Oh if opinion polls could be so simple!

Being a bit of a math addict I sometimes forget that pollsters are not really as versed in scientific sampling as some engineers and scientists, though the case could be made they are better than the scientists who modeled ‘global warming’ and CO2 levels. I found two commentaries very interesting and revealing.

One is from Jay Cost at RCP which illustrates why the polls are a problem, but interestingly Jay cannot find the answer to the conundrum:

On Friday, I noted that the differences among the national polls is large enough to suspect that something other than random variation is causing the disagreements.

It is unlikely that random variation would produce these effects. Today’s Rasmussen poll shows McCain significantly higher than the RCP average, and it has consistently been higher than the RCP average for the last three weeks. IBD/TIPP frequently pegs Obama’s number significantly lower than the RCP average, and it has shown him lower than the RCP average every day since it began. The GWU/Battleground poll has shown McCain consistently higher than the RCP average for 10 of the last 10 release dates, frequently at significant levels.

As I noted in my posts on the Bi-Poller world of polls (here and here) if one groups polls by how they treat party affiliation in their final results into two categories we find two distinct families of polls with consistent results. Either pollster use the historical party turnout models which would give Democrats a maximum advantage of 4% over the GOP side (‘traditional’ polls to steal the Gallup vernacular), or pollster assume Democrats will experience a never seen in modern times advantage of 8-12% in voter turnout (the ‘extended’ polls).

Once you realize we have to opposing views on turnout expectations the polls are quite consistent within their respective group – and the RCP ‘average’ falls between the two groups (showing either higher or lower expectations for Obama and the Dems). Both assumption cannot be true – one model will be proven right next week, and the odds are stacked against the ‘extended’ turnout models. In fact, if the pollsters and analysts were being honest, they would not that the only way for Obama to achieve the lead he has now is for a never before seen advantage in voter turnout for the dems – a historic wave that would break the pattern going back decades.

The other commentary that caught my eye was from the left, and the fact many democrats are subconsciously or intuitively seeing the problem for Obama:

With all due respect to Judis, I’m not popping anything just yet, except maybe some sleeping pills to get me through the night. (That’s for you, John.) In the last few days, pretty much every tracking poll I trust (WaPo, Gallup, Rasmussen) and several I either don’t trust (that would be you, Zogby) or don’t have much of an opinion about (Kos,Investor’s Business Daily) has shifted toward McCain, in some cases sharply. While some of the more traditional (i.e., non-tracking) polls show Obama with a big lead–like the Pew poll Judis cites–these polls tend to reflect older information.

As I noted earlier, Pew is far from ‘traditional’ in their turnout model.  They are assuming some of the highest disparity in turnout models for the Dems amongst all the ‘extended’ polls. But the author sees the danger:

As of this writing, Obama’s lead in the national tracking polls looks to be around five points (I get 5.5 when I average all six of the trackers I mentioned, along with theHotline and Battleground trackers, which haven’t changed much in the last few days). If that drops two-to-three points, as it easily could in a week, I don’t think it’s crazy to think McCain will have a shot at winning Pennsylvania, Virginia, and/or Colorado. 

If it drops to a tie in these ‘traditional’ polls (though Rassmussen took himself out of this category by using his sample results to set his weighting – a big time statistical no-no) then Obama has little chance of winning. And right now my estimate is the ‘traditional’ polls are now at Obama +4% and falling.

Averaging these two classes of polls is a fool’s errand. Either McCain-Palin get out the vote and limit their disadvantage to 2-4% in turnout, there is some mythical wave of Democrat love out there. It is remotely possible the latter is true, but the odds are incredibly low given the historic peaks for great democrat years. As of tonight the irrationally optimistic national number for Obama is +9.2%, and the historically proven model is showing +4%. Only one number is closing in on what the result will be Tuesday – the other one is simply flat out wrong. It is either-or, not an average. People can decide which model they like or believe, but the math says it is one or the other.



Given the tightness of the last two presidential elections and the trend for races to tighten in the last week and tip to the conservative side, I would say it is not surprising Obama and McCain are spending their final days in PA and VA.

Update: I forgot to link to the latest from Obi-wan Kenobi:

That’s why key Obama people are nervous. People like Ed Rendell ought to feel good about Pennsylvania. He doesn’t. He knows it can slip away. And don’t forget Obama lost primaries where he had an eight point lead.

What is justified is hope. A stable week economically and a little bit of a finish by McCain and it is doable. Remember this isn’t 1974 or even 1976 in one important way. There’s a GOP candidate who probably had  blow- out debate . And that may be  the single most important  thing voters most remember on election day.

With falling gas and energy prices, a recovering stock market (fingers crossed there) and home values rebounding here and there it could be just what the doctor ordered. And then there is the image of Obama playing Robin Hood with our hard earned money ….

7 responses so far

7 Responses to “Statistics Is Not Simple Math, And Opinion Polls Are Statistically Dodgy”

  1. sbd says:

    It appears that Chuck Baldwin, candidate for President of the Constitution Party may have gotten the message regarding Obama’s birth certificate. This is the response I received from the Baldwin campaign responding to my suggestion that his campaign has standing to challenge Obama’s eligibility for the office of the President.

    Apparently, he might just challenge both Obama and McCain at the same time which in my opinion is a smart move that will foreclose any doubts from either party.

    This is a quick update to let people know about a special radio show tonight October 28th at 10pm Eastern and 7 Pacific, Constitution On Fire on Restore the Republic Radio, hosted by Bob Andrews of the Georgia Constitution Party. They plan to discuss the implications of the recent court case that has been brought before the Hawaian supreme court, on Sen Obama’s birthplace, combined with the fact that the GOP candidate would be the oldest president ever elected in the US.

    Special guest Clell Drumheller, the southern area states chairman for the national Constitution Party, has new information to share that will show how real the possibility of an Obama and McCain decertification will be, opening the way for Chuck Baldwin to the White House. Now more than ever Americans have reason to hope!

    Tune into RTR Radio and get the news tonight at 10pm Eastern and 7 Pacific.

    Stay Tuned

  2. archtop says:

    Very interesting post AJ. I think that GOP turnout will be as high as it has ever been in recent memory, if, for no other reason, to combat the possibility of single party rule in Washington. In addition, the choice of Sarah Palin as VP really has the base jazzed, and I think you’ll send people coming out of the wood work just to vote for her. Combine this with the arrogance of Obama and the dead head media saying (under their breath) that “it’s over” – which should help keep some Obama voters home! – and I think you will see relatively even percentages of republicans and democrats, at least in the battleground states. It will come down to what the independents (like me – I’m officially undeclared) do. And I think they’ll end up breaking for McCain (which is what I’m expecting in NH).

    I don’t think that people understand that this isn’t 2006! Many conservatives stayed home in 2006 for various reasons, which led to many close contests going to the democrats (plus it was an off-year election, which usually favors the party out of power). That ain’t gonna happen in 2008 – turnout from the conservative base will be very high (hopefully historically high).

  3. archtop says:

    OK…for those who want the *** real **** story on polling statistics, Iowahawk has it all figured out!


  4. […] Let me be clear here – the Democrats alone cannot be generating that kind of voter interest. 81% means both sides are excited and coming out to vote.  Say good-bye to those ridiculously biased voter turnout models! […]

  5. Redteam says:

    arch, thanks, that is funny and well worth the read.

  6. BarbaraS says:

    The polls are so crowded in early voting and will be an avalanche on Tuesday Nov. 4 that I think the college kids will not trouble to vote. I could be wrong but they don’t like to stand in line for hours waiting to vote. In 2004 I voted on election day and spend hours in line. This year I voted early and had comfortable seating but it still was drawn out. Young people want to walk in and vote and be done with it. Instant gratification. This ain’t gonna happen. However, absentee ballots are another story.

  7. […] but it is just strange to see how slow some light bulbs are turning on in the news media. As I noted last night, the poll confusion is not confusing at all.  There are two families of polls out there and every […]