Feeds:
Posts
Comments

Posts Tagged ‘nate silver’

I have often expressed my admiration for Nate Silver’s wonderful polling aggregate/analysis website, 538. However, there are at least two other very popular sites which aggregate polls : Real Clear Politics and Pollster. How did Nate do in comparison to them?

First of all, all three sites were pretty good in calling the winner of each state. 538 only got Indiana — and possibly one of Nebraska’s congressional districts — wrong. Nate called these two slightly for McCain, and Obama won. RCP missed these two as well; they also got North Carolina wrong. Pollster missed both these as well as Missouri. However all these states were really close and it would be unfair to read too much into them.

Time for a more thorough analysis. I will do the following thing – list all the IMPORTANT states, see by how much 538, RCP and Pollster were off in their predicted margins, and calculate who made the least error.

I define an important state as any state in which the final actual margin was 15 points or less. Thus, this definition includes all the swing states as well many of the ‘safe’ states. The reason I only take these states is that it seems somewhat more important for a polling site to get these right. Besides, it saves me some labor.

State            EVs       Obama Margin       538 error      RCP error   Pollster error

AZ                10                      -8.6                  3.7               5.1               3.7

CO                 9                        7.8                  1.2               2.3                 .2

FL                 27                        2.5                   .8                 .7                 .8

GA                15                       -5.5                 1.8               1.5               2.6

IN                 11                          .9                 2.4               2.3                2.1

IA                   7                         9.3                 2.4               6.0               3.6

MN               10                       10.2                   .1                .4                2.2

MS                  6                       -13.8                 2.5             2.4               3.3

MO                 11                      -.2                     0                  .5               1.3

MT                  3                        -2.5                   .2                1.3               .3

NV                  5                        12.4                 7.5               5.9               5.3

NH                  3                         9.5                   .3                1.1              2.4

NJ                   15                      14.6                 .1                 .9                 .9

NM                  5                       14.7                 5.0                 7.4             5.8

NC                 15                          .4                  .6                  .8                0

ND(*)              3                        -8.6               5.9                9.6               9.3

OH                 20                         4                   .6                 1.5               .9

PA                  21                      10.3               2.2                 3.0               3.1

SC                  8                        -8.9                  .8                1.1                .4

SD                   3                       -8.5                  .2                 .2                 .5

TX                  34                     -11.7                 .9                1.3               1.4

VA                 13                          5.5                 .1                 1.1                .1

WV                  5                       -13.1              3.3                 4.1              1.7

WI                 10                         13.9              2.4                 2.9              1.9

Ok, so who is the winner?

One way to see that is to simply take the median error. Pollster has a median error of 1.8, while RCP gets a surprisingly good 1.5. But 538? It’s median error is an amazing 1.05.

A slightly different measure is to take the root-mean-square error. This number is 3.1 for Pollster and 3.58 for RCP. The clear winner, once again, is 538, with a root mean square error of only 2.71.

Other methods, such as weighting the errors by electoral votes, only increases 538’s advantage.

Nate Silver rocks.

Advertisements

Read Full Post »

Andrew Sullivan is effusive in his praise of Five Thirty Eight, the polling aggregation and analysis website created by Nate Silver:

The only state their model got wrong was Indiana, where they expected a narrow Obama loss. He won the state by a hair. Nate Silver owned this election on the polling front: one young guy with a background in baseball stats beat out the mainstream media in a couple of months. And he beat out the old web: I mean if you consider the total joke of Drudge’s recent coverage and compare it with Silver’s, you realize that the web is a brutal competitive medium where only the best survive – and they are only as good as their last few posts.

If you want to know why newspapers are dying: that’s why. They’re just not as good as the web at its best. This election proved that beyond any doubt. For the record, I think the WSJ and the WaPo and the NYT and the Anchorage Daily News rocked in this election. Most of the rest of the old media: not so much.

I completely agree. Five Thirty Eight revolutionized the polling analysis business and was far and away my favourite haunt during the elections.

There’s something else that I am happy about. Nate’s detailed posts were full of conditional probabilities, Bayesian analysis and related tools; yet, they were presented in a layman’s language. Modern probability is one of the core ingredients of rational thought. In its concise and practical demonstration of the power of numbers, Five Thirty Eight, I suspect, has taught a lot of people the basics of probability and the importance of cool, rational thought.

Read Full Post »

Here, in short, is what to expect tomorrow:

Who will win?

The outcome of the election is not in doubt. Barack Obama will be the next president of the USA.

When will we know the results in each state?

Check out this article by Nate Silver for a nice time-line of when polls close in each state and when one can expect the networks to call them.

How low can Obama’s tally possibly go?

A variety of factors can go against BHO tomorrow. It may turn out that the pollsters were overestimating the youth and black votes. Obama may suffer heavily from Coal-gate, especially in some of the midwestern and Appalachian states. There may be a unexpected emergence of some kind of Bradley effect.

Several states, including biggies like FL, MO and NC, remain too close to call. On a bad day, Obama could lose them all. However, some things are not in doubt. Obama will win all Kerry states. He will win IA and NM easily. And even if he has a very bad day, he will still carry CO and NV, albeit by a small margin.

Taking these into account, the lowest Obama can go is 278. And he needs only 269 EVs to be the next president. Obama getting 278 EVs corresponds to a scenario like below:

How high can Obama’s tally go?

A very good day tomorrow, fuelled by massive youth and black turnout, will mean that Obama not only carries all the swing states, but also states like GA, ND and MT. And if he ends up winning these deep red states, he will probably also triumph in AZ, McCain’s home state. However, I do not see him winning WV or AR, notwithstanding some old polls suggesting he was close there.

If you do the math, you will see that the highest Obama can go is 406 . That would be a massive blowout. See the map below for the scenario that leads to this number:

So, what’s the likely range?

A variety of factors make this election rather hard to call. However, let’s face it, the scnearios that lead to Obama geting over 400 EV’s, or keep him under 280, are unlikely. Obama will probably lose GA and AZ . And most likely, he will win VA. So with a fairly high probability, Obama’s final EV count will lie between 291 and 381. My personal hunch is that he will get towards the high end of this range.

And finally, this is just an election. It is useful to remember the following points, articulated by Michael Totten before the 2004 election:

People who vote for the other guy aren’t stupid, brainwashed, or evil. They are your friends and family. Someone you love will almost certainly cancel your vote. (My wife cancels out mine.)

If, by some chance, everyone you know votes for the loser it won’t mean the election was stolen. It will only show that you live in a bubble.

If this thing is close (the victor could easily win by 0.1 percent) try not to read too much into it. We’ll still be closely divided.

If the election doesn’t go your way, don’t pop off as though America were Guatemala under the generals. You’ll get lots of attention, but it won’t be the kind you want. People will laugh, not near you but at you.

To which, let me add: both Obama and McCain are big government statists. Nothing that happens tomorrow will prevent tragedies like this, this or this. Both Obama and McCain will use your money to stop you from enjoying pot, driving without a seatbelt or eating bacon dogs. So chill out and enjoy.

Read Full Post »

As I have emphasized before, Virginia, Colorado and Ohio are the three crucial players in this election. As long as Obama wins one of them, he will be the next president. If he loses all three, then he probably won’t.

Nate Silver gives the precise numbers:

Colorado, Virgina and Ohio remain the three principal focal points of Obama’s offense. Our model makes Obama a very slight favorite in Colorado with a 53.0% probability of winning. Obama wins the election 95.9% of the time that he wins Colorado in our simulations.

Obama remains a small underdog in Virginia, winning that state 43.1% of the time. But he wins the electoral vote 99.3% of the time that he does win Virginia.

And Obama is a slightly longer underdog in Ohio, winning there 39.6% of the time. However, it is nearly impossible for him to lose the election when he wins Ohio, as he takes the election 99.8% of the time that Ohio swings his way.

By the way, Nate’s site, Five Thirty Eight, is far and away the best place to get the precise trends for this election. There are many websites which aggregate opinion polls but Nate beats them all. He uses a sophisticated model, whereby he ranks different polling agencies and uses this to assign weights to the various available opinion polls. He also takes into account underlying demographics of each state. Using these, he assigns probabilities for each of the candidates winning each state and runs 10000 random simulations that take these probabilities into account. The result is a day-by-day update of the state of the nation. If you are obsessed with knowing how these race is behaving, Five Thirty Eight is what you need to watch.

Read Full Post »