Feeds:
Posts
Comments

Posts Tagged ‘statistics’

I have often expressed my admiration for Nate Silver’s wonderful polling aggregate/analysis website, 538. However, there are at least two other very popular sites which aggregate polls : Real Clear Politics and Pollster. How did Nate do in comparison to them?

First of all, all three sites were pretty good in calling the winner of each state. 538 only got Indiana — and possibly one of Nebraska’s congressional districts — wrong. Nate called these two slightly for McCain, and Obama won. RCP missed these two as well; they also got North Carolina wrong. Pollster missed both these as well as Missouri. However all these states were really close and it would be unfair to read too much into them.

Time for a more thorough analysis. I will do the following thing – list all the IMPORTANT states, see by how much 538, RCP and Pollster were off in their predicted margins, and calculate who made the least error.

I define an important state as any state in which the final actual margin was 15 points or less. Thus, this definition includes all the swing states as well many of the ‘safe’ states. The reason I only take these states is that it seems somewhat more important for a polling site to get these right. Besides, it saves me some labor.

State            EVs       Obama Margin       538 error      RCP error   Pollster error

AZ                10                      -8.6                  3.7               5.1               3.7

CO                 9                        7.8                  1.2               2.3                 .2

FL                 27                        2.5                   .8                 .7                 .8

GA                15                       -5.5                 1.8               1.5               2.6

IN                 11                          .9                 2.4               2.3                2.1

IA                   7                         9.3                 2.4               6.0               3.6

MN               10                       10.2                   .1                .4                2.2

MS                  6                       -13.8                 2.5             2.4               3.3

MO                 11                      -.2                     0                  .5               1.3

MT                  3                        -2.5                   .2                1.3               .3

NV                  5                        12.4                 7.5               5.9               5.3

NH                  3                         9.5                   .3                1.1              2.4

NJ                   15                      14.6                 .1                 .9                 .9

NM                  5                       14.7                 5.0                 7.4             5.8

NC                 15                          .4                  .6                  .8                0

ND(*)              3                        -8.6               5.9                9.6               9.3

OH                 20                         4                   .6                 1.5               .9

PA                  21                      10.3               2.2                 3.0               3.1

SC                  8                        -8.9                  .8                1.1                .4

SD                   3                       -8.5                  .2                 .2                 .5

TX                  34                     -11.7                 .9                1.3               1.4

VA                 13                          5.5                 .1                 1.1                .1

WV                  5                       -13.1              3.3                 4.1              1.7

WI                 10                         13.9              2.4                 2.9              1.9

Ok, so who is the winner?

One way to see that is to simply take the median error. Pollster has a median error of 1.8, while RCP gets a surprisingly good 1.5. But 538? It’s median error is an amazing 1.05.

A slightly different measure is to take the root-mean-square error. This number is 3.1 for Pollster and 3.58 for RCP. The clear winner, once again, is 538, with a root mean square error of only 2.71.

Other methods, such as weighting the errors by electoral votes, only increases 538’s advantage.

Nate Silver rocks.

Advertisements

Read Full Post »

Andrew Sullivan is effusive in his praise of Five Thirty Eight, the polling aggregation and analysis website created by Nate Silver:

The only state their model got wrong was Indiana, where they expected a narrow Obama loss. He won the state by a hair. Nate Silver owned this election on the polling front: one young guy with a background in baseball stats beat out the mainstream media in a couple of months. And he beat out the old web: I mean if you consider the total joke of Drudge’s recent coverage and compare it with Silver’s, you realize that the web is a brutal competitive medium where only the best survive – and they are only as good as their last few posts.

If you want to know why newspapers are dying: that’s why. They’re just not as good as the web at its best. This election proved that beyond any doubt. For the record, I think the WSJ and the WaPo and the NYT and the Anchorage Daily News rocked in this election. Most of the rest of the old media: not so much.

I completely agree. Five Thirty Eight revolutionized the polling analysis business and was far and away my favourite haunt during the elections.

There’s something else that I am happy about. Nate’s detailed posts were full of conditional probabilities, Bayesian analysis and related tools; yet, they were presented in a layman’s language. Modern probability is one of the core ingredients of rational thought. In its concise and practical demonstration of the power of numbers, Five Thirty Eight, I suspect, has taught a lot of people the basics of probability and the importance of cool, rational thought.

Read Full Post »

I love Nate Silver!

Read Full Post »

As I have emphasized before, Virginia, Colorado and Ohio are the three crucial players in this election. As long as Obama wins one of them, he will be the next president. If he loses all three, then he probably won’t.

Nate Silver gives the precise numbers:

Colorado, Virgina and Ohio remain the three principal focal points of Obama’s offense. Our model makes Obama a very slight favorite in Colorado with a 53.0% probability of winning. Obama wins the election 95.9% of the time that he wins Colorado in our simulations.

Obama remains a small underdog in Virginia, winning that state 43.1% of the time. But he wins the electoral vote 99.3% of the time that he does win Virginia.

And Obama is a slightly longer underdog in Ohio, winning there 39.6% of the time. However, it is nearly impossible for him to lose the election when he wins Ohio, as he takes the election 99.8% of the time that Ohio swings his way.

By the way, Nate’s site, Five Thirty Eight, is far and away the best place to get the precise trends for this election. There are many websites which aggregate opinion polls but Nate beats them all. He uses a sophisticated model, whereby he ranks different polling agencies and uses this to assign weights to the various available opinion polls. He also takes into account underlying demographics of each state. Using these, he assigns probabilities for each of the candidates winning each state and runs 10000 random simulations that take these probabilities into account. The result is a day-by-day update of the state of the nation. If you are obsessed with knowing how these race is behaving, Five Thirty Eight is what you need to watch.

Read Full Post »

We had 1069 pageviews for the week ending June 22. This makes it our best week yet and the first time we have crossed the 1000 pageviews/week mark.

Read Full Post »

The exit polls have been pretty off the mark this election season but in a consistent manner. As Brendan Loy notes in this post, the polls have been typically off by 7-8 points in Obama’s direction. This pattern was repeated yesterday — Clinton won Pennsylvania by 9 points when the CNN exit polls earlier in the day predicted she should win by 2.

I guess this is due to a combination of two factors.

1) The pollsters are clueless about weighted sampling and ignorant about the demographics of this contest (or more likely, simply too lazy to implement them): Obama does much better among the young, the affluent, the urban and the educated. A polling strategy that picks up a disproportionate number of such individuals, as would happen, for instance, if the pollsters spent most of their time in the big cities or other easily accessible parts of a state, is bound to go wrong.

2) People are not truthful when asked who they voted for: It may be true that a lot of whites vote for Clinton, then lie that they voted for Obama (so as to not appear racist?)

 

Read Full Post »