Feeds:
Posts
Comments

Posts Tagged ‘five thirty eight’

I have often expressed my admiration for Nate Silver’s wonderful polling aggregate/analysis website, 538. However, there are at least two other very popular sites which aggregate polls : Real Clear Politics and Pollster. How did Nate do in comparison to them?

First of all, all three sites were pretty good in calling the winner of each state. 538 only got Indiana — and possibly one of Nebraska’s congressional districts — wrong. Nate called these two slightly for McCain, and Obama won. RCP missed these two as well; they also got North Carolina wrong. Pollster missed both these as well as Missouri. However all these states were really close and it would be unfair to read too much into them.

Time for a more thorough analysis. I will do the following thing – list all the IMPORTANT states, see by how much 538, RCP and Pollster were off in their predicted margins, and calculate who made the least error.

I define an important state as any state in which the final actual margin was 15 points or less. Thus, this definition includes all the swing states as well many of the ‘safe’ states. The reason I only take these states is that it seems somewhat more important for a polling site to get these right. Besides, it saves me some labor.

State            EVs       Obama Margin       538 error      RCP error   Pollster error

AZ                10                      -8.6                  3.7               5.1               3.7

CO                 9                        7.8                  1.2               2.3                 .2

FL                 27                        2.5                   .8                 .7                 .8

GA                15                       -5.5                 1.8               1.5               2.6

IN                 11                          .9                 2.4               2.3                2.1

IA                   7                         9.3                 2.4               6.0               3.6

MN               10                       10.2                   .1                .4                2.2

MS                  6                       -13.8                 2.5             2.4               3.3

MO                 11                      -.2                     0                  .5               1.3

MT                  3                        -2.5                   .2                1.3               .3

NV                  5                        12.4                 7.5               5.9               5.3

NH                  3                         9.5                   .3                1.1              2.4

NJ                   15                      14.6                 .1                 .9                 .9

NM                  5                       14.7                 5.0                 7.4             5.8

NC                 15                          .4                  .6                  .8                0

ND(*)              3                        -8.6               5.9                9.6               9.3

OH                 20                         4                   .6                 1.5               .9

PA                  21                      10.3               2.2                 3.0               3.1

SC                  8                        -8.9                  .8                1.1                .4

SD                   3                       -8.5                  .2                 .2                 .5

TX                  34                     -11.7                 .9                1.3               1.4

VA                 13                          5.5                 .1                 1.1                .1

WV                  5                       -13.1              3.3                 4.1              1.7

WI                 10                         13.9              2.4                 2.9              1.9

Ok, so who is the winner?

One way to see that is to simply take the median error. Pollster has a median error of 1.8, while RCP gets a surprisingly good 1.5. But 538? It’s median error is an amazing 1.05.

A slightly different measure is to take the root-mean-square error. This number is 3.1 for Pollster and 3.58 for RCP. The clear winner, once again, is 538, with a root mean square error of only 2.71.

Other methods, such as weighting the errors by electoral votes, only increases 538’s advantage.

Nate Silver rocks.

Advertisements

Read Full Post »

Andrew Sullivan is effusive in his praise of Five Thirty Eight, the polling aggregation and analysis website created by Nate Silver:

The only state their model got wrong was Indiana, where they expected a narrow Obama loss. He won the state by a hair. Nate Silver owned this election on the polling front: one young guy with a background in baseball stats beat out the mainstream media in a couple of months. And he beat out the old web: I mean if you consider the total joke of Drudge’s recent coverage and compare it with Silver’s, you realize that the web is a brutal competitive medium where only the best survive – and they are only as good as their last few posts.

If you want to know why newspapers are dying: that’s why. They’re just not as good as the web at its best. This election proved that beyond any doubt. For the record, I think the WSJ and the WaPo and the NYT and the Anchorage Daily News rocked in this election. Most of the rest of the old media: not so much.

I completely agree. Five Thirty Eight revolutionized the polling analysis business and was far and away my favourite haunt during the elections.

There’s something else that I am happy about. Nate’s detailed posts were full of conditional probabilities, Bayesian analysis and related tools; yet, they were presented in a layman’s language. Modern probability is one of the core ingredients of rational thought. In its concise and practical demonstration of the power of numbers, Five Thirty Eight, I suspect, has taught a lot of people the basics of probability and the importance of cool, rational thought.

Read Full Post »

According to the latest Five-Thirty-Eight projections, the Democrats’ odds of getting a filibuster-proof 60 seats in the senate are getting longer (currently Nate gives a less than 25% chance of this happening).

A filibuster proof Senate majority by a party that also has control of the Presidency and the House is always a disaster. I am glad this looks unlikelier by the day.

Read Full Post »

The polling seems to be getting stronger for Obama with every passing day. Five Thirty Eight currently gives Obama a 95.8% chance of winning the election, and that is after accounting for some tightening of the polls in the coming weeks.

More astoundingly, no fewer than seven current polls show Obama with a double-digit lead nationally.

Nate Silver writes:

It’s fairly unusual for a candidate to have such a sustained run of momentum so deep into the campaign cycle. And it does appear to be real momentum, with some real feedback loops: the worse McCain’s poll numbers become, the more desperate his campaign looks, and the more desperate his campaign looks, the worse his poll numbers become.

McCain now has to go on a run of his own, a large enough run to wipe at least 8 points off of Obama’s lead, and perhaps more like 9 or 10 to cover his inferior position in the Electoral College and the votes that Obama is banking in early and absentee balloting. It is imperative that McCain does not just draw tomorrow night’s debate, does not just win a victory on points, but emerges with a resounding victory, the sort that leaves the spin room gasping for air. Failing that, we are getting into dead girl, live boy territory.

The question is no longer whether Obama will win, but by how much. The electoral college has a size of 540 EVs. If things continue the way they are, Obama will end up winning close to 400 EVs which would be a stunning landslide.

McCain has to completely destroy Obama in tomorrow’s debate to get his campaign back on track. Unfortunately for him, that is about as likely as Sarah Palin writing a dissertation on the theory of evolution.

Read Full Post »

I love Nate Silver!

Read Full Post »

As I have emphasized before, Virginia, Colorado and Ohio are the three crucial players in this election. As long as Obama wins one of them, he will be the next president. If he loses all three, then he probably won’t.

Nate Silver gives the precise numbers:

Colorado, Virgina and Ohio remain the three principal focal points of Obama’s offense. Our model makes Obama a very slight favorite in Colorado with a 53.0% probability of winning. Obama wins the election 95.9% of the time that he wins Colorado in our simulations.

Obama remains a small underdog in Virginia, winning that state 43.1% of the time. But he wins the electoral vote 99.3% of the time that he does win Virginia.

And Obama is a slightly longer underdog in Ohio, winning there 39.6% of the time. However, it is nearly impossible for him to lose the election when he wins Ohio, as he takes the election 99.8% of the time that Ohio swings his way.

By the way, Nate’s site, Five Thirty Eight, is far and away the best place to get the precise trends for this election. There are many websites which aggregate opinion polls but Nate beats them all. He uses a sophisticated model, whereby he ranks different polling agencies and uses this to assign weights to the various available opinion polls. He also takes into account underlying demographics of each state. Using these, he assigns probabilities for each of the candidates winning each state and runs 10000 random simulations that take these probabilities into account. The result is a day-by-day update of the state of the nation. If you are obsessed with knowing how these race is behaving, Five Thirty Eight is what you need to watch.

Read Full Post »