Nate's recent Rasmussen bias: I sure hope New York Times’ Nate Silver of 538 knows about outliers

By Sam Foster

The brilliant Nate Silver may be just about the best in the polling business, but that doesn't mean he's not subject to insufferable hackery. For one reason or another, Nate has is in for Scott Rassmusen lately and Nate's bias has been spilling into the pages of NY Times this election season.

It's not always been this way
. Nate has defended Rasmussen to some degree in the past. Rasmussen was right on the money during the 2008 Presidential race and very accurate during 2009 races. That's not to say that Nate Silver likes or agrees with Scott's methodology, but Nate has for a large part, dodged the leftwing narratives so prevalent in the newspaper he now associates.

However, Nate Silver caught my eye when he "doth protested a bit too much" with Rasmussen's poll for NY's Gubernatorial race showing Paladino within 11 of Cuomo. Nate accused Rasmussen of spinning a narrative only to look completely foolish when the Q-Poll (who according to Nate was blessedly accurate) had the race even tighter (by 6) the very next day. Today's post on Rasmussen accuracy reeks of similar hackedness.

Nate made much ado of Rasmussen's worst poll ever in his analysis. The critique is germane, but not in the statistical sense. It would certainly be, by Nate's own god statistics, an outlier and as such should have been removed from Nate's accuracy model.

An outlier in statistics is a data point that is outside the norm and that certainly describes the worst poll in Nate's ten years. That's not to say that we ignore it. By all means, Scott Rasmussen ought to take the poll very seriously and find a reason for its inaccuracy. His methodology could be flawed, it could be poor polling questions or polling staff, he should look into why no one noticed it until after released, and it could even be a bad roll of the statistical die. Nate ought to do the same when considering the poll, but if Nate's just objectively looking at the data, he can't just arbitrarily ignore the fact that it is radically outside the norm for Rasmussen. Doing so skews his own model.

Since Nate doesn't publish his methodology here, I have no idea if he treated the poll as an outlier, but clearly he should. In fact, the shear number of polls conducted by Rasmussen over 100, only makes it more likely that one would occur (although it should also make Rasmussen more accurate, provided his methodology is sound). Failure to remove it is either a statistical faux pas or its just Nate being bias toward his "precious" data (kind of like trying to spin a narrative).

However, whether accidental or purposefully overlooked, its Nate's credibility that's at stake.


Don Surber makes the "what's the point of polls?" argument and proves guessing as accurate as Nate Silver's political calculus.

Via Memeorandum

1 comment:

  1. "Nate made much ado of Rasmussen's worst poll ever in his analysis."

    Seriously? In his 1,318 word article, he devoted 37 to the Hawaii poll. Perhaps my definition of "much ado" is different than yours.

    You also ignore all the other evidence Nate includes to support his claim of bad polling and bias. For example:
    --13 of 105 polls Rasmussen conducted in the last 3 weeks of the campaign missed the mark by 10 or more points
    --12/105 polls overestimated the Democrat margin by 3 or more points, while 55/105 polls overestimated the Republicatn margin by 3+ points).

    I'll grant you that Nate should have used the median when calculating his averages (and he might have done just that, but his post doesn't say). But the conclusion you draw on the entirety of the article looks like an overreach to me.


Commenting here is a privilege, not a right. Comments that contain cursing or insults and those failing to add to the discussion will be summarily deleted.