Daily Politics News Magazine
Covering Politics, Candidates & Issues from City Hall to Capitol Hill

The Death Of Polling Is Greatly Exaggerated


Polls probably aren’t at the top of your mind right now. We’re more than four months removed from the 2020 election, and we still have almost 20 months to go until the midterms.

That’s why it’s the perfect time to launch the latest update to our FiveThirtyEight pollster ratings, which we just released today! We’d encourage you to go check out the ratings — as well as our revamp of the interactive featuring individual pages for each pollster with more detail than ever before on how we calculate the ratings.

No, but seriously … I think it’s nice to have a little distance from the heat of an election cycle when talking about polls. When I first looked at the performance of the polls in November, it came after the election had just been called for Joe Biden — and after several anxious days of watching states slowly report their mail ballots, which produced a “blue shift” in several states that initially appeared to have been called wrongly by the polls. We also didn’t yet know that Democrats would win control of the U.S. Senate, thanks to a pair of January runoffs in Georgia. Meanwhile, then-President Donald Trump was still refusing to concede. In that environment, a decidedly mediocre year for the polls was being mistaken for a terrible one when that conclusion wasn’t necessarily justified.

So what does 2020 look like with the benefit of more hindsight — and the opportunity to more comprehensively compare the polls against the final results? The rest of this article will consist of four parts:

  • First, our review of how the polls did overall in 2020, using the same format that we’ve traditionally applied when updating our pollster ratings.
  • Second, a look at which polling firms did best and worst in 2020.
  • Third, our evaluation of how the polls have performed both in the short run and long run based on various methodological categories. And we’ll announce an important change to how our pollster ratings will be calculated going forward. Namely — breaking news here — it’s no longer clear that live-caller telephone polls are outperforming other methods, so they’ll no longer receive privileged status in FiveThirtyEight’s pollster ratings and election models.
  • Finally, some other, relatively minor technical notes about changes in how we’re calculating the pollster ratings. Some of you may want to skip this last part.

How the polls did in 2020

Our pollster ratings database captures all polls conducted in the final 21 days of presidential primary elections since 2000, as well as general elections for president, governor, U.S. Senate and House since 1998. It also includes polls on special elections and runoffs for these offices. So, technically speaking, the data you’ll see below covers the entire 2019-20 election cycle, though the majority of it comes from elections on Nov. 3, 2020. We’re also classifying the Georgia Senate runoffs, held on Jan. 5, 2021, as part of the 2019-20 cycle.

First up, let’s start with our preferred way to evaluate poll accuracy: calculating the average error observed in the polls. We do this by comparing the margin between the top two finishers in the poll to the actual results; for example, if a poll had Biden leading Trump by 2 percentage points in a state and Trump actually won by 4 points, that would be a 6-point error. 

In the table below, we calculate the average error for all polls in our database for 2019-20 and how that compares with previous cycles, excluding polling firms banned by FiveThirtyEight and weighting by how prolific a pollster was in a given cycle. We also break out the polling error by office.

It was a mediocre year for the polls in 2020

Weighted-average error in polls in final 21 days of the campaign

Presidential State Level
Cycle Primary General Governor U.S. Senate U.S. House Combined
1998 8.1 7.5 7.1 7.7
1999-2000 7.9 4.4 4.9 6.0 4.4 5.6
2001-02 5.3 5.4



Read More: The Death Of Polling Is Greatly Exaggerated

Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments

Get more stuff like this
in your inbox

Subscribe to our mailing list and get interesting stuff and updates to your email inbox.

Thank you for subscribing.

Something went wrong.