Despite Innovation, Poll-Based Election Forecasting Was Pretty Much All Wrong

No matter who you voted for, the mainstream consensus is that President-Elect Donald Trump’s Tuesday victory was a huge upset. In the age of digital media, there are a lot of ways you could have watched this unfold. Perhaps you were refreshing a map on a news website’s front page as it turned from white into a red-blue mosaic. You might have watched a tool, like FiveThirtyEight’s live election forecast, completely flip its predictions as the night went on. Or maybe you watched CNN’s John King zoom in and out of a map and point at congressional districts for hours on end.

One thing these tools and information sources have in common is that they don’t tell you anything about the outcome until the polls close. Once real precincts start reporting real ballot counts, news organizations start reporting the trajectory of a state’s electoral votes. That means audiences are left in total suspense until around 7 p.m. E.T.

This year, Slate and Vice tried something a little different. The two online magazines partnered with data startup VoteCastr to give real-time projections of outcomes in several key states. The method is tricky and experimental, so neither site was claiming to report definitive outcomes. Given that the VoteCastr model ended up wrong in five out of seven of the states it was tracking, that was a wise decision. But I’ll get to that.

According to Slate, The model works using a combination of three pieces of information. VoteCastr conducted pre-election sample polls in seven battleground states. It also tracked who voted early in these states and used demographic data to predict each voter’s choice. Finally, VoteCastr’s field workers gathered real-time data about voter turnout in a sample of precincts throughout Election Day.

The point of all of this is to be able to see, within a margin of error, the trajectory of the election before polls close. Slate’s Josh Vorhees admits that it’s a controversial experiment to run. “It will break a decadeslong journalistic tradition whereby media outlets obey a self-imposed embargo on voting information under the unproven theory that it might depress turnout on Election Day,” he wrote on Saturday.

I will go out on a limb and say the fact that two websites reporting election predictions a few hours before polls closed probably didn’t affect the election outcome. But it should be noted that when Slate and Vice stopped updating their VoteCastr models, they was far from right in most of the states they was tracking.

VoteCastr only tracked seven states: Florida, Iowa, Nevada, New Hampshire, Ohio, Pennsylvania and Wisconsin. It projected that Hillary Clinton would win all of these states (and, if she had, she would now be the president-elect). As we now know, Clinton only won Nevada and New Hampshire’s combined 10 electoral votes out of the 93 being tracked.

While real-time projections are relatively new, VoteCastr had one main thing in common with other forecasting tools: It was wrong. For example, FiveThirtyEight’s interactive live election forecast started the night by giving Clinton a 73 percent chance of winning the election. At 10:23 p.m., her chances dipped below 50 percent for the first time, and by midnight it was clear that her path was all but gone.

Donald Trump announced his candidacy nearly a year and half ago and won his party’s nomination nearly four months ago. Journalists and pollsters have been spending much of that time confidently talking about how badly he was going to lose. Obviously, something went wrong.

“America is seen as a leader in the area of data science and polling, so there’s a lot of concern that so many American prognosticators have gotten it wrong in the field,” says Jennifer Lambert, a data scientist at the State Department in an interview with MediaFile. 

Polling is an entire industry. Every election in recent memory has produced an immense amount of polling data. Technology has facilitated the way we receive, process and visualize all the information. And despite the fact that any pollster, statistician, political reporter or news savvy person will tell you to take polls with a grain of salt, it is the primary driver of election predictions. This election will have people reconsidering many common conventions about politics. Perhaps the ways we predict the outcomes will be high on the list.

Brittany Gellerman contributed reporting.

Leave a Reply

Your email address will not be published. Required fields are marked *