On the morning of Jan. 9, 2008, I sat down for what I knew would be a rough television interview. The night before, Hillary Clinton had defeated Barack Obama in the New Hampshire primary. It was a shock to most observers because the polling averages had projected Obama would win by eight points. My final poll had shown Obama up by seven.
As I saw it, it was the worst night for pollsters since the Dewey defeats Truman debacle in 1948. During the interview, I talked about an early review of what we had missed and why. I explained that part of the problem was we’d never had the Iowa caucus and New Hampshire primary so close together. Looking back, our daily tracking numbers had shown significant movement towards Clinton and we should have paid more attention to the trend.
After getting beaten up for a few minutes over the big polling mistake in New Hampshire, I was completely blown away by the very last question: “What do your polls say about next week’s Republican primary in Michigan?” It was more than a little surreal to go from what went wrong and how can we ever trust polls again to what do your numbers say about next week.
Since last Tuesday night, I’ve thought a lot about that long-ago interview. Nothing has changed. I watched countless pundits complain about how the polls messed up again. And yet I know that they will eagerly devour whatever polls are released for the upcoming Georgia run-off elections. Not only that, they will also eagerly hype any polls that confirm their favorite narrative while pretending other data doesn’t exist. Then, they will be shocked if things don’t work out as they envision.
It’s not that pollsters are blameless in all this. Our industry happily feeds the media appetite for more election-forecasting data. And we continue to do so even when the results are absurd. It boggles the mind that a Washington Post/ABC poll was released showing Biden up by 17 in Wisconsin.
Public pollsters act like a bartender who keeps pouring whiskey for a drunk customer and is then surprised when the customer wrecks his car on the way home. Not surprisingly, therefore, a lot of voters see polls as the problem. A post-election survey I conducted confirmed that pollsters today have no more credibility than journalists. In fact, pollsters are viewed a lot like reporters. Most think our work is designed to influence campaigns rather than report on them. That hurts.
This is obviously a problem for the polling industry. It’s something we need to address.
However, there is a more serious problem facing the nation. The failures of the larger election forecasting industry are now threatening the legitimacy of our nation’s governing institutions.
What went wrong
The election forecasting industry begins with media coverage of campaigns and culminates with the spectacle of election night television specials. From beginning to end, the focus is on candidates and campaigns rather than voters. Countless news stories this year were written about campaign strategy and fundraising. When Michael Bloomberg announced he would spend $100 million against Donald Trump in Florida, The Washington Post announced it as “a massive late-stage infusion of cash that could reshape the presidential contest.” The hope was it would “prompt enough early voting that a pro-Biden result would be evident soon after the polls close.”
Obviously, things didn’t work out that way.
Because they focused so much on campaigns and candidates, much of the election forecasting industry missed what was happening in America. In May, 23% of voters believed their personal finances were getting better. That grew to 27% by mid-October and 29% right before the election. That’s the same trend that helped Barack Obama get reelected eight years ago and it almost certainly helped Donald Trump close the gap this year. Data also showed that pessimism about the pandemic was declining.
Those two data points should have caused forecasters to question the more extreme projections of a big “Blue Wave.” But most didn’t see it because they thought the elections were all about what the campaigns said and did.
The failures of the larger election forecasting industry are now threatening the legitimacy of our nation’s governing institutions.
That disconnect is exactly what I saw on election night this year. From my perspective, the actual election results were well within the range of what the polling data suggested. Prior to the election, I had written that the best-case scenario for Republicans would be “a toss-up with the presidency determined by a few key states … places like Pennsylvania, Michigan and Wisconsin.” But that scenario would “involve weeks of chaos before the final outcome can be determined.”
My data suggested that the more likely scenario seemed to be a modest victory for Joe Biden. And as the returns came in on election night, that’s what the numbers showed. When all is said and done, it looks like the former vice president will win the popular vote by about five points. I like that number because my final national poll showed that with a strong Republican turnout, Biden would win by five.
To be clear, my numbers weren’t perfect and some of my state polls missed the mark. But looking at all the polling data — mine and that provided by others — should have prepared all election forecasters for what happened on Nov. 3. Instead, the celebrity analysts missed it because they disregarded or discounted any data that didn’t fit their conventional wisdom.
That led to a wildly misleading night of media coverage. At times, we saw graphics showing President Trump with a massive lead in Michigan while implying that most of the votes had been counted. But that was a completely false impression. The betting markets provided a great measure of just how misleading it was. At one point, gamblers gave President Trump a nearly 80% chance of being reelected. That was never the case.
At one point, gamblers gave President Trump a nearly 80% chance of being reelected. That was never the case.
Part of the problem is that the networks cover election night in much the same way they did half-a-century ago when Walter Cronkite sat in the anchor chair. Sure, the sets are more elaborate, and we can watch in color, but the basic coverage is unchanged. We see graphic tote boards showing the latest numbers in a state with a smaller number showing what percentage of precincts are reporting.
That made sense in earlier times when just about everyone voted in person on Election Day. If most of the precincts had reported their vote, most of the total vote was probably counted. But that approach didn’t make any sense in the midst of an unprecedented pandemic when more than 100 million people voted early. Every now and then, CNN did show an estimate of how many mail-in ballots had been counted. But it came across as more of an afterthought rather than a core part of the discussion.
This was especially significant because polling showed those who voted by mail favored former Vice President Biden by an enormous margin. Those who voted early in person favored Biden by a lesser amount. Among those who voted in-person on Election Day, President Trump was the clear winner.
Given this reality, why didn’t the networks main graphics show who was ahead and by how much along with tables showing how many in-person and mail-in votes remained to be counted? That would have made it much easier for voters to understand.
Think of it this way. If you are in a sports bar and a football game is on, the graphics make it easy to follow even when you can’t hear the sound. If you see that your team is up by six points but it’s early in the first quarter, you know there’s a long way to go. On the other hand, if your team is up by six and has the ball with two minutes left in the game, you can feel pretty confident.
In places like Michigan and Pennsylvania this year, the graphics would have shown President Trump with the early lead but a long way to go. And, rather than feeling cocky, his supporters would have been appropriately worried by the number of mail-in ballots still to be counted.
I suspect that there are two reasons this didn’t happen. The first is inertia — this is the way we’ve always covered election night. But the second is more damning. I don’t think the networks really want to make it easy for voters to figure out. Instead, they enjoy the façade of putting up numbers that can be properly deciphered only by their experts. Among other things, this then sets up their experts to explain what it is that voters were trying to express.
‘It has never worked well’
As I reflected further on what went wrong, I thought back to the afternoon of Nov. 2, 2004. I was in a Washington, D.C., hotel preparing to offer election commentary for the BBC. My polling that year showed that President Bush was expected to win reelection by about 3 points. Yet, that afternoon, the exit polls leaked and projected a massive victory for Sen. John Kerry. One of my competitors rushed out an election morning poll showing the same thing.
I was sick to my stomach. These reports showed that all of my polling — both national and state-by-state — were way off. I dreaded having to spend the evening explaining that while live on camera. It took several hours before Michael Barone pointed out the insight that turned the night around. The early vote totals turned out to be much better for Bush than the exit polls suggested. The knot in my stomach began to loosen. It turned out that the exit polls were wrong, and I could breathe easier.
But for the audience at home, it was just one more embarrassing failure of election night coverage. Those who tuned in early certainly got the impression that John Kerry was about to win the election. That mistake came just four years after we had witnessed the fiasco of television networks initially calling the decisive state of Florida for Al Gore. That call was rescinded, and Florida was called for George W. Bush. That too was rescinded. Along the way, Gore conceded and then rescinded his concession. It took 37 days for the election to be settled and conspiracy theorists dispute the result to this day.
What I came to realize is that the problem is much bigger than the polls being off in 2016 and 2020. It’s not a case of an industry that was doing great until the pollsters messed up in back-to-back elections.
Whatever else it may, this is not an industry that has worked well until Donald Trump decided to run for president. It has never worked well.
In fact, contrary to the myth, the polling averages were further off in 2012 than they were in 2016. President Obama was reelected by a wider margin than projected by all but one poll over the final five weeks of the campaign. However, in that election, the polling error didn’t change the expected outcome. Obama won and the narrative wasn’t disrupted. So, there was nothing to talk about.
Put that in the context of four years ago. It’s likely that we wouldn’t have heard any complaints about 2016 polling except for the fact that Donald Trump won Wisconsin, Michigan and Pennsylvania by a combined total of 78,000 votes. Had Clinton won by that same narrow margin, the election night coverage would have simply focused on celebrating the first woman president of the United States. Nothing would have been said about the polling.
The bottom line is that the election forecasting industry has had problems during every election of the 21st century. In fact, the exit polls also were way off in the 1992 election. But we didn’t hear much about it in the pre-internet era. Whatever else it may, this is not an industry that has worked well until Donald Trump decided to run for president. It has never worked well.
The path forward for pollsters
As for pollsters, there are things we can learn from election 2020 to do our job better. We should certainly do so.
But we must never forget that predicting which politician will win an election is not the primary purpose of polling. Our industry should seek to fairly measure and articulate the views of the public in terms that even a politician can understand. I will talk about some of those issues and how we can do a better job in the coming weeks.
For now, though, I simply want to point out that better last-minute polling will not fix the election forecasting industry. That industry as it currently exists should be scrapped and rebuilt from beginning to end.
Up front, early campaign reporting should focus more on understanding voters and less on what the campaigns want to sell them. At the end, there should be election month rather than election night coverage. And, recognizing the voting-by-mail is the new normal, forecasters should make it clear that the last day of election month is the last day that ballots can be received and counted.
I have lots of thoughts about this, but pollsters can’t fix the election forecasting industry. However, we can and should change our relationship with it. We should act like a responsible bartender who refuses to serve that last shot of whiskey and brings a cup of coffee instead.
Earlier this year, a media client asked me to conduct a last-minute poll on a competitive governor’s primary. I encouraged the client to rethink their request because the results would be roughly the same as our previous survey. My report would basically say the race was close and it would come down to turnout. Some may consider me foolish for turning down the money, but it was the right call. Doing that last-minute poll would have added nothing of value to the public understanding of the race.
Pollsters can’t fix the election forecasting industry. However, we can and should change our relationship with it.
From a broader perspective, there is little value to be gleaned from most last-minute horse race polls. In mid-October I conducted a poll on the Utah 4th Congressional race. We found that the Republican candidate had a 46% to 45% advantage in a poll with a 3.5 percentage point margin of error. We also showed that the Democrat could win with a slightly different turnout model. In other words, it was a pure toss-up. Today, the candidates are less than a percentage point apart and waiting for the final ballots to be counted. No amount of last-minute polling would have changed that.
It’s time for responsible public pollsters to stop conducting state and district polls during the final week or two of a campaign. At best, such polls add nothing of value to the public dialogue. At worst, they generate enormously misleading information that can impact a campaign.
Another reason to avoid late polling is that the final week or so of a campaign is the hardest time to field an accurate poll. That’s because campaigns are bombarding people with never ending texts and phone calls in desperate get-out-the-vote campaigns. Pollsters have a hard enough time trying to get people to take our surveys in normal times! When the campaigns are in overdrive, it’s far more challenging.
By offering fewer last-minute polls, we would encourage forecasters to follow the example of some good role models. Both Real Clear Politics and Larry Sabato’s Crystal Ball use polls as an important source of information. But they use lots of other information along with some independent judgment.
Real Clear Politics always takes an appropriately cautious approach in their projections. This year listed a dozen states as toss-ups on Election Day. The wisdom of their effort was highlighted by the fact that some of the toss-ups ended up going to Trump and other going to Biden. That’s the way it should be when toss-ups are properly defined.
Sabato’s team relied upon lots of reporting and data to end up with an amazingly accurate forecast. They are the only group I know that correctly called Florida for Trump and Georgia for Biden. But they also presented all of their forecasts in a manner acknowledging the uncertainty involved.
Instead of flooding the zone with a deluge of last-minute horse-race polls, public pollsters should offer more data designed to help forecasters and politicians understand America. Importantly, these should be about attitudes and perceptions rather than policies and campaign positions. We need to frame questions in the language of the American people rather than the language of American politics.
Instead of flooding the zone with a deluge of last-minute horse-race polls, public pollsters should offer more data designed to help forecasters and politicians understand America.
A simple example of this distinction can be found in the recent volumes of polling about whether or not Roe v. Wade should be overturned. To political activists and campaigns, that policy has a specific meaning. But my polling found that 56% of voters do not know what would happen if Roe v. Wade is overturned. So, if we want to find out what voters really think, we can’t use the labels that are meaningful only in official Washington.
Finally, two or three weeks before Election Day, public pollsters should conduct larger and more comprehensive surveys. We should offer a voter-centric view of the race, measuring underlying attitudes more than attempting to define likely voters. We should certainly ask about the horse race, but never forget that elections are supposed to be more about the voters than the candidates.
I’ll be taking this approach with our surveys leading up to the Georgia Senate run-offs. As with all new things, it will be a work in progress. We’ll be careful to release the results in a way to give voters and forecasters a clearer understanding of what to expect..
Ending the steady flow of last-minute polls would be a sobering treatment for election forecasters in search of their next fix. But it would be good for the polling industry, and good for America.
Scott Rasmussen is an American political analyst and digital media entrepreneur. He is the author of “The Sun is Still Rising: Politics Has Failed But America Will Not.”