Last week, I noted that the election forecasting industry has had problems during every election of the 21st century. The problem is far deeper than the current narrative of polls being off the mark in 2016 and 2020.
Despite that reality, some want to believe that the problems all revolve around the incredibly powerful brand of Donald Trump. Patrick Murray of the Monmouth University Polling Institute is no fan of President Trump’s, but he credits the Trump brand with almost mystical power over the polling industry. “It just seems to be that if the name ‘Donald Trump’ is on the ballot, all bets are off when it comes to the polls being right.”
If that’s the case, then the industry has nothing to worry about. Pollsters can keep on doing what they’re doing and everything will be OK if the president doesn’t run again in 2024.
I wish it were that simple.
Fans of the president, however, have a different explanation. Many are convinced that “shy Trump voters” explain why the polls were off in 2020. This concept is built around the idea that the president’s supporters are afraid to voice their opinion for fear of how others might react.
In the polling profession, Robert Cahaly of the Trafalgar Group is the biggest proponent of this view. He claims to have a methodology that corrects for social desirability bias and can fully uncover the hidden Trump vote. This year, he boldly projected outcomes in key states that were far more favorable to the president than either the conventional wisdom or polling averages.
I rarely comment on other polling firms directly, but I have tremendous respect for the courage displayed by Cahaly in presenting his work. In this era of hyperpartisan “analysis” of all polls, the attacks from those who prefer the conventional wisdom can be fierce and unrelenting. I have been there.
When all was said and done in 2020, Cahaly slightly overestimated the president’s support. But his efforts were a welcome counterbalance to many polls that dramatically overestimated the numbers for President-elect Biden. The inclusion of his data in the polling averages definitely improved the performance of those averages.
During the campaign, my sense of Cahaly’s claims was a mixture of skepticism along with an openness to the idea that he might be on to something. My gut sense was that shy Trump voters may have caused the polls to understate support for the president by a point or two. Nothing more.
Still, Cahaly was the only public pollster to see Trump winning Michigan and Pennsylvania in 2016. That was reason enough for me to seriously consider and test his thesis. So, during my final polls of the 2020 cycle, I included several questions related to social desirability bias. That would give me data to evaluate during a postelection review. If there was a way to measure and address social desirability bias, I wanted to find it.
Those who believe there’s a much larger number of Shy Trump voters would be quick to point out that many of the president’s supporters might not take a poll. Or, if they do, they might lie.
And, after the election, I explored the theme further. My polling found that 15% of Trump voters said their family and friends didn’t know how they voted. The same was true, however, among Biden’s supporters. When asked why their family and friends did not know, 48% said the reason is that voting is a private matter. Another 25% said it’s because they rarely discuss politics, and 11% because they decided at the last minute. Only 8% said they kept their voting decision from family and friends because they were afraid how others might react.
Still, fear of how others react was more common among Trump supporters than Biden voters. It’s not a huge gap but one suggesting that shy Trump voters accounted for understating the president’s support by about a single percentage point.
Those who believe there’s a much larger number of shy Trump voters would be quick to point out that many of the president’s supporters might not take a poll. Or, if they do, they might lie. Many of the president’s strongest supporters believe this to be a widespread phenomenon.
That is certainly possible, but there may also be another explanation.
Consider this. It appears that the Real Clear Politics average overstated Biden’s popular vote victory by about 2.7 percentage points (the average showed Biden with a 7.2 point lead and it looks like the final numbers will show him winning by around 4.5 points). Seven of the last 13 polls showed Biden with a lead of 8 points or larger. Two (including mine), showed Biden +7. Three were right on the money at 4 or 5 points and one overstated support for the president.
Compare that to the last time a president ran for reelection.
In 2012, the Real Clear Politics average understated support for President Obama by 3.2 percentage points. He won by 3.9 percentage points while the polls projected a 0.7% margin. Not a single poll in the final month of the campaign overestimated Obama’s margin of victory.
In other words, in both elections, the polling average understated support for the president seeking reelection. But the polling miss — and the understating of support for the president — was bigger for Obama in 2012 than for Trump in 2020.
Whenever a poll comes out, people view it differently based upon their own hopes and expectations.
Given that reality, how come we didn’t hear anything about “shy Obama voters” in 2012? I’m sure the answer is because they didn’t exist. I doubt that anybody who believes Trump voters are afraid to speak out also believes that the same fear applied to Obama voters. I certainly don’t.
So, how do we explain this disconnect? Why are we hearing about the polls being off this year when they were further off in 2012?
I believe the answer lies in something that might be called analysis bias. Whenever a poll comes out, people view it differently based upon their own hopes and expectations. If a poll shows a Democrat up by five points, Democrats are more likely to think the lead is a little bigger. Republicans will have the opposite view and think the race is a bit closer. Both can easily see how events might shift the numbers in their direction.
That’s a naturally human response. But it unduly impacts the election forecasting industry because most of the professional analysts lean to the left politically.
So, in 2012, the polls showed the incumbent up by less than a point. In that year, the analysts weren’t at all surprised that Obama did about three points better. It’s what they expected to see and so election night coverage worked out just fine. They worked out because the analysis bias balanced out the modest polling error.
However, in 2020, that analysis bias magnified a slightly smaller polling error. While the RCP polling average showed Biden leading by seven, many pros shared Charlie Cook’s view that the final result might be closer to a 10-point blowout. They were shocked when the actual results came in.
This suggests that the reason we’re hearing so many more complaints this year is because the analysts were embarrassed. For the second straight election, the Election night coverage did not go according to the script.
As I said last week, it is time to put the current election forecasting industry out of its misery. Unfortunately, many analysts are now jumping on the shy Trump voters’ bandwagon as a handy excuse for everything that went wrong. Next week, I’ll take a look at some more specific polling issues. And, I’ll highlight something that Cahaly got right but most analysts got wrong.
For now, though, it’s important to acknowledge that the problems facing the election forecasting industry won’t go away simply by taking Donald Trump’s name off the ballot.
Scott Rasmussen is an American political analyst and digital media entrepreneur. He is the author of “The Sun is Still Rising: Politics Has Failed But America Will Not.”