Why it’s difficult to gauge the presidential race between Trump and Harris

Concerns about the 2024 elections are great. Both parties know their candidate power lose – and they want polls to tell them how scared they should be.

Right now, polls give us the clearest answers to those questions. But there is a problem: every year the elections become more difficult. We pollsters face three core challenges that threaten the accuracy of our surveys all political investigations. No one has solved them, and it’s not clear that anyone can.

This is what we are against:

Challenge #1: Almost no one wants to talk to pollsters – and those who do may be crazy.

Polls are based on a simple idea: if we talk to a representative miniature version of a state or country, we can estimate what the entire state or country is thinking. It’s like getting a sample at an ice cream parlor: one well-mixed scoop tells you what a whole cone will taste like.

But it is becoming increasingly difficult to reach the people needed to build that mini-country or mini-state. Response rates – how often people are willing to pick up a cold call or reply to a text message from us – have been declining for decades. Response rates for the Pew Research Center’s telephone surveys fell from 36% in 1997 to 6% in 2018. Nate Cohn of The New York Times reported a 0.4% response rate to his polls in 2022. And any pollster will tell you that the response rates are also low this year.

Non-response makes good data rare and expensive. Polls are expensive in part because we spend so much money sending unanswered text messages or making calls that go straight to voicemail. And as polls become more expensive, media organizations will either sponsor fewer surveys or opt for polls that reduce costs through budget cuts.

But even if a group can afford to conduct a poll, non-response creates enormous potential data problems.

When only 1 in 100 people takes a poll, pollsters have to make statistical adjustments. Some – like finding the right demographic mix – are simple. If a pollster can’t reach enough Latino, working-class, youth, or rural voters, they often give underrepresented voters the chance to did contact slightly more weight in their calculations. Weighted polls give each demographic group a more accurate amount of input, even if some groups were harder to reach.

Other adjustments are not so simple.

Suppose a pollster has the right demographic mix in their survey But usually done by interviewing, for lack of a better description, nerdy rule followers. This pollster could miss the cranky, anti-establishment Trump voters — and risk underestimating the Trump vote for the third election in a row.

It is almost impossible to directly address these types of problems. The census helps us calculate how many 18 to 34 year olds should be in a poll, but not how many idiots and nerds. So pollsters have to get creative with math, which leads to another problem.

Challenge #2: We have to work our way through the fact that no one is talking to us. That’s risky.

The most common response to this problem – a dearth of pro-Trump, anti-institutional Republicans in the 2020 polls – is weighing in through “recalled votes.” Essentially, pollsters ask people how they voted in 2020 and try to get the right number of Trump and Biden voters in their sample.

While I have used this tactic in some polls, there are drawbacks. Respondents do not always correctly remember who they voted for. Any estimate of how many Trump or Biden voters will vote again in 2024 is just that: an estimate. The list goes on.

That said, many reputable pollsters say that weighting based on recalled votes has improved the accuracy of past surveys. And pollsters who weigh party-only — and don’t recapture votes — may fail to address the problems that damaged the industry’s credibility four years ago.

There is no right answer. Everyone uses math to adjust for the sad fact that normal people don’t take surveys. And every pollster is on edge because if we make even the slightest mistake, we will be punished for years to come.

Challenge #3: The election is closer than ever, so “the polls” will almost certainly be “wrong.”

The last real presidential victory was Ronald Reagan’s re-election in 1984. The last forty years have seen the most consistently competitive presidential races in living memory. That’s bad news for polls, which are blunt instruments rather than accurate forecasters.

When a pollster draws a random sample of the electorate, they may – through no fault of their own – accidentally pick up a few too many voters from one side or the other. When we try to gauge an upcoming election, we make (fallible) projections about who will and will not turn out. And there’s a lot more uncertainty – from non-response, weighting decisions and more – that just isn’t easily conveyed to the lay reader.

In a race this tight, where survey after survey has killed Harris and Trump even in swing states, a good pollster could do everything right and still miss the outcome by a few points and be ridiculed for years by a huge audience of readers.

But we pollsters can’t look at these problems and shout, “It’s not fair!” and go home. I’ve built election prediction models and I’ve seen firsthand that polls are the best tool for predicting elections. More importantly, they are the only way to ask the audience in real time what they think about a question.

The problems with the elections are real. Perhaps at some point non-response or some other problem will become unsolvable and cause a catastrophic collapse of the entire industry. But unless that happens, we’ll have to keep polling, because nothing else does exactly what polls do.