Pollsters prepare for major changes after presidential election misses

CHICAGO — The polling industry is on the precipice of its biggest change in decades, as pollsters try to battle back from consecutive presidential election failures.

Pollsters are increasingly embracing new methods in the run-up to the 2022 midterms after notable misses in recent races. Front of mind is the looming 2024 election cycle, when former President Donald Trump — whose support among the electorate has bedeviled pollsters trying to measure it for the past seven years, including missing low on Trump’s vote before his 2016 win and underestimating the closeness of his 2020 loss — could be on the ballot for the third consecutive presidential election.

Both the internal polling that drives campaigns’ decisions and the media surveys that help shape coverage of the races are already changing: Pollsters are trying new ways to collect data, like contacting potential respondents by text message instead of phone calls, and seeking new ways of adjusting the data after to make it more accurately reflect the whole electorate.

At the American Association for Public Opinion Research’s annual conference here last week, the papers presented and the sideline conversations presaged the biggest change in election polling since Americans began cutting the cord and ditching landlines for cell phones.

As the polling industry seeks to pick up the pieces after “a disaster for our field,” as longtime Pew Research Center pollster Scott Keeter referred to the 2020 election, some of these new methods are seeping into the mainstream. Major media polls are now reaching some respondents via text message or mail solicitations, replacing or supplementing the phone calls that have dominated polling for decades — but are less likely than ever to be answered by potential respondents.

Increasingly, pollsters across the field favor combining multiple methods of contact in the same poll, seeking to include the hardest-to-reach Americans.

“The future’s multi-modal,” said Republican pollster David Kanevsky, who attended the conference. “We’re going to reach you any way we can.”

It’s already happening across the board. After ending their more-than-30-year partnership with NBC News after the 2020 election, the Wall Street Journal’s new poll — a cooperative effort between the pollsters for both Trump and President Joe Biden — reaches a quarter of its respondents via text message. The technique, called “text-to-web,” sends a link to an internet survey, and those interviews are added to others conducted by voice over landlines and cell phones.

Over the past year, CNN’s polling has spanned a wide range of methods. Conducted both over the phone and the internet, some polls are conducted from respondents who have joined a panel maintained by SSRS, the Pennsylvania-based company that conducts polls for CNN and other outlets. But the samples for two other CNN polls — one earlier this year and one in the summer of 2021 — were obtained by mailing solicitations to people at home and asking them to participate, either on the phone or the web.

Prior to last summer, all of CNN’s national polling had been conducted by phone.

Large media outlets have experimented with emerging methodologies before. CBS News and The New York Times did some of the first major media polling over the internet in 2014.

At the time, the CBS News/New York Times polling with web pollster YouGov was controversial: AAPOR’s then-president called it “a disappointing precedent being set by two of our leading media institutions,” before ultimately walking back some of the criticism.

There’s no such compunction now about new methods. Panel discussions at the AAPOR conference included titles like “Innovations in Election Polling” and “The Horse Race: We Can’t Tell Who Wins, But Know Pollsters Lose.”

That mostly reflects a realization that, as Americans become increasingly more difficult to reach, traditional methods are becoming more untenable.

“These things are only getting harder,” said one pollster who was granted anonymity to offer a candid assessment of the state of the industry. “So if you’re just doing the same thing, it’s only going to get worse.”

The innovations are not limited to sampling and data collection. But devising new weighting parameters — ways to adjust the results to better reflect the electorate — is more difficult. That’s because one of the main culprits of the 2020 election miss appears to be people who don’t respond to polls — so-called “nonresponse bias.” Voters in that group were more likely to support Trump, which made it harder for polls to reflect the true measure of his support.

After the 2016 election, one of AAPOR’s main recommendations was to weight by educational attainment, which some pollsters were not doing. But almost every pollster addressed this before 2020, and the polls still underestimated Trump.

Now, much of the research centers around figuring out which voters pollsters are missing. It’s insufficient just to “weight up” Republicans, because the Republicans who do and don't respond to polls are different — the ones who talk to pollsters are less likely to support Trump, for example.

Instead, some of the new proposals include weighting data to some social benchmarks, like the percentage of people who know and talk to their neighbors, or who volunteer in their community. Others suggest asking whether people trust media or polling in order to determine if their sample is too establishment-friendly.

The implementation of new methods is also happening at the campaign level, though to a lesser degree. Campaign pollsters — whose jobs involve giving candidates and outside groups strategic advice more than simply measuring the horse race — are dipping their toes tentatively into less-familiar waters.

In general, Democratic firms — many of which banded together after the 2020 election to study went wrong — are more open to experimentation.

“I think there’s more of a willingness to try different things and have this discussion,” said Nick Gourevitch, a partner and managing director for research at the Democratic polling firm Global Strategy Group.

But multiple campaign pollsters said their field is experimenting at the margins, though many of the old methods — traditional phone surveys, calling people on lists of registered voters — remain dominant.

“For us, any experimentation in real time has consequences,” said Dan Judy, a Republican pollster with North Star Opinion Research. “You have to be careful about what you’re doing. If you try it the old way, and the old way is dead, then you end up with 2020. But if you try something new, and the new way is also improper, maybe you get a different result — but it could also be wrong. That’s what keeps you awake at night, trying to strike that balance.”

Judy added that he feels good about polling in 2022, “because I feel like what we know works is still likely to work in this midterm. But in the next presidential, that’s when things are going to get dicey.”