Case Study: The 2025 Voter Poll
By Kevin Collins
At the recent annual meeting of the American Association for Public Opinion Research, we presented on Survey 160’s contribution to the 2025 Voter Poll by SSRS, a multi-state, multi-mode pre-election survey effort fielded across four states in the run-up to, through Election Day of, the November 2025 off-year and special elections. The poll was a collaboration with researchers at SSRS, Verasight and Survey 160, and our role was fielding the text-to-web surveys to the registration-based sample (RBS) frame. The 2025 Voter Poll is a useful, if unusually large, case study in how text messaging can be a useful addition to other modes of contact. In this update, we share four findings from that work: how texting fit in with the other components of the Voter Poll, an experiment in encouraging inbound phone interviews, how the unweighted text-to-web sample looked against other RBS modes and post-election benchmarks, and what it all cost.
Spoiler: the text component reduced the cost per interview 93.7% relative to live outbound calls. Read to the end to learn how many months of interviewer hours texting saved as part of this project.
The text-eligible portion of the RBS frame was 305,671 records. We fielded between October 24 and November 3, with a soft launch on October 24, and reminder texts to non-responders beginning October 30, and a tail of completes coming in through Election Day. Texting produced 1,871 completed interviews, representing about 55% of the RBS-mode completes and around 10% of the overall 2025 Voter Poll dataset.
As a research-focused firm, we regularly experiment with new fielding tactics to build up tested best practices. One of the experiments we built into this fielding was a test of whether adding a dial-in phone number alongside the web link in the survey invitation would generate additional response. The hypothesis was reasonable on its face. A call-in option might pick up respondents who are wary of clicking links from senders they do not recognize, and the visible presence of a working phone number might itself function as a credibility cue. We randomized whether the invitation included a call-in number, holding everything else constant.
The answer was a clean null. The estimated effect of including the call-in number on cooperation was -0.0012 with a p-value of 0.9334 — about as precise a zero as a regression will give you. We will not claim that no version of text-to-call can ever work, but in this design, on this sample, it added nothing.
The deeper question for any text-to-web component of a multi-mode poll is whether the unweighted distribution of respondents looks anything like the rest of the sample. We compared the text-to-web respondents to two benchmarks: completes collected by the other modes from the same RBS frame, and the final weighted distribution estimates from post-election public reporting. The state-by-state picture is genuinely noisy. Age patterns differ across states — texting produced a younger sample in some states but in Virginia produced an older one. Gender patterns are similarly inconsistent across states. Texting sampled somewhat more white respondents than the post-election benchmarks in all four states, though in California the text-to-web racial composition closely matched what the other RBS modes produced. Educational attainment skewed somewhat more college-educated in two of four states. Reported income skewed higher across the board. Partisan identification skewed Republican in two of four states, and texted respondents were more likely than other RBS respondents to report having voted for Donald Trump in 2024, although they were also less likely to report abstaining from 2024 voting altogether.
To put these state-level patterns on a more systematic footing, we pooled across the four states and ran regressions of mode (texting versus other RBS) on each demographic and political covariate, with state fixed effects. Three differences survived pooling at p<0.05: texting produced fewer older respondents, more high-income respondents, and more 2024 Trump voters than the other RBS modes did.
Two points are worth pulling out of these distributional comparisons. The first is that none of these differences were so large as to make the text-to-web sample unrecognizable as a slice of the same RBS population — the unweighted distributions are, broadly, similar to what the other modes produced from the same frame. The second is that the direction of difference on partisanship is worth filing away. The lean toward 2024 Trump voters runs in the opposite direction of the conventional wisdom about web-based modes, which is shaped largely by experience with opt-in online panels. That conventional wisdom does not transfer cleanly to probability text-to-web, and this finding is consistent with our earlier work showing that probability text-to-web and online panels reach meaningfully different slices of the public.
The cost story is the most striking part of the project, and the easiest to summarize. On a per-complete basis, the texting interviews cost 6.3% of what the same interview would have cost via outbound live phone calls — a 93.7% reduction. Put another way, live phone interviews were roughly 15 times more expensive than texting on a per-complete basis. Together with SSRS we estimate that the texting component substituted for approximately 11,820 hours of caller effort, which is about 16 months of full-time caller time. On a project with the fielding window pre-election polling actually has, those are hours that simply would not have existed to spend.
Taken together, four points stand out. First, on a multi-state pre-election RBS effort like the 2025 Voter Poll, texting can carry a substantial share of completes — in our case a majority of the RBS interviews and a meaningful share of the overall sample — at a small fraction of the live-phone cost. Second, not every creative idea works as hoped; the call-in option here was a clean null and not worth the additional complexity. Third, the unweighted text-to-web sample looked similar to the other RBS modes but skewed somewhat younger, higher-income, and more Trump-voting in pooled analysis, populations that are somewhat harder to reach generally. And fourth, the cost differential is large enough that it changes what is feasible to field on the timelines and budgets pre-election polling actually operates under — 11,820 caller-hours is not a rounding error.
We are grateful to our partners on the 2025 Voter Poll by SSRS for this collaboration. If you are running pre-election polling that uses, or might use, probability-sampled text message surveys alongside other modes, please reach out at info@survey160.com. There is more here to learn together.
