The impact of question order on surveillance of nicotine pouch and smokeless tobacco use among adults in the United States

Abstract

Objective

To improve understanding about measurement of smokeless tobacco and nicotine products, we conducted a survey experiment to investigate the effect of question order on calculated prevalence of use.

Methods

We embedded a split sample experiment within the May 2025 Rutgers Omnibus Study; 1891 US adults aged 18–45 were recruited from Amazon MTurk for an online survey about health behaviors, randomized to answer questions about nicotine pouches before smokeless tobacco or vice versa. We compared prevalence by experimental condition using chi-square tests, with alpha = 0.05.

Results

Prevalence of ever smokeless tobacco use was significantly higher among those who received the smokeless tobacco questions first (21.5 % vs 16.4 %, p = .005); however, there was only a small and non-significant difference in prevalence of ever nicotine pouch use (15.5 % vs 17.6 %, p = .23). Those who saw smokeless tobacco questions first were also more likely to name a nicotine pouch brand as their regular brand, suggesting that the smokeless tobacco prevalence in this group was overestimated.

Conclusion

Nicotine pouch use may be misreported as SLT use on surveys that assess smokeless tobacco first. Consequently, efforts to monitor prevalence of smokeless tobacco use as nicotine pouches gain popularity may be undermined by measurement error.