Addressing Survey Fraud in Online Health Research: A Case Study of Latine Sexual Minority Men.

Abstract

Online survey research has become an increasingly popular and effective method in the social sciences for exploring and addressing health-related issues. However, the increasing prevalence of fraudulent activities, particularly survey bots, threatens data integrity and can compromise health research by generating misleading data. The purpose of this paper was to describe the implementation of bot detection strategies in an online survey with Latine sexual minority men (SMM). Eleven bot detection indicators, including AI-detection software for open-ended responses, were used in two approaches to differentiate bot-generated from human responses. In the first approach, bot detection indicators were applied stepwise to identify valid entries. In the second approach, a fraud detection algorithm was used to identify three fraud categories. Key demographics and study variables were compared across fraud categories using chi-square/Fisher's Exact tests for categorical data and Kruskal-Wallis tests for continuous data (significance set at 0.05). Of the 1147 total survey entries, 837 (73%) completed at least 20% of the survey (814 completed all items). A total of 739 (88%) of the 837 completed surveys were classified as fraudulent. Among the 837 completed surveys, 333 (40%) had an AI-generated open-ended response and fast completion time (≤ 20 min) and 234 (28%) entries were flagged for all three of these indicators. Sociodemographic characteristics and HIV prevention outcomes were largely similar across bot-generated and human responses. Findings suggest that survey bots are a pervasive threat to online research and are effective at providing human-like responses. To protect data integrity and ensure the development of effective health policies and interventions, health science researchers should adopt comprehensive bot detection and prevention strategies.