Could generative artificial intelligence replace fieldwork in pain research?

Background: Generative artificial intelligence (AI) models offer potential assistance in pain research data acquisition, yet concerns persist regarding data accuracy and reliability. In a comparative study, we evaluated open generative AI models' capacity to acquire data on acute pain in rock climbers comparable to field research.

Methods: Fifty-two rock climbers (33 m/19 f; age 29.0 [24.0-35.75] years) were asked to report pain location and intensity during a single climbing session. Five generative pretrained transformer models were tasked with responses to the same questions.

Results: Climbers identified the back of the forearm (19.2%) and toes (17.3%) as primary pain sites, with reported median pain intensity at 4 [3-5] and median maximum pain intensity at 7 [5-8]. Conversely, AI models yielded divergent findings, indicating fingers, hands, shoulders, legs, and feet as primary pain localizations with average and maximum pain intensity ranging from 3 to 4.4 and 5 to 10, respectively. Only two AI models provided references that were untraceable in PubMed and Google searches.

Conclusion: Our findings reveal that, currently, open generative AI models cannot match the quality of field-collected data on acute pain in rock climbers. Moreover, the models generated nonexistent references, raising concerns about their reliability.