Digital Food Safety: Insights from Fact-Checking ChatGPT Consumer Interactions

IF 2.3 3区医学 Q2 EDUCATION, SCIENTIFIC DISCIPLINES

Journal of Nutrition Education and Behavior Pub Date : 2025-08-01 DOI:10.1016/j.jneb.2025.05.032

Lorena Correia BS, Yaohua Feng PhD

{"title":"Digital Food Safety: Insights from Fact-Checking ChatGPT Consumer Interactions","authors":"Lorena Correia BS, Yaohua Feng PhD","doi":"10.1016/j.jneb.2025.05.032","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>Consumers frequently seek nutrition and food safety information for guidance in food preparation practices. According to Consumer Reports, 21% of American consumers have used generative language models, such as ChatGPT, for health-related purposes, reflecting a growing trend in AI assistance for health information. Despite their quick, conversational responses, these models can \"hallucinate,\" producing confident but incorrect or fabricated information.</div></div><div><h3>Objective</h3><div>This study evaluates ChatGPT 4.0 Mini's capability to generate accurate references to support its statements in answering if 20 consumers’ questions about food safety practices were risky.</div></div><div><h3>Study Design, Settings, Participants</h3><div>The practices were extracted from the food safety podcast Risky or Not, hosted by food science professors Dr. Ben Chapman and Dr. Don Schaffner. The prompt tested after entering food safety inquiries sent by consumers (n=20) on ChatGPT 4.0 was: “Fact check: Provide evidence for my question from research articles, with author’s name, title of the article, publication venue (volume, issue, page number), year and link.”.</div></div><div><h3>Measurable Outcome/Analysis</h3><div>It was considered a reference invalid if the chatbot did not provide any reference; generated a link that led to a paper that was different from the one mentioned; if the link referenced had a false DOI; if it did not provide the paper information; if it pointed to a nonexistent paper.</div></div><div><h3>Results</h3><div>The results have shown that ChatGPT 4.0 Mini generated valid references only 35% of the time for the n=20 questions analyzed in the study. In some cases, the ChatGPT 4.0 Mini initially produced correct reference information but, upon repetition, it fabricated a nonexistent link.</div></div><div><h3>Conclusions</h3><div>These findings highlight the limitation of AI-driven chatbots in providing the reference for the content generated, emphasizing the need for cautious interpretation and further evaluation of their reliability in consumer food safety advice.</div></div><div><h3>Funding</h3><div>None</div></div>","PeriodicalId":50107,"journal":{"name":"Journal of Nutrition Education and Behavior","volume":"57 8","pages":"Page S14"},"PeriodicalIF":2.3000,"publicationDate":"2025-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Nutrition Education and Behavior","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1499404625001484","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Consumers frequently seek nutrition and food safety information for guidance in food preparation practices. According to Consumer Reports, 21% of American consumers have used generative language models, such as ChatGPT, for health-related purposes, reflecting a growing trend in AI assistance for health information. Despite their quick, conversational responses, these models can "hallucinate," producing confident but incorrect or fabricated information.

Objective

This study evaluates ChatGPT 4.0 Mini's capability to generate accurate references to support its statements in answering if 20 consumers’ questions about food safety practices were risky.

Study Design, Settings, Participants

The practices were extracted from the food safety podcast Risky or Not, hosted by food science professors Dr. Ben Chapman and Dr. Don Schaffner. The prompt tested after entering food safety inquiries sent by consumers (n=20) on ChatGPT 4.0 was: “Fact check: Provide evidence for my question from research articles, with author’s name, title of the article, publication venue (volume, issue, page number), year and link.”.

Measurable Outcome/Analysis

It was considered a reference invalid if the chatbot did not provide any reference; generated a link that led to a paper that was different from the one mentioned; if the link referenced had a false DOI; if it did not provide the paper information; if it pointed to a nonexistent paper.

Results

The results have shown that ChatGPT 4.0 Mini generated valid references only 35% of the time for the n=20 questions analyzed in the study. In some cases, the ChatGPT 4.0 Mini initially produced correct reference information but, upon repetition, it fabricated a nonexistent link.

Conclusions

These findings highlight the limitation of AI-driven chatbots in providing the reference for the content generated, emphasizing the need for cautious interpretation and further evaluation of their reliability in consumer food safety advice.

Funding

None

查看原文本刊更多论文

数字食品安全：来自事实核查ChatGPT消费者互动的见解

背景：消费者经常寻求营养和食品安全信息，以指导食品制备实践。根据消费者报告，21%的美国消费者使用生成语言模型，如ChatGPT，用于健康相关目的，这反映了人工智能辅助健康信息的增长趋势。尽管这些模型可以快速、对话式地做出反应，但它们可能会产生“幻觉”，产生自信但不正确或捏造的信息。目的：本研究评估ChatGPT 4.0 Mini在回答20个消费者关于食品安全实践是否存在风险的问题时生成准确参考以支持其陈述的能力。研究设计，设置，参与者这些实践摘自食品安全播客“风险与否”，由食品科学教授本·查普曼博士和唐·沙夫纳博士主持。在ChatGPT 4.0上输入消费者（n=20）发送的食品安全查询后，测试的提示是：“Fact check：从研究文章中为我的问题提供证据，包括作者姓名、文章标题、出版地点（卷、期、页码）、年份和链接。”可测量的结果/分析如果聊天机器人没有提供任何参考，则认为参考无效；生成一个链接，指向一篇与提到的不同的论文；如果引用的链接DOI为假；未提供书面资料的；如果它指向一篇不存在的论文。结果结果表明，在研究中分析的n=20个问题中，ChatGPT 4.0 Mini仅在35%的时间内生成有效参考。在某些情况下，ChatGPT 4.0 Mini最初会生成正确的参考信息，但在重复时，它会伪造一个不存在的链接。这些发现突出了人工智能驱动的聊天机器人在为生成的内容提供参考方面的局限性，强调了谨慎解释和进一步评估其在消费者食品安全建议中的可靠性的必要性。FundingNone

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Nutrition Education and Behavior 医学-学科教育

CiteScore

4.20

自引率

11.50%

发文量

379

审稿时长

44 days

期刊介绍： The Journal of Nutrition Education and Behavior (JNEB), the official journal of the Society for Nutrition Education and Behavior, is a refereed, scientific periodical that serves as a global resource for all professionals with an interest in nutrition education; nutrition and physical activity behavior theories and intervention outcomes; complementary and alternative medicine related to nutrition behaviors; food environment; food, nutrition, and physical activity communication strategies including technology; nutrition-related economics; food safety education; and scholarship of learning related to these areas. The purpose of JNEB is to document and disseminate original research and emerging issues and practices relevant to these areas worldwide. The Journal of Nutrition Education and Behavior welcomes evidence-based manuscripts that provide new insights and useful findings related to nutrition education research, practice and policy. The content areas of JNEB reflect the diverse interests in nutrition and physical activity related to public health, nutritional sciences, education, behavioral economics, family and consumer sciences, and eHealth, including the interests of community-based nutrition-practitioners. As the Society''s official journal, JNEB also includes policy statements, issue perspectives, position papers, and member communications.