Jamil S. Samaan , Yee Hui Yeo , Wee Han Ng , Peng-Sheng Ting , Hirsh Trivedi , Aarshi Vipani , Ju Dong Yang , Omer Liran , Brennan Spiegel , Alexander Kuo , Walid S. Ayoub
{"title":"ChatGPT用阿拉伯语理解和回答肝硬化相关问题的能力。","authors":"Jamil S. Samaan , Yee Hui Yeo , Wee Han Ng , Peng-Sheng Ting , Hirsh Trivedi , Aarshi Vipani , Ju Dong Yang , Omer Liran , Brennan Spiegel , Alexander Kuo , Walid S. Ayoub","doi":"10.1016/j.ajg.2023.08.001","DOIUrl":null,"url":null,"abstract":"<div><h3>Background and study aims</h3><p>Cirrhosis is a chronic progressive disease which requires complex care. Its incidence is rising in the Arab countries making it the 7th leading cause of death in the Arab League in 2010. ChatGPT is a large language model with a growing body of literature demonstrating its ability to answer clinical questions. We examined ChatGPT’s accuracy in responding to cirrhosis related questions in Arabic and compared its performance to English.</p></div><div><h3>Materials and methods</h3><p>ChatGPTs responses to 91 questions in Arabic and English were graded by a transplant hepatologist fluent in both languages. Accuracy of responses was assessed using the scale: 1. Comprehensive, 2. Correct but inadequate, 3. Mixed with correct and incorrect/outdated data, and 4. Completely incorrect.<!--> <!-->Accuracy of Arabic compared to English responses was assessed using the scale: 1. Arabic response is more accurate, 2. Similar accuracy, 3. Arabic response is less accurate.</p></div><div><h3>Results</h3><p>The model provided 22 (24.2%) comprehensive, 44 (48.4%) correct but inadequate, 13 (14.3%) mixed with correct and incorrect/outdated data and 12 (13.2%) completely incorrect Arabic responses. When comparing the accuracy of Arabic and English responses, 9 (9.9%) of the Arabic responses were graded as more accurate, 52 (57.1%) similar in accuracy and 30 (33.0%) as less accurate compared to English.</p></div><div><h3>Conclusion</h3><p>ChatGPT has the potential to serve as an adjunct source of information for Arabic speaking patients with cirrhosis. The model provided correct responses in Arabic to 72.5% of questions, although its performance in Arabic was less accurate than in English. The model produced completely incorrect responses to 13.2% of questions, reinforcing its potential role as an adjunct and not replacement of care by licensed healthcare professionals. Future studies to refine this technology are needed to help Arabic speaking patients with cirrhosis across the globe understand their disease and improve their outcomes.</p></div>","PeriodicalId":48674,"journal":{"name":"Arab Journal of Gastroenterology","volume":null,"pages":null},"PeriodicalIF":1.1000,"publicationDate":"2023-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"3","resultStr":"{\"title\":\"ChatGPT’s ability to comprehend and answer cirrhosis related questions in Arabic\",\"authors\":\"Jamil S. Samaan , Yee Hui Yeo , Wee Han Ng , Peng-Sheng Ting , Hirsh Trivedi , Aarshi Vipani , Ju Dong Yang , Omer Liran , Brennan Spiegel , Alexander Kuo , Walid S. Ayoub\",\"doi\":\"10.1016/j.ajg.2023.08.001\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Background and study aims</h3><p>Cirrhosis is a chronic progressive disease which requires complex care. Its incidence is rising in the Arab countries making it the 7th leading cause of death in the Arab League in 2010. ChatGPT is a large language model with a growing body of literature demonstrating its ability to answer clinical questions. We examined ChatGPT’s accuracy in responding to cirrhosis related questions in Arabic and compared its performance to English.</p></div><div><h3>Materials and methods</h3><p>ChatGPTs responses to 91 questions in Arabic and English were graded by a transplant hepatologist fluent in both languages. Accuracy of responses was assessed using the scale: 1. Comprehensive, 2. Correct but inadequate, 3. Mixed with correct and incorrect/outdated data, and 4. Completely incorrect.<!--> <!-->Accuracy of Arabic compared to English responses was assessed using the scale: 1. Arabic response is more accurate, 2. Similar accuracy, 3. Arabic response is less accurate.</p></div><div><h3>Results</h3><p>The model provided 22 (24.2%) comprehensive, 44 (48.4%) correct but inadequate, 13 (14.3%) mixed with correct and incorrect/outdated data and 12 (13.2%) completely incorrect Arabic responses. When comparing the accuracy of Arabic and English responses, 9 (9.9%) of the Arabic responses were graded as more accurate, 52 (57.1%) similar in accuracy and 30 (33.0%) as less accurate compared to English.</p></div><div><h3>Conclusion</h3><p>ChatGPT has the potential to serve as an adjunct source of information for Arabic speaking patients with cirrhosis. The model provided correct responses in Arabic to 72.5% of questions, although its performance in Arabic was less accurate than in English. The model produced completely incorrect responses to 13.2% of questions, reinforcing its potential role as an adjunct and not replacement of care by licensed healthcare professionals. Future studies to refine this technology are needed to help Arabic speaking patients with cirrhosis across the globe understand their disease and improve their outcomes.</p></div>\",\"PeriodicalId\":48674,\"journal\":{\"name\":\"Arab Journal of Gastroenterology\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":1.1000,\"publicationDate\":\"2023-08-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"3\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Arab Journal of Gastroenterology\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1687197923000588\",\"RegionNum\":4,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q4\",\"JCRName\":\"GASTROENTEROLOGY & HEPATOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Arab Journal of Gastroenterology","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1687197923000588","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"GASTROENTEROLOGY & HEPATOLOGY","Score":null,"Total":0}
ChatGPT’s ability to comprehend and answer cirrhosis related questions in Arabic
Background and study aims
Cirrhosis is a chronic progressive disease which requires complex care. Its incidence is rising in the Arab countries making it the 7th leading cause of death in the Arab League in 2010. ChatGPT is a large language model with a growing body of literature demonstrating its ability to answer clinical questions. We examined ChatGPT’s accuracy in responding to cirrhosis related questions in Arabic and compared its performance to English.
Materials and methods
ChatGPTs responses to 91 questions in Arabic and English were graded by a transplant hepatologist fluent in both languages. Accuracy of responses was assessed using the scale: 1. Comprehensive, 2. Correct but inadequate, 3. Mixed with correct and incorrect/outdated data, and 4. Completely incorrect. Accuracy of Arabic compared to English responses was assessed using the scale: 1. Arabic response is more accurate, 2. Similar accuracy, 3. Arabic response is less accurate.
Results
The model provided 22 (24.2%) comprehensive, 44 (48.4%) correct but inadequate, 13 (14.3%) mixed with correct and incorrect/outdated data and 12 (13.2%) completely incorrect Arabic responses. When comparing the accuracy of Arabic and English responses, 9 (9.9%) of the Arabic responses were graded as more accurate, 52 (57.1%) similar in accuracy and 30 (33.0%) as less accurate compared to English.
Conclusion
ChatGPT has the potential to serve as an adjunct source of information for Arabic speaking patients with cirrhosis. The model provided correct responses in Arabic to 72.5% of questions, although its performance in Arabic was less accurate than in English. The model produced completely incorrect responses to 13.2% of questions, reinforcing its potential role as an adjunct and not replacement of care by licensed healthcare professionals. Future studies to refine this technology are needed to help Arabic speaking patients with cirrhosis across the globe understand their disease and improve their outcomes.
期刊介绍：
Arab Journal of Gastroenterology (AJG) publishes different studies related to the digestive system. It aims to be the foremost scientific peer reviewed journal encompassing diverse studies related to the digestive system and its disorders, and serving the Pan-Arab and wider community working on gastrointestinal disorders.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
