Lucas Alessandro, Santiago Crema, Juan I Castiglione, Daiana Dossi, Federico Eberbach, Alejandro Kohler, Alfredo Laffue, Abril Marone, Vanesa Nagel, José M Pastor Rueda, Francisco Varela, Diego Fernandez Slezak, Sofía Rodríguez Murúa, Carlos Debasa, Pensa Claudio, Mauricio F Farez
{"title":"Validation of an Artificial Intelligence-Powered Virtual Assistant for Emergency Triage in Neurology.","authors":"Lucas Alessandro, Santiago Crema, Juan I Castiglione, Daiana Dossi, Federico Eberbach, Alejandro Kohler, Alfredo Laffue, Abril Marone, Vanesa Nagel, José M Pastor Rueda, Francisco Varela, Diego Fernandez Slezak, Sofía Rodríguez Murúa, Carlos Debasa, Pensa Claudio, Mauricio F Farez","doi":"10.1097/NRL.0000000000000594","DOIUrl":null,"url":null,"abstract":"<p><strong>Objectives: </strong>Neurological emergencies pose significant challenges in medical care in resource-limited countries. Artificial intelligence (AI), particularly health chatbots, offers a promising solution. Rigorous validation is required to ensure safety and accuracy. Our objective is to evaluate the diagnostic safety and effectiveness of an AI-powered virtual assistant (VA) designed for the triage of neurological pathologies.</p><p><strong>Methods: </strong>The performance of an AI-powered VA for emergency neurological triage was tested. Ten patients over 18 years old with urgent neurological pathologies were selected. In the first stage, 9 neurologists assessed the safety of the VA using their clinical records. In the second stage, the assistant's accuracy when used by patients was evaluated. Finally, VA performance was compared with ChatGPT 3.5 and 4.</p><p><strong>Results: </strong>In stage 1, neurologists agreed with the VA in 98.5% of the cases for syndromic diagnosis, and in all cases, the definitive diagnosis was among the top 5 differentials. In stage 2, neurologists agreed with all diagnostic parameters and recommendations suggested by the assistant to patients. The average use time was 5.5 minutes (average of 16.5 questions). VA showed superiority over both versions of ChatGPT in all evaluated diagnostic and safety aspects (P<0.0001). In 57.8% of the evaluations, neurologists rated the VA as \"excellent\" (suggesting adequate utility).</p><p><strong>Conclusions: </strong>In this study, the VA showcased promising diagnostic accuracy and user satisfaction, bolstering confidence in further development. These outcomes encourage proceeding to a comprehensive phase 1/2 trial with 100 patients to thoroughly assess its \"real-time\" application in emergency neurological triage.</p>","PeriodicalId":49758,"journal":{"name":"Neurologist","volume":" ","pages":""},"PeriodicalIF":1.1000,"publicationDate":"2025-02-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neurologist","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1097/NRL.0000000000000594","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"CLINICAL NEUROLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Objectives: Neurological emergencies pose significant challenges in medical care in resource-limited countries. Artificial intelligence (AI), particularly health chatbots, offers a promising solution. Rigorous validation is required to ensure safety and accuracy. Our objective is to evaluate the diagnostic safety and effectiveness of an AI-powered virtual assistant (VA) designed for the triage of neurological pathologies.
Methods: The performance of an AI-powered VA for emergency neurological triage was tested. Ten patients over 18 years old with urgent neurological pathologies were selected. In the first stage, 9 neurologists assessed the safety of the VA using their clinical records. In the second stage, the assistant's accuracy when used by patients was evaluated. Finally, VA performance was compared with ChatGPT 3.5 and 4.
Results: In stage 1, neurologists agreed with the VA in 98.5% of the cases for syndromic diagnosis, and in all cases, the definitive diagnosis was among the top 5 differentials. In stage 2, neurologists agreed with all diagnostic parameters and recommendations suggested by the assistant to patients. The average use time was 5.5 minutes (average of 16.5 questions). VA showed superiority over both versions of ChatGPT in all evaluated diagnostic and safety aspects (P<0.0001). In 57.8% of the evaluations, neurologists rated the VA as "excellent" (suggesting adequate utility).
Conclusions: In this study, the VA showcased promising diagnostic accuracy and user satisfaction, bolstering confidence in further development. These outcomes encourage proceeding to a comprehensive phase 1/2 trial with 100 patients to thoroughly assess its "real-time" application in emergency neurological triage.
期刊介绍：
The Neurologist publishes articles on topics of current interest to physicians treating patients with neurological diseases. The core of the journal is review articles focusing on clinically relevant issues. The journal also publishes case reports or case series which review the literature and put observations in perspective, as well as letters to the editor. Special features include the popular "10 Most Commonly Asked Questions" and the "Patient and Family Fact Sheet," a handy tear-out page that can be copied to hand out to patients and their caregivers.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
