NeuroGPT, evaluando ChatGPT: Diagnóstico y tratamiento de 72 pacientes neurológicos

Introduction

There has been a significant boom in the field of artificial intelligence in recent years, especially in terms of accessibility and its use in different areas. This study attempts to determine if an AI can diagnose neurology patients.

Objective

To evaluate the utility and accuracy of ChatGPT 3.5 as a tool for conducting patient history, diagnosis, and treatment in cases of neurological pathology.

Materials and methods

A descriptive qualitative observational study was conducted, without intervention in patients, focused on evaluating the utility and accuracy of ChatGPT 3.5 for taking patient history, diagnosis, and treatment in patients with neurological pathology. The information provided to the neurologist was entered into the language model. Subsequently, the questions determined by ChatGPT were asked, and the complete neurological examination was provided. ChatGPT's diagnosis was compared with that of two different neurologists. Recruitment took place from May 2022 to June 2023 in a neurology consultation at a medium-sized hospital in Spain.

Results

A total of 72 patients (median age 58.71 years and 55.6% female) were enrolled in this study. Complementary tests suggested by the AI were considered correct in 33.3% of cases. The accuracy of the AI's diagnosis was 44.4%, and treatment recommendations were correct in 37.5%. The diagnosis was checked by two different neurologists following the latest national and international Neurology guidelines. In most cases, the diagnosis between the two neurologists agreed, with a kappa coefficient of 0.94.

Conclusions

Although we are in an unprecedented era of advancement in the field of artificial intelligence, it does not seem that ChatGPT can currently replace the evaluation of a neurology specialist.