Chat GPT, Gemini or Meta AI: A comparison of AI platforms as a tool for answering higher-order questions in microbiology.

Abstract

Introduction: Artificial intelligence (AI) platforms have achieved a noteworthy role in various fields of medical sciences, ranging from medical education to clinical diagnostics and treatment. ChatGPT, Gemini, and Meta AI are some large language models (LLMs) that have gained immense popularity among students for solving questions from different branches of education.

Materials and methods: A cross-sectional study was conducted in the Department of Microbiology to assess the performance of ChatGPT, Gemini, and Meta AI in answering higher-order questions from various competencies of the microbiology curriculum (MI 1 to 8), according to CBME guidelines. Sixty higher-order questions were compiled from university question papers of two universities. Their responses were assessed by three faculty members from the department.

Results: The mean rank scores of ChatGPT, Gemini, and Meta AI were found to be 102.76, 108.5, and 60.23 by Evaluator 1; 106.03, 88.5, and 76.95 by Evaluator 2; and 104.85, 85.6, and 81.04, respectively, indicating lowest overall mean rank score for Meta AI. ChatGPT had the highest mean score in MI 2,3,5,6,7, and 8 competencies, while Gemini had a higher score for MI 1 and 4 competencies. A qualitative assessment of the three platforms was also performed. ChatGPT provided elaborative responses, some responses from Gemini lacked certain significant points, and Meta AI gave answers in bullet points.

Conclusions: Both ChatGPT and Gemini have created vast databases to correctly respond to higher-order queries in medical microbiology in comparison to Meta AI. Our study is the first of its kind to compare these three popular LLM platforms for microbiology.