Editorial Commentary: Studies Evaluating Artificial Intelligence Large Language Models' Ability to Respond to Questions Are Repetitive and Out-of-Date: Artificial Intelligence Must Now Be Applied to Improve Clinical Practice and Patient Care.

Abstract

While artificial intelligence (AI) technologies such as ChatGPT have shown very real and powerful capabilities to date, this does not mean that research studying these technologies is immune from "shiny object syndrome," a psychological phenomenon in which individuals tend to focus on new and fashionable ideas only to be distracted from those that truly matter. In parallel with the increased publicity that AI has received since the release of large language models (LLMs) like ChatGPT, there has been an explosion in the number of studies evaluating LLMs' ability to answer hypothetical questions from patients on a variety of conditions. Nevertheless, these studies tend to leave us with the same conclusion: LLMs are generally capable of providing reliable and relevant responses to patient questions but are not without limitations. Given the abundance of studies demonstrating similar outcomes regardless of whether the LLMs are asked to respond to a patient's questions about their diabetes or about their shoulder dislocation, I'm afraid we are at risk of making AI more of a "shiny object" than a tool that can be used to change clinical practice and improve patient care. Specifically, we may be approaching a point at which a "publish or perish" mindset has promoted studies with repetitive methodologies that only confirm well-established theories around the capabilities and limitations of AI and has created a distraction from new use cases and more meaningful applications for patient care. We are now at a crossroads at which we can either remain stuck in the past, repeating old studies' methodologies on a different procedure or injury, or progress by expanding the number and impact of applications that these tools have in orthopaedic surgery. The capabilities of AI will continue to increase at a rapid pace, but it will be up to those with intricate knowledge of orthopaedics and patient care to keep up.