Large language models in orthopedics: An exploratory research trend analysis and machine learning classification

Abstract

Background

Large Language Models (LLMs) are set to transform orthopedic practice with promising applications and a growing body of research. This exploratory study analyzed research trends in orthopedic LLMs and validated a machine learning classifier for categorizing publications into predefined domains. We hypothesized that LLM-related research would exhibit distinct thematic trends, and that a machine learning classifier would be able to accurately categorize research domains.

Methods

A bibliometric analysis of 140 Scopus-indexed publications (2019–2024) was performed using keyword co-occurrence and thematic clustering. Articles were categorized into five areas: Patient Education, Research and Ethics, Surgeon Education, Clinical Support, and Diagnostics and Radiology Interpretation. Machine learning classifiers were trained on TF-IDF vectorized text and evaluated using precision, recall, F1-score, and area under the receiver operating characteristic curve (AUC-ROC). Exploratory projections using linear regression assessed the volume and growth trends within the five research areas.

Results

The exploratory analysis revealed a substantial increase in LLM publications increased significantly from 28 in 2023 to 108 articles in 2024. The support vector machine (SVM) model outperformed others, achieving 82 % accuracy (AUC-ROC: 0.97), with high precision for categorizing research in Clinical Assistance Tools and strong recall for Diagnosis and Radiology Interpretation. Subgroup analysis showed that Patient Education achieved balanced performance (precision: 88 %, recall: 78 %, F1-score: 82 %), but overlapping terminology caused misclassifications between research and education domains. Temporal analysis predicted continued growth in these research domains, with Patient Education (+26 %) and Research and Ethics (+57 %) leading the way through 2027.

Conclusion

LLMs are exploring advancements in patient engagement, surgeon training, and orthopedic research, but challenges in reliability and ethics require careful implementation. Future work should focus on real-world validation, specialty-specific applications, and integrating multimodal AI systems. The SVM classifier demonstrated robust capabilities, providing a valuable tool for navigating the growing body of literature.