The utility of machine learning-based decision support system in referral of suspected rheumatic disease.

Objectives: The rising prevalence of rheumatic diseases (RD), coupled with a global shortage of rheumatologists, creates significant challenges for timely and accurate diagnosis. This study aimed to develop and evaluate an adaptive machine learning (ML)-based decision support system for facilitating accurate referral of patients with suspected RD to rheumatology clinics.

Methods: Participants attending a rheumatology outpatient clinic for the first time were enrolled in this study. A web-based survey, designed for patient accessibility, collected data on clinical symptoms associated with various rheumatic diseases. At the end of a 6-month follow-up, the rheumatologic disease status (correct referral/unnecessary referral) of the patients was added to the database. A fivefold cross-validation approach was employed to assess model performance. The reported results are the average of these five-fold models, reporting sensitivity, specificity, and area under the curve (AUC).

Results: During the 6-month follow-up period involving 843 participants, 574 were diagnosed with a rheumatologic disease. Overall, 31.9% of participants were found to have been referred unnecessarily. The ML model accurately identified patients who were appropriately referred, achieving a mean AUC of 77.9% (95% CI: 74.9%-80.9%), with a mean sensitivity of 87.1% (95% CI: 84.4%-89.8%), and a mean specificity of 67.8% (95% CI: 62.2%-73.3%) across five folds. The best-performing fold reached an AUC of 81.34% (95% CI: 78.58%-84.22%) with the sensitivity of 81.74% (78.58%- 4.90%) and a specificity of 80.95% (76.26%-85.64%). The addition of four questions (n=245) significantly improved performance metrics, with an AUC of 90.77% (95% CI 87.20-94.34), sensitivity of 89.74% (95% CI 85.14-94.34), and specificity of 92.05% (95% CI 86.05-98.05) for best fold.

Conclusions: This ML-based triage tool demonstrates strong potential for accurately identifying appropriate referrals, reducing unnecessary consultations, and enhancing resource utilisation in rheumatology clinics. Our results show that performance improved through an iterative, patient-feedback-driven refinement process. Future multicentre studies are needed for validation, and collaborative efforts will be essential to maximise its impact.