AI-based assessment of Clinical Activity Score and detection of active thyroid eye disease using facial images: validation of Glandy CAS.

Abstract

Purpose: The Clinical Activity Score (CAS) is widely used to assess thyroid eye disease (TED) activity but can vary based on the evaluator's expertise. We developed and externally validated Glandy CAS, a machine learning (ML)-assisted system for detecting active TED (CAS ≥3) using digital facial images. This clinical trial aimed to gain approval from the Korea Ministry of Food and Drug Safety (KMFDS) for this Software as a Medical Device (SaMD).

Methods: This is a clinical trial based on the retrospective cohort. Glandy CAS analysed 756 photos of patients with TED, classifying them as having active or inactive TED. Its diagnostic performance was compared with that of three general ophthalmologists (less than 5 years of experience), using the F1 score. The reference CAS was determined by an oculoplastic specialist.

Results: Active TED was detected in 207 of 756 patients. Glandy CAS achieved a sensitivity of 87.9%, specificity of 95.8% and an F1 score of 0.88. In comparison, general ophthalmologists had a sensitivity of 60.4%, specificity of 83.0% and an F1 score of 0.57. Glandy CAS predicted CAS within 1 point of the reference score in 82.3% of cases, with a mean absolute error of 0.83.

Conclusions: Glandy CAS, an ML-assisted system for detecting active TED using facial images, showed high accuracy and outperformed general ophthalmologists. This system can consistently and accurately assess disease activity, facilitating early detection and timely treatment of active TED. Based on this clinical trial, the SaMD received KMFDS approval (Product Licence No., 24-93).