Validation of an artificial intelligence-based automated PRAGMA and mucus plugging algorithm in pediatric cystic fibrosis

Abstract

Background

PRAGMA-CF is a clinically validated visual chest CT scoring method, quantifying relevant components of structural airway damage in CF. We aimed to validate a newly developed AI-based automated PRAGMA-AI and Mucus Plugging algorithm using the visual PRAGMA-CF as reference.

Material and Methods

The study included 363 retrospective chest CT’s of 178 CF patients (100 New-Zealand and Australian, 78 Dutch) with at least one inspiratory CT matching the image selection criteria. Eligible CT scans were analyzed using visual PRAGMA-CF, automated PRAGMA-AI and Mucus Plugging algorithm. Outcomes were compared using descriptive statistics, correlation, intra- and interclass correlation and Bland-Altman plots. Sensitivity analyses evaluated the impact of disease severity, study cohort, number of slices and convolution kernel (soft vs. hard).

Results

The algorithm successfully analyzed 353 (97 %) CT scans. A strong correlation between the methods was found for %bronchiectasis ( %BE) and %disease ( %DIS), but weak for %Airway wall thickening ( %AWT). The automated Mucus plugging outcomes showed strong correlation with visual %mucus plugging ( %MP). ICC’s between visual and automated sub-scores witnessed average agreement for %BE and %DIS, except for %AWT which was weak. Sensitivity analyses revealed that convolution kernel did not affect the correlation between visual and automated outcomes, but harder kernels yielded lower disease scores, especially for %BE and %AWT.

Conclusion

Our results show that AI-derived outcomes are not identical to visual PRAGMA-CF scores in size, but strongly correlated on measures of bronchiectasis, bronchial-disease and mucus plugging. They could therefore be a promising alternative for time-consuming visual scoring, especially in larger studies.