Building and Evaluating an Orthodontic Natural Language Processing Model for Automated Clinical Note Information Extraction.

Abstract

Introduction: Malocclusion presents functional and aesthetic challenges, necessitating accurate diagnosis and treatment. However, variability in orthodontic treatment planning persists due to subjective assessments, limiting consistency and objectivity. Electronic dental records (EDRs) contain vast patient data that could address these challenges, but much of the rich clinical information is documented as free text, complicating analysis. This study aims to develop an Orthodontic Natural Language Processing (ONLP) model to extract structured orthodontics-related information from unstructured EDRs and identify critical features influencing malocclusion using machine learning (ML).

Methods: Data from 7693 orthodontic patients were analysed to train, test and validate the ONLP and ML models. A gold-standard dataset was created through manual review. The ONLP model utilised supervised (Named Entity Recognition-NER) and unsupervised (K-means clustering) approaches to structure information from free text. Machine learning models, including Logistic Regression, Gaussian Naive Bayes, Random Forest and XGBoost, were subsequently applied to identify feature importance for malocclusion classification.

Results: The ONLP model achieved 89% sensitivity, 92% specificity and 91% accuracy in extracting orthodontics-related information. The supervised model demonstrated 84% accuracy, 82% F1-score and 84% recall, excelling in identifying Classes I and III malocclusions but showing reduced sensitivity for Class II. Machine learning analysis highlighted key features for malocclusion classification: maxillary crowding, overjet and arch perimeter discrepancy for Class I; maxillary spacing and anterior crossbite for Class II; and dental midline deviation and occlusal wear for Class III.

Conclusion: This study demonstrates a novel approach to automating orthodontic data extraction using the ONLP model, enabling advanced big data analytics and enhancing data-driven orthodontic research and care.