Eric Coy, William Santo, Bonnie Jue, Helen Betts, Francisco Ramos-Gomez, Stuart A Gansky
{"title":"Among Artificial Intelligence/Machine Learning Methods, Automated Gradient-Boosting Models Accurately Score Intraoral Plaque in Non-Standardized Images.","authors":"Eric Coy, William Santo, Bonnie Jue, Helen Betts, Francisco Ramos-Gomez, Stuart A Gansky","doi":"10.1080/19424396.2024.2422146","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Previous automated models inaccurately scored non-standardized plaque images. The objectives were to develop and test automated image selection and intraoral plaque-scoring (primary outcome measure in a prevention trial for preschoolers).</p><p><strong>Methods: </strong>Evaluating 1650 plaque-disclosed primary teeth (teeth D, E, F, G) from 435 photographs from UCSF/UCLA clinical trials, data were cleaned, transformed, and modeled with statistical and machine learning (ML) algorithms; data visualizations utilized Jupyter Notebooks, Python, OpenCV, and Sci-kit Learn libraries, with Laplacian filter preprocessing. Image selection and plaque-scoring used 8 ML classification models. Mean plaque-scoring used 8 ML regression models. Models were tuned with 80:20 train:test split, stratified 5-fold cross-validation (5-CV) (unstratified in regression models), and hyperparameter optimization. Area-under-the-curve receiver operating characteristic (AUC-ROC) curve and R<sup>2</sup> determined the best classification and regression models, respectively, compared to calibrated dentist researcher ratings. Training time was a secondary metric. Manual segmentation used Photoshop's lasso tool. Average and dominant hue, saturation, and brightness values were features for training plaque-scoring algorithms.</p><p><strong>Results: </strong>Best performing models were: Support Vector Machine-Gaussian for image selection, 5-CV AUC-ROC of 0.99 and 0.76s of training time; Gradient-Boosting classification and regression models for individual teeth (5-CV AUC-ROC of 0.99 with 105s training); and mean plaque-scoring algorithms (5-CV R<sup>2</sup> of 0.72 with 1415s training).</p><p><strong>Conclusions: </strong>Accurate automated plaque-scoring is attainable without the high computational and financial costs of deep learning (DL) models. Automated plaque-scoring is attainable with little user-manipulation.</p><p><strong>Practical implications: </strong>Implementing automated tooth segmentation and synthetic sample generation with DL training may strengthen reliability, validity, and efficiency for clinical, research, and teledentistry applications by eliminating manual image preprocessing.</p>","PeriodicalId":94113,"journal":{"name":"Journal of the California Dental Association","volume":"52 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11845216/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of the California Dental Association","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1080/19424396.2024.2422146","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/11/26 0:00:00","PubModel":"Epub","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
Abstract
Background: Previous automated models inaccurately scored non-standardized plaque images. The objectives were to develop and test automated image selection and intraoral plaque-scoring (primary outcome measure in a prevention trial for preschoolers).
Methods: Evaluating 1650 plaque-disclosed primary teeth (teeth D, E, F, G) from 435 photographs from UCSF/UCLA clinical trials, data were cleaned, transformed, and modeled with statistical and machine learning (ML) algorithms; data visualizations utilized Jupyter Notebooks, Python, OpenCV, and Sci-kit Learn libraries, with Laplacian filter preprocessing. Image selection and plaque-scoring used 8 ML classification models. Mean plaque-scoring used 8 ML regression models. Models were tuned with 80:20 train:test split, stratified 5-fold cross-validation (5-CV) (unstratified in regression models), and hyperparameter optimization. Area-under-the-curve receiver operating characteristic (AUC-ROC) curve and R2 determined the best classification and regression models, respectively, compared to calibrated dentist researcher ratings. Training time was a secondary metric. Manual segmentation used Photoshop's lasso tool. Average and dominant hue, saturation, and brightness values were features for training plaque-scoring algorithms.
Results: Best performing models were: Support Vector Machine-Gaussian for image selection, 5-CV AUC-ROC of 0.99 and 0.76s of training time; Gradient-Boosting classification and regression models for individual teeth (5-CV AUC-ROC of 0.99 with 105s training); and mean plaque-scoring algorithms (5-CV R2 of 0.72 with 1415s training).
Conclusions: Accurate automated plaque-scoring is attainable without the high computational and financial costs of deep learning (DL) models. Automated plaque-scoring is attainable with little user-manipulation.
Practical implications: Implementing automated tooth segmentation and synthetic sample generation with DL training may strengthen reliability, validity, and efficiency for clinical, research, and teledentistry applications by eliminating manual image preprocessing.