Machine Learning Feasibility in Cochlear Implant Speech Perception Outcomes-Moving Beyond Single Biomarkers for Cochlear Implant Performance Prediction.

Abstract

Objectives: Machine learning (ML) is an emerging discipline centered around complex pattern matching and large data-based prediction modeling and can improve precision medicine healthcare. Cochlear implants (CI) are highly effective, however, outcomes vary widely, and accurately predicting speech perception performance outcomes between patients remains a challenge. This study aims to evaluate the ability of ML to predict speech perception performance among CI recipients at 6-month post-implantation using only preoperative variables on one of the largest CI datasets to date, with an emphasis placed on identification of poor performers.

Design: All patients enrolled in the national CI outcome tracking database, HERMES, and the institutional CI registry. Data were split 90/10 training/testing with hyperparameter tuning designed to optimize AUPRC performed during 10-fold cross-validation within 100 iterations. Multiple models were developed to predict final and delta (Δ) in consonant-nucleus-consonant (CNC) words and AzBio sentences at 6-month post-implantation. Two metrics, (1) final performance scores and (2) equally distributed 20th percentile performance ranking were used as primary outcomes. All models were compared with currently used "gold standard," defined as linear or logistic regression models leveraging Lazard features (LF). Final metrics for comparison included mean absolute error (MAE), calibration curves, heat accuracy maps, area under the receiver operating curve (AUROC), and F1 score.

Results: A total of 1877 patients were assessed through an ML pipeline. (1) XGBoost (XGB) predicted CNC with MAE of 17.4% (95% confidence interval [CI]: 17.34 to 17.53%) and AzBio with MAE of 20.39% (95% CI: 20.28 to 20.50%) and consistently outperformed linear regression with LF (CNC MAE 18.36% [95% CI: 18.25 to 18.47]; AzBio 21.62 [95% CI: 21.49 to 21.74]). Although statistically significant, the 1 to 2% boost of performance is clinically insignificant. (2) Predicting quintiles/20th percentile categories for CI performance, XGB outperformed logistic regression (Log-LF) across all metrics. XGB demonstrated superior calibration compared with Log-LF and provided a larger proportion of predicted probabilities predictions at the extremes (e.g., 0.1 or 0.9). XGB outperformed Log-LF predicting ≤40th percentile for CNC (AUROC: 0.708 versus 0.594; precision: 0.708 versus 0.596; F1 score: 0.708 versus 0.592) and AzBio (AUROC: 0.709 versus 0.572; precision: 0.710 versus 0.572; F1 score: 0.709 versus 0.572). This was consistent for ΔCNC and ΔAzBio. Last, accuracy heat maps demonstrated superior performance of XGB in stratifying sub-phenotypes/categories of CI performance compared with Log-LF.

Conclusions: This study demonstrates how ML models can offer superior performance in CI speech perception outcomes prediction modeling compared with current gold standard (Lazard-linear or logistic regression). ML offers novel insights capable of capturing nonlinear complex relationships and can identify novel sub-phenotypes at the extremes of CI performance using preoperative clinical variables alone. This is the first study to our knowledge to offer any type of meaningful preoperative stratification for CI speech perception outcomes and may have significant implications that need to be carefully explored when it comes to patient counseling, auditory rehabilitation, and future CI clinical trials. While prospective validation is a necessary next step and performance is still limited based on current traditional CI variables, these results highlight the potential of artificial intelligence (AI) in CI care, the critical need to integrate novel variables that better account for CI performance, and the need for improved data collaboration and standardized registries moving forward.