Clinical Validation and Machine Learning Optimization of MyCog: A Self-Administered Cognitive Screener for Primary Care Settings.

Abstract

Background: Primary care presents an ideal opportunity for early detection of cognitive impairment, yet primary care clinics face barriers to cognitive screening. MyCog, an EHR-integrated tablet app that is self-administered during the rooming process of a primary care visit, streamlines the screening process to reduce barriers and encourage broader screening.

Methods: We compared MyCog performance from 65 adults with diagnosed cognitive impairment to 80 cognitively normal adults, all aged 65+, recruited from clinical settings. We leveraged the consensus of five machine learning models (LASSO, Elastic Net, Random Forest, Bayesian Logistic Regression, and Gradient Boosting) to select consistently discriminative variables for the final detection algorithm. Performance was assessed at for two evidence-based thresholds (Youden's J and Top Left) with ROC AUC, sensitivity, specificity, and accuracy as the primary metrics.

Results: All five models showed strong diagnostic performance, with ROC AUC values ranging from 0.839 to 0.876. The consensus modeling approach consistently identified the MyCog Picture Sequence Memory (PSM) exact match score and the MyCog Dimensional Change Card Sort (DCCS) overall rate-correct score as predictors of cognitive impairment. The final logistic regression achieved a robust AUC of 0.890. Depending on the cut point selected, sensitivity ranged from 0.723-0.785 (95% CI: 0.547-0.877), specificity 0.825-0.912 (95% CI: 0.716-0.954), accuracy 0.807-0.828 (95% CI: 0.731-0.869).

Discussion: MyCog provides practical, accurate cognitive screening for primary care. The sub-7-minute self-administered assessment eliminates staffing requirements and automates evaluation, addressing screening barriers to facilitate earlier detection and improve clinical outcomes. The algorithm's robust performance and parsimony demonstrate clinical utility while maintaining diagnostic accuracy.