Validity and usability for digital cognitive assessment tools to screen for mild cognitive impairment: a randomized crossover trial.

Abstract

Background: The practicality of implementing digital cognitive screening tests in primary health care (PHC) for the detection of cognitive impairments, particularly among populations with lower education levels, remains unclear. The aim of this study is to assess the validity and usability of digital cognitive screening tests in PHC settings.

Methods: We utilized a randomized crossover design, whereby 47 community-dwelling participants aged 65 and above were randomized into two groups. One group completed the paper-based Mini-Mental State Examination (MMSE) and Clock Drawing Test (CDT) first, followed by the tablet-based digital version after a two-week washout period, while the other group did the reverse. Validity was assessed by Spearman correlation, linear mixed-effects models, sensitivity specificity, and area under the curve (AUC). Usability was assessed through the Usefulness, Satisfaction, and Ease of Use (USE) questionnaire, participant preferences and assessment duration. Regression analyses were conducted to explore the impact of usability on digital test scores, controlling for cognitive level, education, age, and gender.

Results: Regarding validity, digital tests showed moderate correlations with paper-based versions and superior AUC performance. The AUC was 0.65 for the MMSE versus 0.82 for the electronic MMSE (eMMSE), and 0.45 for the CDT compared to 0.65 for the electronic CDT (eCDT). Regarding usability, while older participants gave positive feedback on digital tests (P < 0.001), they preferred paper-based versions. The eMMSE took significantly longer to complete than the MMSE, averaging 7.11 min versus 6.21 min (P = 0.01). Notably, digital test scores were minimally affected by subjective attitudes but strongly linked to test duration (β = -0.62, 95% CI: -1.07 to -0.17).

Conclusions: Digital cognitive tests are valid and feasible in PHC settings but face implementation challenges, especially in usability and adaptability among individuals with lower education levels.