An iterative approach for estimating domain-specific cognitive abilities from large scale online cognitive data

Abstract

Online cognitive tasks are gaining traction as scalable and cost-effective alternatives to traditional supervised assessments. However, variability in peoples’ home devices, visual and motor abilities, and speed-accuracy biases confound the specificity with which online tasks can measure cognitive abilities. To address these limitations, we developed IDoCT (Iterative Decomposition of Cognitive Tasks), a method for estimating domain-specific cognitive abilities and trial-difficulty scales from task performance timecourses in a data-driven manner while accounting for device and visuomotor latencies, unspecific cognitive processes and speed-accuracy trade-offs. IDoCT can operate with any computerised task where cognitive difficulty varies across trials. Using data from 388,757 adults, we show that IDoCT successfully dissociates cognitive abilities from these confounding factors. The resultant cognitive scores exhibit stronger dissociation of psychometric factors, improved cross-participants distributions, and meaningful demographic’s associations. We propose that IDoCT can enhance the precision of online cognitive assessments, especially in large scale clinical and research applications.