Cognitive status assessment of older adults - test administration by conversational artificial intelligence (AI) chatbot: proof-of-concept investigation.

Background: The Telephone Interview for Cognitive Status-Modified (TICS-M) is a widely utilized tool for remotely assessing cognitive function, particularly among community-dwelling older adults who are unable to attend in-person evaluations. In healthcare, AI has the potential to enhance service delivery by increasing efficiency, expanding accessibility, and reducing the cost per service. Using a conversational AI chatbot, we automated administration of TICS-M (traditionally administered by psychologists), referring to this chatbot-administered version as TICS-M-AI. The aim was to investigate proof-of-concept for chatbot automation of cognitive assessment. We report three studies evaluating psychometric properties of TICS-M-AI and an additional study on safety.

Method: Study1: Concurrent validity of the TICS-M-AI was assessed by administration of the TICS-M (by Psychologist) and the TICS-M-AI to the same participants (n = 100), one week apart. Study 2: Test-retest reliability was assessed by administering the TICS-M-AI twice to each participant, one week apart (n = 82) and comparing results. Study 3: Construct validity was assessed by attempted replication, using TICS-M-AI data (n = 264), of a previously published study by Lindgren et al. (2019) of item response patterns observed using data obtained by traditional clinician administered TICS-M. Study 4: Safety was assessed by comparing rates of reported assessment-related distress between TICS-M (n = 100) and TICS-M-AI (n = 264) administrations.

Results: TICS-M-AI concurrent validity (r = 0.81, 88% classification agreement, κ = 0.73) with the TICS-M and good test-retest reliability (r = 0.76, ICC = 0.72, 83% agreement, κ = 0.65). Using the TICS-M-AI we replicated Lindgren et al. (2019) result which used the TICS-M.

Conclusions: TICS-M-AI administered by an AI chatbot performed well compared to traditional TICS-M administration by a psychologist. TICS-M-AI is reliable, valid, and equally safe with added advantages of lower cost, scalability, and broader accessibility. Future research should address generalizability across diverse populations and refine AI adaptability.