A Multimodal Engagement-Aware Recommender System for People with Dementia

Dementia places an immeasurable burden on affected individuals and caregivers. In non-pharmacological therapy, physical, social, and cognitive activation of People with Dementia (PwD) is known to be crucial. However, effective activation requires sustained engagement. Technical activation systems thus require a means to automatically recognize if the user is engaged. While research has shown that engagement can be automatically recognized in healthy individuals, this task is especially challenging for PwD who might suffer from aphasia or blunted affect. In this project, I aim to investigate whether PwD provide sufficient verbal and non-verbal signals for the automatic recognition of engagement. Next, I aim to build a multimodal engagement recognition system for PwD using a technical activation system. Lastly, I aim to leverage this knowledge to build and evaluate an engagement-aware recommender system to promote the usage of engaging activation contents.