Deep-Cogn: Skeleton-based Human Action Recognition for Cognitive Behavior Assessment

Abstract

Skeleton-based human action recognition has received increasing attention in recent years. It aims at extracting features on top of human skeletons and estimating human pose. However, existing methods capture only the action information while in a real world application such as cognitive assessment, we need to measure the executive functioning that helps psychiatrists to identify some mental disease such as Alzheimer, Schizophrenia and ADHD. In this paper, we propose a skeleton-based action recognition named Deep-Cogn for cognitive assessment. Deep-Cogn integrates a pose estimator to extract the human body joints and then automatically measures the executive functioning employing the distance and elbow angle calculation. Three score functions were designed to measure the executive functioning: the accuracy score, the rhythm score and the functioning score. We evaluate our model on two different datasets and show that our approach significantly outperforms the existing methods.