Towards a visual Sign Language dataset for home care services

Abstract

We present our work towards creating a dataset, which is intended to be used for the implementation of a home care services system for the deaf. The dataset includes recorded realistic scenarios of interactions between deaf patients and mental health experts in their native sign language. The scenarios allow for contextualized representations, in contrast to typical datasets presenting isolated signs or sentences. It includes continuous videos in RGB and depth, which are challenging to analyze and closely resemble real-life scenarios. The research on representation of signs is supported by providing the hand shapes and trajectories for every video using hand and skeleton models, as well as facial features. Furthermore, the dataset may be used for studying the emotional context, since such conversations are typically emotionally charged.