SignEvaluator: A Gesture and Sentence Characteristic-Based Sign Language Quality Assessment System

Abstract

Sign language is a basic form of communication for hearing-impaired individuals. An evaluation of the quality of sign language gestures helps improve the efficiency of sign language learning. This article proposes SignEvaluator, a sign language quality assessment system with a movement quality feature extractor and assessment generator. In the former, three quality measures are proposed for gestures and sentences. The trajectory of the palm is mapped onto position space with kernel density estimation. For finger movements, the instantaneous energy and curvature of the gesture signals are extracted with Bézier curves. Meanwhile, the performer's familiarity with gestures is indicated by the movement fluency metric of sentences. In the assessment generator, the final assessment results are calculated by combining the weights of different quality metrics and the confidence of different gesture levels. The results indicate that SignEvaluator obtained an F1-score of 0.89 for 702 sentences collected from 20 performers.