Human-Computer Interaction using Finger Signing Recognition with Hand Palm Centroid PSO Search and Skin-Color Classification and Segmentation

Abstract

This paper presents a novel image processing technique for recognizing finger signs language alphabet. A human-computer interaction system is built based on the recognition of sign language which constitutes an interface between the computer and hearing-impaired persons, or as an assistive technology in industrial robotics. The sign language recognition is articulated on the extraction of the contours of the sign language alphabets, therefore, converting image recognition into one dimensional signal processing, which improves the recognition efficiency and significantly reduces the processing time. The pre-processing of images is performed by a novel skin-color region segmentation defined inside the standard RGB (sRGB) color space, then a morphological filtering is used for non-skin residuals removal. Afterwards, a circular correlation achieves the identification of the sign language after extracting the sign closed contour vector and performing matching between extracted vector and target alphabets vectors. The closed contour vector is generated around the hand palm centroid with position optimized by a particle swarm optimization algorithm search. Finally, a multi-objective function is used for computing the recognition score. The results presented in this paper for skin color segmentation, centroid search and pattern recognition show high effectiveness of the novel artificial vision engine.