Personalized interactive gesture recognition assistive technology

Computing systems have the potential to contribute with interactive and low-cost solutions to support Augmentative and Alternative Communication (AAC) by applying different technologies to address different users' characteristics and needs. Computer Vision-based AAC systems can support users with motor difficulties by tracking and recognizing their remaining functional motions. To investigate possibilities for people with motor and speech impairments, this paper presents the PGCA: a Computer Vision system that allows the creation of a personalized gestural interaction as assistive technology for communication purposes. PGCA system takes into account the motor abilities and limitations of its users and the knowledge of caregivers in recognizing the gestures performed by the users. Results from interviews with special education professionals and from an experiment with the target audience suggest the use of personalized gestures is a common practice for AAC, and that creating custom datasets can be challenging, mainly due to the level of understanding of participants, the similarity between gestures, and variations in performing the same gestures. Improvements for the system were identified and described aiming to make the interface easier and more effective.