Motion initiated music ensemble with sensors for motor rehabilitation

Abstract

Advances in inertial sensors technology used in smartwatches make it possible for movement-based therapies to be both therapeutic and evaluative at the same time. Therapeutic Instrumental Music Performance (TIMP) uses instrument playing for motor rehabilitation. However, applications of TIMP are not generalizable to a variety of motor disabilities and do not often consider the benefits of social interaction. To address the missing opportunities and potential strengths of an integrated technological and therapeutic application, we introduce MIMES (Motion Initiated Music Ensemble with Sensors). MIMES uses commercially available smartwatches and hand gesture recognition techniques to trigger instrumental sounds with simple hand gestures and support multiple users. We developed this system to provide rehabilitation programs and music therapy for a variety of individuals with motor impairments to improve both physical and mental health outcomes. We discuss MIMES system architecture, gesture recognition algorithm, future experimental design and a self-evaluation module to test the efficacy of the technology for rehabilitation.