Soundy: A Multimodal Audio Interface for Educational Music Production

Abstract

Multimodal interaction has emerged as a promising approach to enrich user experience and foster engagement in Educational Music Production (EMP) environments. Traditional audio interfaces and Digital Audio Workstations (DAWs), while powerful, often present engagement challenges, particularly for beginners, and tend to limit opportunities for spontaneous creativity and hands-on exploration during the learning process. To overcome these limitations, this paper introduces Soundy, a multimodal audio interface that integrates an on-board, lightweight web-based DAW. The system is accessible via a standard browser through a Local Area Network (LAN) and allows users to control audio production tasks using predefined and custom voice and facial commands. A comparative within-subject study was conducted involving 20 participants, each of whom tested both Soundy and a standard configuration composed of a Behringer UMC404HD audio interface and the Reaper DAW. The evaluation focused on two key dimensions, usability and engagement, measured using two standardized and validated instruments: The Post-Study System Usability Questionnaire (PSSUQ) and the User Engagement Scale - Short Form (UES-SF), complemented by open-ended participant feedback. Results revealed a trade-off between configurations: the standard setup ensured higher usability and task efficiency, while Soundy promoted greater engagement, creativity, and exploratory behavior. These findings suggest that embedded multimodal solutions based on voice and facial interaction hold strong potential for enhancing student experience in EMP and support future developments in adaptive command recognition and integrated hardware design.