Sonifying gait kinematics: generating water wading sounds through a digital Foley approach

Abstract

In the growing field of sonic interaction design, increasing emphasis is being placed on walking-based interactions within an array of applications, including virtual reality, interactive media, and rehabilitation. Our study focuses on recreating the aural experience of wading through water, specifically the challenge of eliciting accurate and natural movement-sound associations for wading, both in interactive and listening-only contexts. We engineered a real-time wading simulation using a digital Foley technique that maps lower limb angular velocity contours to the intensity of steady-state flowing sounds. Our first iteration was evaluated in a perceptual experiment involving 16 participants, as well as an interactive test with 9 participants, which revealed the need for additional sensors along with time-domain preprocessing to ensure a consistently natural sound envelope across walking cadences (step tempi). We then refined the mapping function and incorporated more sound layers. In our subsequent listening test, 55 participants compared the realism of the initial and refined versions with real-life wading sounds at various step cadences. While the refined version demonstrated a notable improvement over the initial one and was deemed fairly realistic overall, it fell just short of the authenticity of the real-life recordings at faster cadences, suggesting room for further improvement of our simulation. Nevertheless, this research marks a stride in the evolution of walking-based sonic interactions, instigating wider acceptance and application of such systems in the future.