PepperOSC: enabling interactive sonification of a robot’s expressive movement

Abstract This paper presents the design and development of PepperOSC, an interface that connects Pepper and NAO robots with sound production tools to enable the development of interactive sonification in human-robot interaction (HRI). The interface uses Open Sound Control (OSC) messages to stream kinematic data from robots to various sound design and music production tools. The goals of PepperOSC are twofold: (i) to provide a tool for HRI researchers in developing multimodal user interfaces through sonification, and (ii) to lower the barrier for sound designers to contribute to HRI. To demonstrate the potential use of PepperOSC, this paper also presents two applications we have conducted: (i) a course project by two master’s students who created a robot sound model in Pure Data, and (ii) a museum installation of Pepper robot, employing sound models developed by a sound designer and a composer/researcher in music technology using MaxMSP and SuperCollider respectively. Furthermore, we discuss the potential use cases of PepperOSC in social robotics and artistic contexts. These applications demonstrate the versatility of PepperOSC and its ability to explore diverse aesthetic strategies for robot movement sonification, offering a promising approach to enhance the effectiveness and appeal of human-robot interactions.