Context-aware sensor-doll as a music expression device

Abstract

We present a sensor-doll capable of music expression as a sympathetic communication device. The doll is equipped with a computer and various sensors such as a camera, microphone, accelerometer, and touch-sensitive sensors to recognize its own situation and the activities of the user. The doll has its own internal "mind" states reflecting different situated contexts. The user's multi-modal interaction with the passive doll is translated into musical expressions that depend on the state of mind of the doll.