Mixed Reality MIDI Keyboard Demonstration

Abstract

The Mixed Reality MIDI Keyboard is a prototype designed to augment virtual reality experiences through the inclusion of a physical interface which aligns the user's senses with the virtual environment. It also serves as a platform on which the uses of virtual reality in music interaction and art installations can be experimented with. The main problem is that of synchronizing the real and virtual environments in a convincing way that makes the user feel more connected to the experience. To accomplish this a system of devices including an HTC Vive, Leap Motion hand tracker, and MIDI Keyboard are used together to produce a convincing mixed reality instrument that aligns with the user's visual, tactile and proprioceptive senses. The system is being developed as both a mixed reality musical instrument for use with common digital audio workstations, and as an installation piece which allows users to explore the nature of perception which this virtual reality system itself takes advantage of.