MICOO (multimodal interactive cubes for object orientation): a tangible user interface for the blind and visually impaired

Abstract

This paper presents the development of Multimodal Interactive Cubes for Object Orientation (MICOO) manipulatives. This system provides a multimodal tangible user interface (TUI), enabling people with visual impairments to create, modify and naturally interact with diagrams and graphs on a multitouch surface. The system supports a novel notion of active orientation and proximity tracking of manipulatives against diagram and graph components. If the orientation of a MICOO matches a component, then a user is allowed to modify that component by moving the MICOO. Conversely, if a MICOO does not match orientation or is far from a component, audio feedback is activated to help the user reach that component. This will lessen the need for manual intervention, enable independent discovery on the part of the user, and offers dynamic behavior, whereas the representation interacts and provides feedback to the user. The platform has been developed and it is undergoing formal evaluation (e.g., browse, modify and construct graphs on a Cartesian plot and diagrams).