Toffee: enabling ad hoc, around-device interaction with acoustic time-of-arrival correlation

Abstract

The simple fact that human fingers are large and mobile devices are small has led to the perennial issue of limited surface area for touch-based interactive tasks. In response, we have developed Toffee, a sensing approach that extends touch interaction beyond the small confines of a mobile device and onto ad hoc adjacent surfaces, most notably tabletops. This is achieved using a novel application of acoustic time differences of arrival (TDOA) correlation. Previous time-of-arrival based systems have required semi-permanent instrumentation of the surface and were too large for use in mobile devices. Our approach requires only a hard tabletop and gravity -- the latter acoustically couples mobile devices to surfaces. We conducted an evaluation, which shows that Toffee can accurately resolve the bearings of touch events (mean error of 4.3° with a laptop prototype). This enables radial interactions in an area many times larger than a mobile device; for example, virtual buttons that lie above, below and to the left and right.