SEMarbeta: mobile sketch-gesture-video remote support for car drivers

Abstract

Uneven knowledge distribution is often an issue in remote support systems, creating the occasional need for additional information layers that extend beyond plain videoconference and shared workspaces. This paper introduces SEMarbeta, a remote support system designed for car drivers in need of help from an office-bound professional expert. We introduce a design concept and its technical implementation using low-cost hardware and techniques inspired by augmented reality research. In this setup, the driver uses a portable Android tablet PC while the expert mechanic uses a stationary computer equipped with a video camera capturing his gestures and sketches. Hence, verbal instructions can be combined with supportive gestures and sketches added by the expert mechanic to the car's video display. To validate this concept, we carried out a user study involving two typical automotive repair tasks: checking engine oil and examining fuses. Based on these tasks and following a between-group (drivers and expert mechanics) design, we compared voice-only with additional sketch- and gesture-overlay on video screenshots measuring objective and perceived quality of help. Results indicate that sketch- and gesture-overlay can benefit remote car support in typical breakdown situations.