Touch Point Prediction for Interactive Public Displays Based on Camera Images

Abstract

Feedback latency during the use of interactive displays is an issue currently being considered in the HCI field. Several studies have focused on reducing latency using various approaches. This paper proposes a framework that uses a convolutional neural network to predict user touch points for interactive public displays. The framework predicts user touch events before the finger reaches the display surface to reduce the latency in feedback. As a training dataset, 1,651 tapping actions were collected from 18 participants in front of a display. The training of the convolutional neural network architecture was performed using the collected tapping actions. Validation test results showed that reasonable accuracy could be achieved at 390 ms before touching the display.