View-based multi-touch gesture interface for furniture manipulation robots

This paper presents a user interface for manipulating structured furniture and electric equipments based on view-based multi-touch gesture interface and demonstrational mechanism for action candidates. The contribution of this papers is summarized as follows: 1) we define multi-touch gesture for push, pull and rotate manipulation of the robot. 2) We propose demonstrational feedback mechanism of daily environment manipulation. 3) In order to obtain a 3D point that corresponds to the user-touched point on the interface, we show a method to estimate 3D points from screen points of the robot's view images. A prototype system has been implemented using iPad browser and we have evaluated the prototype system on our office and kitchen within environment and showed some preliminary results on usability assessment.