Extracting grasping, contact points and objects motion from assembly demonstration

This paper presents a framework to extract the grasping, contact points and object parts motion from an assembly demonstration. With this framework the object parts are recognized and tracked using Augmented Reality (AR) markers. The data of the user's hand assembling the object are acquired with a motion capture device. The grasping and contact points are determined with the motion capture data, the models of the object parts and point cloud based algorithms. The functionality of the framework is demonstrated with an experiment where the user assembles two parts of a toy airplane. The grasping and contact points between the object parts are extracted and visualized. This framework aims at capturing the necessary data to reproduce the assembly motion on a dual-arm robot for future work.