A RGB-D sensor and speech interaction based object manipulation system with improved CBR-BDI reasoning

Abstract

We design a kind of object manipulation system, which is based on Robot Operating System (ROS) distributed processing framework. This system can communicate with human beings; can percept the 3D environment by RGB-D sensor; has the ability of reasoning; can transfer the natural language intention to machine instruction to control the movement of manipulator. In particular, an improved Case-Based Reasoning-Belief-Desire-Intention (CBR-BDI) reasoning method is proposed. Three parts are added on the CBR-BDI reasoning system, which are map matching, desire analysis and guidance. Such an improvement makes the reasoning engine can carry out in-depth reasoning actively when the user's intention is incomplete and/or mismatched with actual scene. The system could analysis the information which user inputted, interact with human and environment, guide user by conversation, improve user's beliefs, finally get the standardization of complete intention and transfer the intention to machine instructions. Experiment results prove the validity and practicability of our proposed method.