Utilizing human vision and computer vision to direct a robot in a semi-structured environment via task-level commands

A novel approach to directing a highly autonomous robot operating in a semi-structured environment is presented. In this approach, the human operator assists the robot in perceiving unexpected situations in the environment through simple point-and-click type interaction with a live video display from cameras on-board the robot. As a result of this high-level guidance, the robot is now able to invoke a variety of computer vision algorithms to augment the world model accordingly. This novel approach utilizes the complimentary vision capabilities of both the human and computer to extend the capability of the human/robot team to overcome the challenges of semi-structured environments without sacrificing the high-degree of autonomy and resilience to time delay of the task-level command architecture. Preliminary experimental results with a laboratory robot are presented.