Joint Behavioural Control of Autonomous Multi-Robot Systems for Lead-Follower Formation to Improve Human-Robot Interaction

Abstract

The effective autonomous cooperation between unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) has led to an increase in research and development for improving human robot interaction, due to their high potential for achieving successful missions in challenging environments. In this paper, a voice command system is developed to control a UAV and a vision-based approach system controls the UGV in a lead-follower formation to enhance human-robot teams for military defence and rescue missions in urban areas. The proposed voice control framework is executed using open source interface, Robot Operating System (ROS), for aerial vehicle communication supported by spoken dialogues using a wireless microphone, while the UGV detects and tracks a target image captured by the on board low-cost camera from the height of the UAV, using the proposed vision-based approach. The lead-follower formation is based on communication and visibility between the UAV and UGV in a given environment. In addition, a speech recognition module translates voice input to text and the published text is then mapped to the control input of the UAV in order to execute corresponding voice commands. The relative distance and position between the UAV and UGV is estimated from the image received via laptop. UGVs and UAVs have sophisticated design features such as network device communication, navigation, vision sensors and obstacle detection sensors. The UAV is a quad-copter that leads the UGV to accessible areas, while the UGV plays the role of the follower with wheel activators to track and locate the target image on the UAV, given a defined geographic path. The joint behavioural control system of the proposed voice command and vision-based approach is verified by experimental setups that show the UAV has a high voice recognition response and the UGV autonomously tracks the moving UAV in real time operation. As a consequence of the experimental tests, it is expected that the voice control system and vision-based approach developed play an important role in future combat fields.