Human localization based on the fusion of vision and sound system

In this paper, a method for accurate human localization using a sequential fusion of sound and vision is proposed. Although the sound localization alone works well in most cases, there are situations such as noisy environment and small inter-microphone distance, which may produce wrong or poor results. A vision system also has deficiency, such as limited visual field. To solve these problems we propose a method that combines sound localization and vision in real time. Particularly, a robot finds rough location of the speaker via sound source localization, and then using vision to increase the accuracy of the location. Experimental results show that the proposed method is more accurate and reliable than the results of pure sound localization.