SepLocNet: Multi-speaker localization with separation-guided TDOA estimation in wireless acoustic sensor networks

Abstract

Time difference of arrival (TDOA)-based multi-speaker localization allows three-dimensional localization using a low-cost wireless acoustic sensor network (WASN). However, two obstacles hinder accurate localization of multiple speakers. Firstly, the performance of existing multi-speaker TDOA estimation methods suffers from reverberation and noise. Secondly, the correct association of TDOA estimates and speakers across multiple sensor nodes needs to be solved. To address these issues, we develop a hybrid approach that combines the excellent performance of deep learning-based TDOA estimation with the interpretability of signal processing-based TDOA association, referred to as SepLocNet. In each node of a WASN, a speech separation-guided algorithm is used to accurately estimate multi-speaker TDOAs and construct auxiliary association features in reverberant noisy environments. In the fusion center, the similarity of the auxiliary association features is exploited to readily define the cost of associating TDOA estimates from multiple nodes to the same speaker. With these association costs, an efficient algorithm is derived to solve the TDOA association problem based on difference of convex functions programming. Finally, the locations of multiple speakers are readily estimated by imposing a well-established single-source localization method on the estimated TDOA associations. Experimental results show the proposed SepLocNet achieves much higher accuracy in terms of multi-speaker TDOA estimation and position estimation than existing signal processing-based algorithms. Moreover, the computational time at the fusion center is comparable.