Deep learning-based DOA estimation for hybrid massive MIMO receive array with overlapped subarrays

Abstract As massive MIMO is a key technology in the future sixth generation (6G), the large-scale antenna arrays are widely considered in direction-of-arrival (DOA) estimation for they can provide larger aperture and higher estimation resolution. However, the conventional fully digital architecture requires one radio-frequency (RF) chain per antenna, and this is challenging for the high hardware costs and much more power consumption caused by the large number of RF chains. Therefore, an overlapped subarray (OSA) architecture-based hybrid massive MIMO array is proposed to reduce the hardware costs, and it can also have better DOA estimation accuracy compared to non-overlapped subarray (NOSA) architecture. The simulation results also show that the accuracy of the proposed OSA architecture has $$6^{\circ }$$ 6 ∘ advantage over the NOSA architecture with signal-to-noise ratio (SNR) at 10 dB. In addition, to improve the DOA estimation resolution, a deep learning (DL)-based estimator is proposed by combining convolution denoise autoencoder (CDAE) and deep neural network (DNN), where CDAE can remove the approximation error of sample covariance matrix (SCM) and DNN is used to perform high-resolution DOA estimation. From the simulation results, CDAE-DNN can achieve the accuracy lower bound at $$\textrm{SNR}=-8$$ SNR = - 8 dB and the number of snapshots $$N=100$$ N = 100 , this means it has better performance in poor communication situation and can save more software resources compared to conventional estimators.