Near-ML Detection over a Reduced Dimension Hypersphere

Abstract

In this paper, we propose a near-maximum likelihood (ML) detection method referred to as reduced dimension ML search (RD-MLS). The RD-MLS detector is based on a partitioned search method that divides the symbol space into two groups and searches over the vector space of one group instead of that comprising all of the symbols. First, a minimum mean square error (MMSE) dimension reduction operator suppressing the interference from the second group is applied, and then a list tree search (LTS) is performed over the symbols in the first group. For each lattice point of symbols for the first group found from the LTS, the rest of symbols are estimated by MMSE-decision feedback (MMSE-DF) estimation. Among these lattice point candidates, a final solution is chosen as a minimizer of the L2-norm criterion. From an asymptotic error probability analysis, we show that the dimension reduction loss is potentially compensated by the LTS gain proportional to the size of the list. Furthermore, we demonstrate through simulation on multi-input multi-output (MIMO) transmissions that the RD-MLS detector achieves substantial complexity reduction with relatively little performance loss over ML detection.