Optimum noncoherent multiuser detection for nonlinear nonorthogonal modulation over the frequency-selective Rayleigh fading channel

The jointly optimum noncoherent multiuser detector for nonlinear modulation over the frequency selective Rayleigh fading multiple-access channel is obtained. The particular case of the frequency non-selective channel was previously considered by the authors in Russ and Varanasi (1997). Upper and lower bounds on average bit error probability of the optimum detector are derived for the frequency-selective channel. In several numerical examples, these bounds are seen to be asymptotically convergent in the high SNR regions. Remarkably, the error probability asymptote is found to be independent of the signal strengths of the interfering users. In stark contrast, the noncoherent detector which would be optimal in a single-user channel (the "conventional detector"), when used over the multiuser channel, exhibits an error floor for any set of non-zero powers of the interfering users.