MLSE of shaped BPSK in bandlimited nonlinear channels

Abstract

The application of the Viterbi algorithm (VA) to perform maximum-likelihood sequence estimation (MLSE) of shaped biphase-shift keyed (SBPSK) modulation is presented, and its performance characterized in channels typical of UHF military satellite communications by computer simulation. The channel models utilized consist of five-pole Chebyshev and 16 two-pole cascaded Butterworth filters followed, in some cases, by a nonlinear model for a traveling-wave tube (TWT) amplifier proposed by A. A. M. Saleh (1981). The VA detector is compared with integrate-and-dump (I&D) detection and a simplified metric calculation for the VA. The results show that even in severely bandlimited channels with the nonlinearity, VA detection gives improved performance. Without the nonlinearity, VA detection provides an E/sub b//N/sub o/ gain over I&D detection from two to three dB, depending on the amount of bandlimiting.<>