Cancellation of ISI in non-linear voice-band data channels

A technique for cancellation of intersymbol interference (ISI) in nonlinear voice band data channels that contain second and third order nonlinear elements is presented. A data-aided nonlinear canceller structure based on a configuration proposed by E. Biglieri et al. (IEEE J. Sel. Areas Commun. vol.SAC-2, no.5, p.765-77, Sept. 1984) is used to remove nonlinear ISI without excessive noise enhancement. While the scheme proposed by Biglieri et al. uses a least mean square (LMS) adaptation algorithm, the authors choose to use a recursive least squares (RLS) adaptation algorithm in order to provide a faster convergence of the algorithm and improved robustness when applied to a variety of channels. A fast version of the RLS algorithm with computational complexity of order O(9N) (where N is the order of the nonlinear filter) is derived. The performance of the nonlinear canceller with both fast RLS and LMS adaptation is evaluated by simulation for 16-QAM (quadrature amplitude modulation) data transmission over nonlinear channels with a rate of 9600 bit/s. The simulation results show a performance improvement when the fast RLS version is used as compared with the LMS.<>