Interference and Phase Noise Mitigation in a Dual-Polarized Faster-than-Nyquist Transmission

Dual-polarized (DP) faster-than-Nyquist (FTN) transmission using higher-order modulation (HoM) schemes can significantly increase the spectral efficiency (SE) of the existing wireless backhaul links. However, FTN transmissions introduce inter-symbol interference (ISI), antenna polarization multiplexing suffers from cross-polarization interference (XPI) and HoM makes a communication system vulnerable to phase-noise (PN) distortions. In this work, we investigate for the first time a DP-FTN HoM transmission system that offers more than 150% increase in SE compared to a single-polarized Nyquist transmission. We propose an interference cancellation and PN mitigation structure coupled with adaptive decision-feedback equalization to jointly mitigate XPI, FTN and multipath ISI, and accomplish carrier-phase tracking. Numerical results confirm that a DP-FTN transmission can yield a 3–6 dB performance improvement over an equivalent DP-Nyquist system that uses a higher modulation order to achieve the same data rate.