A Novel LSTM Architecture for Automatic Modulation Recognition: Comparative Analysis With Conventional Machine Learning and RNN-Based Approaches

The recognition of modulation types in received signals is essential for signal detection and demodulation, with broad applications in telecommunications, defense, and wireless communications. This paper introduces a pioneering approach to automatic modulation recognition (AMR) through the development of a highly optimized long short-term memory (LSTM) network. The proposed framework is engineered to capture intricate temporal dependencies within modulated signals, leveraging a gated architecture that effectively mitigates the vanishing gradient problem. This innovation markedly improves recognition accuracy, particularly in low-SNR conditions where traditional methods are often limited. A defining contribution of this work is the introduction of a novel, adaptive temporal-spectral feature learning mechanism, which seamlessly integrates both temporal and spectral characteristics of the signal. This paradigm eliminates the need for manual feature extraction, enhances interpretability, and significantly boosts classification efficiency. Furthermore, the proposed framework is designed for low-complexity deployment, ensuring its scalability and suitability for next-generation wireless networks and real-time communication systems. The proposed architecture is capable of distinguishing between seven modulation classes: BASK, 4-ASK, BFSK, 4-FSK, BPSK, 4-PSK, and 16-QAM. Performance is evaluated across a broad range of signal-to-noise ratios (SNR), from −10 dB to +30 dB, through extensive simulations. Experimental results demonstrate that the model achieves a recognition accuracy of 99.87% at an SNR of -5 dB, outperforming several conventional machine learning techniques, including multi-layer perceptron (MLP), radial basis function (RBF) networks, adaptive neuro-fuzzy inference systems (ANFIS), decision trees (DT), naïve Bayes (NB), support vector machines (SVM), probabilistic neural networks (PNN), k-nearest neighbors (KNN), and ensemble learning models, as well as recurrent neural networks (RNNs). Comparative analysis reveals that the proposed framework outperforms conventional machine learning techniques, with accuracy improvements ranging from 1.77% to 34.03% over the best- and worst-performing methods. Additionally, the proposed model achieves a performance gain of 2.02% over the deep learning (DL)-based RNN, further highlighting its superior capability in AMR.