Leveraging Swin Transformer for advanced sentiment analysis: a new paradigm.

Abstract

As healthcare text data becomes increasingly complex, it is vital for sentiment analysis to capture local patterns and global contextual dependencies. In this paper, we propose a hybrid Swin Transformer-BiLSTM-Spatial MLP (Swin-MLP) model that leverages hierarchical attention, shifted-window mechanisms, and spatial MLP layers to extract features from domain-specific healthcare text better. The framework is tested on domain-specific datasets for Drug Review and Medical Text, and performance is assessed against baseline models (BERT, LSTM, and GRU). Our findings show that the Swin-MLP model performs significantly better overall, achieving superior metrics (accuracy, precision, recall, F1-score, and AUC) and improving mean accuracy by 1-2% over BERT. Statistical tests to assess significance (McNemar's test and paired t-test) indicate that improvements are statistically significant (p < 0.05), suggesting the efficacy of the architectural innovations. The results' implications indicate that the model is robust, efficiently converges to classification, and is potentially helpful for a wide range of domain-specific sentiment analyses in healthcare. We will examine future research directions into exploring lightweight attention mechanisms, cross-domain multimodal sentiment analysis, federated learning to protect privacy, and hardware implications for rapid training and inference.