Quantum-inspired multimodal fusion with Lindblad master equation for sentiment analysis

Abstract

In multimodal sentiment analysis, the primary challenge lies in effectively modeling the complicated interactions among different data modalities. A promising approach is leveraging quantum concepts like superposition and entanglement to enhance the feature representation ability. However, existing quantum-inspired models neglect the intricate nonlinear dynamics inside their multimodal components. Drawing inspiration from the Lindbladian concept in quantum mechanics, we proposes quantum-inspired neural network with the Lindblad Master Equation (LME) and complex-valued LSTM. The proposed model treats each modality as an individual quantum system and superposes them into a mixed quantum system. The trainable LME process models the interaction of this multimodal system with its semantic environment, thereby enhancing the representation of complex interactions among modalities. The efficacy of the proposed model, along with its key components, are validated through extensive experiments on the MVSA and Memotion datasets. The performance are complemented by a comparative analysis that benchmarks the model against state-of-the-art methods, including traditional methods, large language models and quantum-insipred methods. Furthermore, the interpretability of the model is enhanced by quantifying the entanglement entropy of modality combinations using the von-Neumann Entanglement entropy.