Biosignal Sequence Real-Time Prediction for Game Users Based on Features Fusion of Local–Global and Time–Frequency Domain

Abstract

Biosignal sequence real-time prediction (BSRP) is essential for predicting the future emotional experience of game users. However, BSRP for game users faces challenges, including poor real-time performance and limited feature fusion dimensions. To address these issues, we proposed a method for BSRP based on the features fusion of local–global and time–frequency domain (LGTF) for game users, which integrates real-time capabilities with multidimensional features fusion. Specifically, LGTF meets real-time requirements and achieves the features fusion of local–global (LG) through multichannel synchronized adaptive convolution. In addition, LGTF implements the features fusion of interband and intraband in the frequency domain and the features fusion of time–frequency (TF) domain by incorporating the self-attention mechanism and Fourier Transform. Furthermore, we conducted comprehensive validation experiments on LGTF using the public dataset. The results indicate that: first, in the comparison study, LGTF outperformed other methods, achieving the lowest average mean squared error (MSE) and mean absolute error values across different prediction lengths of 0.61 and 0.47, respectively. Second, ablation studies revealed that the addition of TF domain feature fusion and LG feature fusion both have the positive effect on the prediction performance, reducing the average MSE by 0.11 and 0.09, respectively. Third, generalization study shows that LGTF exhibits stable performance and generalization across different subjects and shows performance advantages in specific game scenarios. Fourth, time performance analysis suggests LGTF has the real-time performance. Finally, case study demonstrates that LGTF is practical for predicting game users' future emotions and enhancing their emotional experiences.