Decoding Cross‐Modal Haptic Neural Coupling Through EEG‐LSTM Spatiotemporal Modeling for Vibration−Roughness Interaction

Abstract

Haptic feedback is crucial for enhancing virtual immersion, but a neural coding mechanism that correlates the vibration frequency with surface roughness in haptic substitution remains unknown, which hinders the development of tribologically driven haptic interfaces. To address this limitation, this study models cross‐modal neural coupling between mechanical vibrations and roughness systematically through double‐blind experiments, event‐related potential analysis, and electroencephalography (EEG) space−time modeling based on the long short‐term memory (LSTM) method. By dynamically extracting the spatiotemporal dependence of the EEG signals by the LSTM method and quantifying neural representation similarity using Euclidean distances, this study reveals that cortical responses activated by specific vibration frequencies are highly consistent with natural roughness perception. In addition, the results of the behavioral verification confirm neurobehavioral consistency in perceptual equivalence. The results also show that vibration‐touch substitution can simulate roughness perception through frequency‐tuned neural coding. Further, this study proposes a cortical response‐aligned haptic framework that provides a theoretical paradigm for virtual reality and teleoperation applications, thus advancing tribological cross‐modal neural engineering.