Low-delay haptic texture display method based on user action information and texture image

In recent years, tool-mediated vibrotactile display of virtual surface texture has become a hot research topic in the field of haptics. When interacting with a virtual texture, incorporating the user’s interactive behavior into the haptic display process of the texture is an effective means, that can enhance the sense of realism in the interaction. To address the problems of weak generalization ability and low interactive realism in texture modeling and rendering, this paper proposes a haptic rendering model for known textures that varies with action conditions. This model takes texture image and the user’s real-time action information (velocity and normal force) as input. It introduces a self-attention mechanism to assign weights to each feature and combines the previous vibrotactile information to generate corresponding vibrotactile signals. Additionally, we designed a haptic device that integrates a real-time collection of action information and vibrotactile expression capabilities in combination with the 3D Systems Touch device, which together with the haptic rendering model, forms a haptic texture display system. In order to ensure that the system delay is below the perception threshold, this paper further introduced knowledge distillation to optimize the system delay. Based on this, we conducted three user experiments. The results show that our method not only has a certain generalization ability for new textures outside the database, but also can obtain a higher perceptual similarity score. In addition, the delay time of the system we tested is only 29$\sim$37 ms, which can bring users a more realistic texture perception experience.