Terrain Classification Based on Spatiotemporal Multihead Attention With Flexible Tactile Sensors Array

Abstract

Reliable task execution of wheeled platform requires high perceptive ability in terrains. Currently, vision perception is susceptible to external factors such as lighting conditions and air particles, and vibration perception reflects no surface features of terrains. In this article, we propose a novel system geared toward terrain classification based on tactile perception, well addressing those shortcomings. We develop a type of capacitive flexible tactile sensors array for 3-D forces with a wide measuring range, high sensitivity, considerable adaptability, and strong durability. To fully exploit the terrain features of the collected data, we propose a characterization method that encodes tactile information as image flow encompassing spatiotemporal information and establish a novel tactile-based terrain classification dataset. We construct the image flow as special tokens and feed them to a multihead spatiotemporal attention network, with spatial and temporal heads evenly constructed, to ultimately realize terrain classification. Our network achieves an accuracy of 91.9%, demonstrating the superiority over existing algorithms. Accuracies achieved are 81.3% and 76.3%, respectively, with 8-kg burden and at triple speed. Moreover, the performance degradation caused by increasing speed can be alleviated by decreasing time steps.