A probabilistic neural network-based bimanual control method with multimodal haptic perception fusion

Abstract

In the master-slave robot system, single-modal tactile perception has problems such as collision detection delay (>120 ms), force estimation error (>2.3 N), and sensor conflicts, resulting in a 37 % failure rate of robot operations in nuclear decommissioning scenarios and a 19.2 % risk of excessive tissue compression in laparoscopic surgery. To address this, this paper proposes a multimodal tactile perception fusion control method based on a probabilistic neural network (PNN). Pressure, vibration, and temperature signals are synchronously collected through bionic artificial skin. A hierarchical heterogeneous feature alignment (HHFA) module is designed to solve the spatio-temporal asynchrony of multi-source signals (root mean square error <0.8 ms), and a dynamic Bayesian fusion layer (DBFL) is developed to achieve adaptive weighting based on the entropy-variance coupling index, suppressing noise interference and modal conflicts. The dual-channel PNN encodes the fused sensory information into a Gaussian mixture model (8 components) and generates high-precision control instructions by maximizing the posterior probability. Experiments show that in grasping and fine operation tasks, the positioning error of this method is reduced to 0.15 mm, the operation success rate is increased by 19.6 % (reaching 96.4 %), and the signal-to-noise ratio remains stable at $40.2\pm 1.5\mathrm{dB}$ under humidity changes (30–90 %RH) and mechanical strain (15 %).