Deep Tactile Learning for Tumor Detection and Depth Classification in Robot-Assisted Surgery.

Abstract

This work presents a phantom-based proof-of-concept learning framework for robot-assisted palpation to detect tissue abnormalities and classify abnormality depth. A probe equipped with a rigidly mounted tactile sensor acquires spatially aligned pressure and indentation-depth maps during semi-autonomous, force-regulated exploration. A gated multi-task convolutional neural network (CNN) is developed to jointly perform tumor detection and tumor-positive depth classification using pressure-specific, indentation-specific, and fused pressure-indentation expert representations. Task-specific gates learn independent expert weightings for the two prediction tasks, while a masked loss restricts depth supervision to tumor-positive samples with valid depth labels. The framework learns spatial representations directly from paired pressure and indentation-depth maps without requiring an explicitly computed stiffness map as a primary model input. The method is evaluated using operator-independent and uneven-surface phantom test data. On the combined independent test set, the proposed model achieved a tumor-detection accuracy of 95.2%, an ROC-AUC of 0.934, an F1-score of 0.889, and a specificity of 98.0%. For tumor-positive depth classification, it achieved an accuracy of 85.7%, a mean absolute error of 0.143, and a quadratic weighted kappa of 0.811. Comparisons with pressure-only, indentation-only, stiffness-based, and multimodal baseline models indicate that models combining pressure and indentation-depth information produced the strongest depth-classification results. The proposed task-specific multi-head formulation also yielded the highest tumor-detection accuracy and F1-score point estimates while matching the strongest depth results.