Calibration-Jitter: Augmentation of hyperspectral data for improved surgical scene segmentation

Abstract

Semantic surgical scene segmentation is crucial for accurately identifying and delineating different tissue types during surgery, enhancing outcomes and reducing complications. Hyperspectral imaging provides detailed information beyond visible color filters, offering an enhanced view of tissue characteristics. Combined with machine learning, it supports critical tumor resection decisions. Traditional augmentations fail to effectively train machine learning models on illumination and sensor sensitivity variations. Learning to handle these variations is crucial to enable models to better generalize, ultimately enhancing their reliability in deployment. In this article, Calibration-Jitter is introduced, a spectral augmentation technique that leverages hyperspectral calibration variations to improve predictive performance. Evaluated on scene segmentation on a neurosurgical dataset, Calibration-Jitter achieved a F1-score of 74.35% with SegFormer, surpassing the previous best of 70.2%. This advancement addresses limitations of traditional augmentations, improving hyperspectral imaging segmentation performance.