Evaluation of a task specific self-supervised learning framework in digital pathology relative to transfer learning approaches and existing foundation models.

Abstract

An integral stage in typical digital pathology workflows involves deriving specific features from tiles extracted from a tessellated whole slide image. Notably, various computer vision neural network architectures, particularly the ImageNet pre-trained, have been extensively used in this domain. This study critically analyzes multiple strategies for encoding tiles to understand the extent of transfer learning and identify the most effective approach. The study categorizes neural network performance into three weight initialization methods: random, ImageNet-based, and self-supervised learning. Additionally, we propose a framework based on task-specific self-supervised learning (TS-SSL) which introduces a shallow feature extraction method, employing a spatial-channel attention block to glean distinctive features optimized for histopathology intricacies. Across two different downstream classification tasks (patch classification, and weakly supervised whole slide image classification) with diverse classification datasets, including Colorectal cancer histology, Patch Camelyon, PANDA, TCGA and CIFAR-10, our task specific self-supervised encoding approach consistently outperforms other CNN-based encoders. The better performances highlight the potential of task-specific-attention based self-supervised training in tailoring feature extraction for histopathology, indicating a shift from utilizing pretrained models originating outside the histopathology domain. Our study supports the idea that task-specific self-supervised learning allows domain-specific feature extraction, encouraging a more focused analysis.