CLAV: clustering latent vector aggregation for whole slide image retrieval leveraging foundation models

Abstract

Content-Based Image Retrieval (CBIR) is crucial in cancer diagnosis, assisting pathologists by providing similar image data from previous records for analysis, especially when there is uncertainty in diagnosing a case. This process supports decision-making by providing valuable reference points to guide the diagnostic process. Foundation models have become increasingly important in the medical field due to their ability to generalize across various tasks and datasets, offering valuable support to pathologists by enhancing the accuracy and efficiency of diagnostic processes. In this article, a foundation model pre-trained on histopathology data is leveraged as a feature extractor without the need for task-specific training, in contrast to existing models that require extensive training to learn significant data representations. The proposed method, Clustering Latent Vector Aggregation (CLAV), condenses the significant feature vectors into a unique representative vector for the Whole Slide Image (WSI). Using a unique feature vector offers the advantage of reducing the size of the memory bank, thereby making the process of querying and retrieving similar WSIs more efficient. The experimental results presented in this study demonstrate that the proposed method enhances performance in CBIR tasks. This article highlights the potential of foundation models to achieve superior retrieval metrics compared to state-of-the-art methods specifically trained for CBIR.