Multi-Scale Hierarchical Context-Aware Survival Prediction Network based on whole slide images

Abstract

In high-resolution whole slide images (WSIs), multi-scale information is crucial for survival prediction. However, due to the ultra-large sizes of WSIs, existing methods have not fully utilized the multi-scale information at the gigapixel scale. Additionally, WSI-based survival prediction, as a patient-level multiple instance learning (MIL) task, is far more complex than WSI-level MIL which presents a significant challenge. To address these challenges, we propose a Multi-Scale Hierarchical Context-Aware Survival Prediction Network (MSASurv). This network progressively explores the tumor microenvironment, tumor-associated tissue structures, and patient-level tumor heterogeneity in WSIs. We validated our approach using five types of cancer from The Cancer Genome Atlas (TCGA), including 3,068 H&E-stained WSIs. Experimental results demonstrate that our proposed MSASurv algorithm outperforms previous weakly supervised methods by 3.3% to 16.8%. The code and models are publicly available at https://github.com/yatonghao/MSASurv.