A hypergraph-based model for tumor prognosis using local and global information fusion on H&E-stained histology images

Abstract

Prognostic variables play a critical role in guiding clinical treatment decisions for cancer patients. However, extracting prognostic information from gigapixel histopathology slides remains a significant challenge. While attention-based deep learning models trained on histologic images have been extensively investigated, existing approaches often fail to effectively model slide-level contextual information or demonstrate generalizability across diverse cancer types and multi-center datasets. We propose a Hypergraph-based Multi-instance Contrastive Reinforcement learning model (HeMiCoRe), which integrates cluster-restricted local features and cross-cluster global representations from 5196 H&E-stained slides across 10 cancer types, leveraging both morphological and spatial relationships. HeMiCoRe employs hypergraph neural networks to predict patient survival outcomes and achieves state-of-the-art (SOTA) performance on 8 cancer types, demonstrating superior generalization compared to existing weakly supervised methods. This framework holds promise for clinical adoption, offering a robust tool for cancer prognosis and supporting treatment decision-making.