M4: Multi-proxy multi-gate mixture of experts network for multiple instance learning in histopathology image analysis

Abstract

Multiple instance learning (MIL) has been successfully applied for whole slide images (WSIs) analysis in computational pathology, enabling a wide range of prediction tasks from tumor subtyping to inferring genetic mutations and multi-omics biomarkers. However, existing MIL methods predominantly focus on single-task learning, resulting in not only overall low efficiency but also the overlook of inter-task relatedness. To address these issues, we proposed an adapted architecture of Multi-gate Mixture-of-experts with Multi-proxy for Multiple instance learning (M4), and applied this framework for simultaneous prediction of multiple genetic mutations from WSIs. The proposed M4 model has two main innovations: (1) adopting a multi-gate mixture-of-experts strategy for multiple genetic mutation simultaneous prediction on a single WSI; (2) introducing a multi-proxy CNN construction on the expert and gate networks to effectively and efficiently capture patch-patch interactions from WSI. Our model achieved significant improvements across five tested TCGA datasets in comparison to current state-of-the-art single-task methods. The code is available at: https://github.com/Bigyehahaha/M4.