Modeling Uncertainty in Multi-Modal Fusion for Lung Cancer Survival Analysis

Abstract

Fusion of multimodal data is important for disease understanding. In this paper, we propose a new method of fusion exploiting the uncertainty in prediction produced by the individual modality learners. Specifically, we extend the joint label fusion method by taking model uncertainty into account when estimating correlations among predictions produced by different modalities. Through experimental study in survival prediction for non-small cell lung cancer patients who received surgical resection, we demonstrated promising performance produced by the proposed method.