MTECC: A Multitask Learning Framework for Esophageal Cancer Analysis

Abstract

In the field of esophageal cancer diagnostics, the accurate identification and classification of tumors and adjacent tissues within whole slide images (WSIs) are critical. However, this task is complicated by the difficulty in annotating normal tissue on tumor-bearing slides, as the infiltration results in a blend of different tissue types, making annotation difficult for pathologists. To overcome this challenge, we introduce the multitask esophageal cancer classification (MTECC) framework, featuring an innovative dual-branch architecture that operates at both global and local levels. The framework initially employs a masked autoencoder (MAE) for self-supervised learning. A distinctive feature of MTECC is the integration of RandoMix, an innovative image augmentation technique that randomly exchanges patches between different images. This method significantly enhances the model's generalization ability, especially for recognizing tissues within cancerous slides. MTECC ingeniously integrates two tasks: tumor detection using global tokens, and fine-grained tissue classification at the patch level using local tokens. The empirical evaluation of the MTECC on our extensive esophageal cancer dataset substantiates its efficacy. The performance metrics indicate robust results, with an accuracy of 0.811, an F1 score of 0.735, and an AUC of 0.957. The MTECC method represents a significant advancement in applying deep learning to complex pathological image analysis, offering valuable tools for pathologists in diagnosing and treating esophageal cancer.