TSAA-Net: Triple-semantic-aware attention network for colorectal cancer grading from histology images

Colorectal cancer grading is crucial for follow-up treatment and overall patient prognosis. However, it is challenging to perform clinical practice because of these problems: 1) the class token derived from small image patches inadequately represents the label information of the entire histological image; 2) image patches often causes distortion in token feature details across the full image; 3) small patches always fail to incorporate the entire tissue micro-architecture, missing regions of interest. To solve such problems, a Triple-Semantic-Aware Attention Network (TSAA-Net) is proposed for colorectal cancer grading from histology images, where the Class-Token Semantic Aware Attention (CLTSAA) module is devised to capture global information by learning class tokens from different classifications; then, the Channel-Token Semantic Aware Attention (CHTSAA) module is designed to compensate for feature detail loss by refining local pixel-level information; finally, a Space-Token Semantic Aware Attention (SPTSAA) module is developed to integrate the entire tissue micro-architecture by capturing dual attention space-token information. Experiments results on two colorectal cancer datasets demonstrate the effectiveness of the proposed TSAA-Net, providing valuable support for pathologists in the grading of colorectal cancer.