ViMST: vision transformer-based dual modality multi-task graph contrastive network for spatial transcriptomics microenvironments investigation.

Abstract

Background: Investigating spatial transcriptomics microenvironments is crucial for unraveling cellular heterogeneity. Existing methods struggle to extract non-redundant information from histopathological images, as well as to simultaneously and spatially resolve gene expression profiles. We propose a vision transformer-based dual-modality multi-task graph contrastive network for exploring the spatial transcriptomics domain (ViMST), which integrates gene expression, image features, and spatial coordinates to investigate tissue microenvironments. It employs Vision Transformer (ViT) for feature extraction and dual masked Graph Convolutional Networks (GCNs) to model modalities separately. A novel joint topology decoder learns the spatial covariation between morphology and expression, thereby enhancing relationship modeling across multiple tasks.

Results: The evaluation results across nine spatial transcriptomics datasets reveal that ViMST consistently outperforms eight state-of-the-art methods in spatial domain identification and data denoising. It demonstrates robust performance in multiple tissue microenvironment research tasks, including data visualization, trajectory inference, identification of spatially variable genes (SVGs), horizontal integration analysis, cellular heterogeneity analysis, and epithelial-mesenchymal transition (EMT) studies.

Conclusions: ViMST is a powerful and versatile multimodal framework for spatial transcriptomics analysis. Its robust performance across multiple datasets and tasks highlights its broad applicability and practical value in deciphering tissue spatial organization. By integrating histological, spatial, and transcriptional information, ViMST enables comprehensive characterization of spatial heterogeneity and provides new opportunities for understanding disease mechanisms, identifying spatial biomarkers, and discovering potential therapeutic targets.