Bridging imaging and genomics: Domain knowledge guided spatial transcriptomics analysis

Abstract

Spatial Transcriptomics (ST) provides spatially resolved gene expression distributions mapped onto high-resolution Whole Slide Images (WSIs), revealing the association between cellular morphology and gene expression profiles. However, the high costs and equipment constraints associated with ST data collection have led to a scarcity of ST datasets. Moreover, existing ST datasets often exhibit sparse gene expression distributions, which limit the accuracy and generalizability of gene expression prediction models derived from WSIs. To address these challenges, we propose DomainST (Domain knowledge-guided Spatial Transcriptomics analysis), a novel framework that leverages domain knowledge through Large Language Models (LLMs) to extract effective gene representations and utilizes foundation models to obtain robust image features for enhanced spatial gene expression prediction. Specifically, we utilize public gene reference databases to retrieve comprehensive gene summaries and employ LLMs to refine gene descriptions and generate informative gene embeddings. Concurrently, we apply medical visual-language foundation models to distill robust image representations at multiple scales, capturing the spatial context of WSIs. We further design a multimodal mixture of experts fusion module to effectively integrate multimodal data, leveraging complementary information across modalities. Extensive experiments conducted on three public ST datasets indicate that our method consistently outperforms state-of-the-art (SOTA) methods, with increases ranging from 6.7 % to 13.7 % in PCC@50 across all datasets compared to the SOTA, demonstrating the effectiveness of combining foundation models and LLM-derived domain knowledge for gene expression prediction. Our code and gene features are available at https://github.com/coffeeNtv/DomainST.