Deep learning-based pathway-centric approach to characterize recurrent hepatocellular carcinoma after liver transplantation

Liver transplantation (LT) is offered as a cure for Hepatocellular carcinoma (HCC), however 15–20% develop recurrence post-transplant which tends to be aggressive. In this study, we examined the transcriptome profiles of patients with recurrent HCC to identify differentially expressed genes (DEGs), the involved pathways, biological functions, and potential gene signatures of recurrent HCC post-transplant using deep machine learning (ML) methodology. We analyzed the transcriptomic profiles of primary and recurrent tumor samples from 7 pairs of patients who underwent LT. Following differential gene expression analysis, we performed pathway enrichment, gene ontology (GO) analyses and protein-protein interactions (PPIs) with top 10 hub gene networks. We also predicted the landscape of infiltrating immune cells using Cibersortx. We next develop pathway and GO term-based deep learning models leveraging primary tissue gene expression data from The Cancer Genome Atlas (TCGA) to identify gene signatures in recurrent HCC. The PI3K/Akt signaling pathway and cytokine-mediated signaling pathway were particularly activated in HCC recurrence. The recurrent tumors exhibited upregulation of an immune-escape related gene, CD274, in the top 10 hub gene analysis. Significantly higher infiltration of monocytes and lower M1 macrophages were found in recurrent HCC tumors. Our deep learning approach identified a 20-gene signature in recurrent HCC. Amongst the 20 genes, through multiple analysis, IL6 was found to be significantly associated with HCC recurrence. Our deep learning approach identified PI3K/Akt signaling as potentially regulating cytokine-mediated functions and the expression of immune escape genes, leading to alterations in the pattern of immune cell infiltration. In conclusion, IL6 was identified to play an important role in HCC recurrence.