GeneRAIN: multifaceted representation of genes via deep learning of gene expression networks

Abstract

We develop GeneRAIN, a suite of Transformer-based models that learn gene expression relationships from 410 K human bulk RNA-seq samples. Featuring a novel Binning-By-Gene normalization technique, our models capture diverse biological information beyond expression. We introduce GeneRAIN-vec, a multifaceted vectorized gene representation that outperforms those from existing models. We demonstrate knowledge transfer from protein-coding genes to Make 62.5 million biological attribute predictions for 13,030 long noncoding RNAs. This work advances Transformer and self-supervised deep learning applications to expression data, enhancing biological exploration.