Language models for protein design

Abstract

The recent surge of large language models has shown that machines are capable of reading, understanding, and communicating through language, even sometimes displaying capabilities surpassing those of humans. Proteins can be represented as strings of amino acids akin to words in a sentence, and the same principles of language modeling can be used to learn informative representations for protein structure prediction, design, and property prediction. In this review, we will focus on applications of language modeling to protein design. We will first cover the foundations of protein language modeling and discuss recent advances such as context-conditioned design and structure integration. We also consider current shortcomings and promising avenues of research for protein language modeling to facilitate future development of improved protein language models for design.