LLMs predict material properties based on atom types and not atom positions

Abstract

Accurately predicting properties of materials is one of the main challenges in computational materials science. Systematic experiments now show that large language models (LLMs) struggle to use positional information for the prediction of those properties. This limitation does not reverse as model or dataset sizes grow.