CrysToGraph: A Comprehensive Predictive Model for Crystal Material Properties and the Benchmark

The bonding across the lattice and ordered structures endow crystals with unique symmetry and determine their macroscopic properties. Crystals with unique properties such as low-dimensional materials, metal-organic frameworks, and defected crystals, in particular, exhibit different structures from bulk crystals and possess exotic physical properties, making them intriguing subjects for investigation. To accurately predict the physical and chemical properties of crystals, it is crucial to consider long-range orders. While GNNs excel at capturing the local environment of atoms in crystals, they often face challenges in effectively capturing longe range interactions due to their limited depth. In this paper, we propose CrysToGraph (Crystals with Transformers on Graph), a transformer-based geometric graph network designed for unconventional crystalline systems, and UnconvBench, a benchmark to evaluate models' predictive performance on multiple categories of crystal materials. CrysToGraph effectively captures short-range interactions with transformer-based graph convolution blocks as well as long-range interactions with graph-wise transformer blocks. CrysToGraph proves its effectiveness in modelling all types of crystal materials in multiple tasks, and moreover, it outperforms most existing methods, achieving new state-of-the-art results on two benchmarks. This work has the potential to accelerate the development of novel crystal materials in various fields, including the anodes, cathodes, and solid-state electrolytes.