Anisotropic magnetic interactions in a candidate Kitaev spin liquid close to a metal-insulator transition

In the Kitaev honeycomb model, spins coupled by strongly-frustrated anisotropic interactions do not order at low temperature but instead form a quantum spin liquid with spin fractionalisation into Majorana fermions and static fluxes. The realization of such a model in crystalline materials could lead to major breakthroughs in understanding entangled quantum states, however achieving this in practice is a very challenging task. The recently synthesized honeycomb material RuI3 shows no long-range magnetic order down to the lowest probed temperatures and has been theoretically proposed as a quantum spin liquid candidate material on the verge of an insulator to metal transition. Here we report a comprehensive study of the magnetic anisotropy in un-twinned single crystals via torque magnetometry and detect clear signatures of strongly anisotropic and frustrated magnetic interactions. We attribute the development of sawtooth and six-fold torque signal to strongly anisotropic, bond-dependent magnetic interactions by comparing to theoretical calculations. As a function of magnetic field strength at low temperatures, torque shows an unusual non-parabolic dependence suggestive of a proximity to a field-induced transition. Thus, RuI3, without signatures of long-range magnetic order, displays key hallmarks of an exciting candidate for extended Kitaev magnetism with enhanced quantum fluctuations. Quantum spin liquids are materials predicted to be absent of magnetic ordering at low temperature, giving rise to fractionalised electronic states, but conclusive experimental evidence is still absent. Here, the authors conduct angular dependent torque measurements on the candidate spin liquid material RuI3 and, through a comparison of experimental and theoretical results, provide evidence indicating the presence of frustrated magnetic interactions in the system.