Hypercomplex neural networks: Exploring quaternion, octonion, and beyond in deep learning

Abstract

Hypercomplex Neural Networks (HNNs) represent the next frontier in deep learning, building on the mathematical theory of quaternions, octonions, and higher-dimensional algebras to generalize conventional neural architectures. This review synthesizes cutting-edge methods with their theoretical bases, architectural advancements, and primary applications, tracing the development of hypercomplex mathematics and its implementation in computational models. We distil key advances in quaternion and octonion networks, highlighting their ability to provide compact representations and computational efficiency. Particular attention is given to the unique challenge of non-associativity in octonions—where the order in which numbers are multiplied affects the result—requiring careful design of network operations. The article also discusses training complexity, interpretability, and the lack of standardized frameworks, alongside comparative performance with real- and complex-valued networks. Future directions include scalable algorithm construction, lightweight architectures through tensor decompositions, and integration with quantum-inspired systems using higher-order algebras. By presenting a systematic synthesis of current literature and linking these advances to practical applications, this review aims to equip researchers and practitioners with a clear understanding of the strengths, limitations, and potential of HNNs for advancing multidimensional data modelling.