A Scalable Unsupervised and Back Propagation Free Learning With SACSOM: A Novel Approach to SOM-Based Architectures

Abstract

The field of computer vision is predominantly driven by supervised models, which, despite their efficacy, are computationally expensive and often intractable for many applications. Recently, research has expedited alternative avenues such as self-organizing maps (SOM)-based architectures, which offer significant advantages such as tractability, the absence of back-propagation, and feed-forward unsupervised learning. However, these SOM-based approaches frequently suffer from lower accuracy and limited generalization capabilities. To address these shortcomings, we propose a novel model called split and concur SOM (SACSOM). SACSOM overcomes the limitations of closely related SOM-based algorithms by utilizing multiple parallel branches, each equipped with its own SOM modules that process data independently with varying patch sizes. Furthermore, by creating groups of classes and using respective training samples to train independent subbranches in each branch, our approach accommodates datasets with a large number of classes. SACSOM employs a simple yet effective labeling technique requiring minimal labeled samples. The outputs from each branch, filtered by a threshold, contribute to the final prediction. Experimental validation on MNIST-digit, Fashion-MNIST, CIFAR-10, and CIFAR-100 demonstrates that SACSOM achieves competitive accuracy with significantly reduced computation time. Furthermore, it exhibits superior performance and generalization capabilities, even in high-noise scenarios. The weights of the single-layered SACSOM provide meaningful insights into the patch-based learning pattern, enhancing its tractability and making it ideal from the perspective of explainable AI. This study addresses the limitations of current clustering techniques, such as K-means and traditional SOMs, by proposing a lightweight, manageable, and fast architecture that does not require a GPU, making it suitable for low-powered devices.