Towards optimized forward–forward training for neural networks

Abstract

This study investigates and enhances the Forward–Forward (FF) algorithm, a novel neural network training approach recently introduced by Geoffrey Hinton. Unlike Back-Propagation (BP), which relies on separate forward and backward passes, FF performs two forward passes, one with positive (real) data and one with negative data, potentially generated by the network itself. We evaluate FF across six diverse datasets to assess its general performance and employ two well-established and four state-of-the-art metaheuristic algorithms on four datasets to optimize its hyper-parameters. While FF offers a notable advantage in memory efficiency, making it suitable for low-resource hardware, it initially lags behind BP in predictive accuracy. Through automated hyper-parameter optimization, FF demonstrates substantial improvements, with validation accuracy increasing from 93% to 97% on Pneumonia-MedMnist, 87% to 91% on Fashion-MNIST, 82% to 94% on OrganC-MedMnist, and 88% to 95% on OrganA-MedMnist, while maintaining or reducing training time. These results demonstrate that metaheuristic optimization not only accelerates convergence but also significantly enhances FF’s predictive performance, bridging the gap between memory efficiency and accuracy.