Broad Learning System with Particle Swarm Optimization and Singular Value Decomposition

Abstract

Broad Learning System (BLS), a newly-developing alternative approach of learning for deep neural network, has attracted much attentions from researchers all over the world due to its straightforward network structure and powerful performance to deal with classification and regression problems. The number of feature nodes and enhancement nodes in classical BLS is determined by grid search method which leads to heavy training burden, while the weights between input data and feature nodes are randomly initialized and fine-tuned taking advantages of sparse autoencoder. Different from that, a new BLS with Particle Swarm optimization (PSO) and Singular Value Decomposition (SVD) is raised in this paper. PSO algorithm is introduced to acquire the optimal number of feature nodes and enhancement nodes, which greatly reduces the search time. In addition, the weights between input data and feature nodes are initialized by SVD method, which avoids using iteration method to optimize them and also reduces computational cost. The experimental results on several regression datasets demonstrate that BLS with PSO and SVD can not only find optimal number of system nodes much faster than classical BLS but also achieve considerable satisfactory accuracy.