Robust Reconstructed Neural Network With Spectral Reshaping Activation

Abstract

Neural network (NN) is a prominent intelligent model to process information through the connection and activation of multilayer neurons. However, NNs usually encounter with the incorrect activation of neurons because of the excessive coverage for the boundary of compound noises. To address this issue, this article proposes a robust reconstructed NN (RRNN) with spectral reshaping activation (SRA). Primarily, an SRA is designed to replace the original activation of NN, which shrinks the spectrums of the compound noises toward the cluster center through spectral subtraction. It enables RRNN to reshape a concentrated noise space for easy coverage. Then, a hierarchical gradient descent (HGD) algorithm is developed to update the parameters of RRNN. The HGD algorithm establishes a noise-contrastive degree of SRA to penalize the loss function of RRNN, which holds robust performance with different noises. Furthermore, the theoretical proof of RRNN is presented to validate its robustness. Finally, the experimental results confirm the superior robustness of RRNN for tackling noisy samples compared to other methods.