Random Fourier Features Based Kernel Risk Sensitive Loss Algorithm with Adaptive Moment Estimation Optimization Technology

Abstract

Random Fourier features based kernel risk sensitive loss (RFFKRSL) is a popular nonlinear adaptive filtering algorithm developed in the random Fourier features space. The most attractive feature of such algorithm is that it would not cause the issue of linearly increasing network structure like the well-known kernel adaptive filtering algorithms, while having the ability to curb the negative influence induced by non-Gaussian noises. The stochastic gradient descent (SGD) method, however, is a default choice to determine the filtering coefficients of RFFKRSL, which can result in an undesirable convergence performance of the algorithm in many cases. To address this issue, two alternative optimization technologies, including adaptive moment estimation (Adam) and its extended version, i. e., Nesterov-accelerated Adam (Nadam), have been adopted to re-derive RFFKRSL. For simplicity, the proposed two algorithms are named as RFFKRSL with Adam (AdamRFFKRSL) and RFFKRSL with Nadam (NadamRFFKRSL), respectively. Although Adam and Nadam are common to deep neural network based learning methods, their applications to adaptive filtering are seldom to be seen, and the combination of them with RFFKRSL may open a new way to design nonlinear adaptive filtering algorithms that have been built with SGD method. Simulations on two time series prediction tasks are reported to demonstrate the desirable performance of the proposed algorithms.