Constructing re-substitution entropy estimator with discontinuous kernels

Abstract

Gaussian kernel is a continous kernel which is always used in the re-substitution entropy estimator (RSEE) to estimate the underlying entropy for the continuous random variable. Meanwhile, there are also some other discontinuous kernels that can be used to conduct the probability density function estimation and kernel regression analysis. The theoretical study indicates that some of these discontinuous kernels can obtain higher estimation efficiencies than Gaussian kernel. Thus, in this paper, we introduce six discontinuous kernels, i.e. Uniform, Triangular, Epanechnikov, Biweight, Triweight and Cosine, to establish the RESS. Firstly, we analyses mathematical properties of employed discontinuous kernels. Then, RESSs based discontinuous kernels are designed. Finally, extensive experiments are carried out to compare discontinuous kernels based RESSs with the Gaussian kernel based RESS in terms of estimation accuracy and stability. Experimental results show that discontinuous kernels can obtain better performances than Gaussian kernel.