On the performance of histogram-based entropy estimators

Abstract

Histograms are widely used for estimating the density of a continuous signal from existing data. In some practical applications, they are also employed for entropy estimation. However, a histogram involves implicitly a discretization procedure because the unknown density is approximated by a piecewise constant density model. In the previous literature, the impact of the discretization procedure on the accuracy of the entropy estimate was either ignored or evaluated in the particular case of a regular histogram, in which all bins are equally wide. In this work, we provide bounds on the performance of the histogram-based entropy estimators without relying on the restrictive assumptions which have been used by other authors. The proof of our theoretical results is mainly based on concentration inequalities which have been already employed to analyze the performance of histograms as density estimators. After establishing the theoretical results, we illustrate them by numerical examples.