On the estimation of entropy for unknown support size

Abstract

We consider the problem of estimating the entropy of a distribution P of unknown support size N provided with k independently drawn samples, where we are mainly focused on the low sampling regime where k <; N. By extending a method of Hausser and Strimmer [1], which assumes a known support size, by using the support size estimator of Chao and Lee [2], we obtain a simple estimator that performs equally good as the methods of Nemenman, Shafee, and Bialek [3] and Chao and Chen [4].