On Some Distributions and Their Pattern Entropies

Abstract

We study the block entropy of patterns of sequences generated by uniform and monotonic memoryless source distributions. In the former case, the pattern entropy decreases the most from the memoryless entropy, and in the latter the least. General upper and lower bounds are presented and then applied to these distributions. Tighter bounds are derived for a uniform case. All bounds provide almost precise characterization of the pattern entropies of uniform distributions, distributions over the integers, and the geometric distribution. Of specific interest are distributions over the integers that have infinite entropy rates in the memoryless case but bounded pattern block entropies