Information theory and statistics

Abstract

The main theorems in information theory and statistics are put in context, the differences are discussed, and some of the open research problems are mentioned. The author demonstrates some of the points of intersection of information theory and statistics, and discuss some problems in physics and computer science that require a rigorous probabilistic treatment.