Kullback-Leibler Distance and Statatistical Model

: Kullback-Leibler distance is discovered in statistical mechanics of the 19th century and is a concept called the relative entropy. In the 20th century, it came to be known that it is important quantity in statistics and learning theory. We cannot only define it in Euclidean space, but also can expand the concept in general probability distribution. Kullback-Leibler distance is not only important as the concept, but also derive an algorithm. Kullback-Leibler distance is not only an important concept, but also it can derive the algorithm. As for the information science meaning of Kullback-Leibler distance and the relations with the mathematic properties, there exist many unknown structures. We constituted some examples about the pole using a computer algebra system and explored the properties.