A. V. Kvasnov, Anatoliy A. Baranenko, Evgeniy Y. Butyrsky, Uliana P. Zaranik
{"title":"集中趋势对机器学习中最大熵密度分布性质的影响","authors":"A. V. Kvasnov, Anatoliy A. Baranenko, Evgeniy Y. Butyrsky, Uliana P. Zaranik","doi":"10.21638/11701/spbu10.2023.204","DOIUrl":null,"url":null,"abstract":"The principle of maximum entropy (ME) has a number of advantages that allow it to be used in machine learning. The density distribution of maximum entropy (WEO) requires solving the problem of calculus of variations on the a priori distribution, where the central tendency can be used as a parameter. In Lebesgue space, the central tendency is described by the generalized Gelder average. The paper shows the evolution of the density of the ME distribution depending on the given norm of the average. The minimum Kulback — Leibler divergence between the WEO and the a prior density is achieved at the harmonic mean, which is effective in reducing the dimensionality of the training sample. At the same time, this leads to a deterioration in the function of loss in the conditions of machine learning by precedents.","PeriodicalId":43738,"journal":{"name":"Vestnik Sankt-Peterburgskogo Universiteta Seriya 10 Prikladnaya Matematika Informatika Protsessy Upravleniya","volume":"16 1","pages":""},"PeriodicalIF":0.3000,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"On the influence of the cental trend on the nature of the density distribution of maximum entropy in machine learning\",\"authors\":\"A. V. Kvasnov, Anatoliy A. Baranenko, Evgeniy Y. Butyrsky, Uliana P. Zaranik\",\"doi\":\"10.21638/11701/spbu10.2023.204\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The principle of maximum entropy (ME) has a number of advantages that allow it to be used in machine learning. The density distribution of maximum entropy (WEO) requires solving the problem of calculus of variations on the a priori distribution, where the central tendency can be used as a parameter. In Lebesgue space, the central tendency is described by the generalized Gelder average. The paper shows the evolution of the density of the ME distribution depending on the given norm of the average. The minimum Kulback — Leibler divergence between the WEO and the a prior density is achieved at the harmonic mean, which is effective in reducing the dimensionality of the training sample. At the same time, this leads to a deterioration in the function of loss in the conditions of machine learning by precedents.\",\"PeriodicalId\":43738,\"journal\":{\"name\":\"Vestnik Sankt-Peterburgskogo Universiteta Seriya 10 Prikladnaya Matematika Informatika Protsessy Upravleniya\",\"volume\":\"16 1\",\"pages\":\"\"},\"PeriodicalIF\":0.3000,\"publicationDate\":\"2023-01-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Vestnik Sankt-Peterburgskogo Universiteta Seriya 10 Prikladnaya Matematika Informatika Protsessy Upravleniya\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.21638/11701/spbu10.2023.204\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q4\",\"JCRName\":\"MATHEMATICS, INTERDISCIPLINARY APPLICATIONS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Vestnik Sankt-Peterburgskogo Universiteta Seriya 10 Prikladnaya Matematika Informatika Protsessy Upravleniya","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.21638/11701/spbu10.2023.204","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"MATHEMATICS, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
On the influence of the cental trend on the nature of the density distribution of maximum entropy in machine learning
The principle of maximum entropy (ME) has a number of advantages that allow it to be used in machine learning. The density distribution of maximum entropy (WEO) requires solving the problem of calculus of variations on the a priori distribution, where the central tendency can be used as a parameter. In Lebesgue space, the central tendency is described by the generalized Gelder average. The paper shows the evolution of the density of the ME distribution depending on the given norm of the average. The minimum Kulback — Leibler divergence between the WEO and the a prior density is achieved at the harmonic mean, which is effective in reducing the dimensionality of the training sample. At the same time, this leads to a deterioration in the function of loss in the conditions of machine learning by precedents.
期刊介绍:
The journal is the prime outlet for the findings of scientists from the Faculty of applied mathematics and control processes of St. Petersburg State University. It publishes original contributions in all areas of applied mathematics, computer science and control. Vestnik St. Petersburg University: Applied Mathematics. Computer Science. Control Processes features articles that cover the major areas of applied mathematics, computer science and control.