文本同质性的统计测试:使用不同单词数量的前向和后向处理

IF 0.2 Q4 LINGUISTICS

Glottometrics Pub Date : 2022-01-01 DOI:10.53482/2022_53_401

Berhane Abebe, M. Chebunin, A. Kovalevskii, N. Zakrevskaya

{"title":"文本同质性的统计测试:使用不同单词数量的前向和后向处理","authors":"Berhane Abebe, M. Chebunin, A. Kovalevskii, N. Zakrevskaya","doi":"10.53482/2022_53_401","DOIUrl":null,"url":null,"abstract":"The processes of growth in the number of diverse words in a text, when reading in the forward and backward directions, are studied in this article. Based upon the statistics achieved from the difference between these two processes, we construct a statistical test. This statistical test is used for text homogeneity checks. The elementary model states that words in a text are selected from some dictionary independent of each other according to the Zipf–Mandelbrot law. P-values of the statistical test are calculated based on the elementary probabilistic model using the asymptotic normality of corresponding statistics. At last but not least, this statistical test is applied for the analysis of homogeneity of sequences of sonnets.","PeriodicalId":51918,"journal":{"name":"Glottometrics","volume":"37 1","pages":"42-58"},"PeriodicalIF":0.2000,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"Statistical tests for text homogeneity: using forward and backward processes of numbers of different words\",\"authors\":\"Berhane Abebe, M. Chebunin, A. Kovalevskii, N. Zakrevskaya\",\"doi\":\"10.53482/2022_53_401\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The processes of growth in the number of diverse words in a text, when reading in the forward and backward directions, are studied in this article. Based upon the statistics achieved from the difference between these two processes, we construct a statistical test. This statistical test is used for text homogeneity checks. The elementary model states that words in a text are selected from some dictionary independent of each other according to the Zipf–Mandelbrot law. P-values of the statistical test are calculated based on the elementary probabilistic model using the asymptotic normality of corresponding statistics. At last but not least, this statistical test is applied for the analysis of homogeneity of sequences of sonnets.\",\"PeriodicalId\":51918,\"journal\":{\"name\":\"Glottometrics\",\"volume\":\"37 1\",\"pages\":\"42-58\"},\"PeriodicalIF\":0.2000,\"publicationDate\":\"2022-01-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Glottometrics\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.53482/2022_53_401\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q4\",\"JCRName\":\"LINGUISTICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Glottometrics","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.53482/2022_53_401","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"LINGUISTICS","Score":null,"Total":0}

引用次数: 1

摘要

本文研究了在向前阅读和向后阅读时，文本中不同词汇数量的增长过程。根据这两个过程之间的差异所获得的统计量，我们构造了一个统计检验。这个统计检验用于文本同质性检查。基本模型表明，文本中的单词是根据Zipf-Mandelbrot定律从相互独立的字典中选择出来的。利用相应统计量的渐近正态性，基于初等概率模型计算统计检验的p值。最后，将此统计检验应用于十四行诗序列的同质性分析。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Statistical tests for text homogeneity: using forward and backward processes of numbers of different words

The processes of growth in the number of diverse words in a text, when reading in the forward and backward directions, are studied in this article. Based upon the statistics achieved from the difference between these two processes, we construct a statistical test. This statistical test is used for text homogeneity checks. The elementary model states that words in a text are selected from some dictionary independent of each other according to the Zipf–Mandelbrot law. P-values of the statistical test are calculated based on the elementary probabilistic model using the asymptotic normality of corresponding statistics. At last but not least, this statistical test is applied for the analysis of homogeneity of sequences of sonnets.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Glottometrics LINGUISTICS-

CiteScore

0.50

自引率

0.00%

发文量

期刊介绍： The aim of Glottometrics is quantification, measurement and mathematical modeling of any kind of language phenomena. We invite contributions on probabilistic or other mathematical models (e.g. graph theoretic or optimization approaches) which enable to establish language laws that can be validated by testing statistical hypotheses.