创造必要的技术和专家分析条件——开发评估开放文本信息源对社会影响的信息系统

2019 15th International Asian School-Seminar Optimization Problems of Complex Systems (OPCS) Pub Date : 2019-08-01 DOI:10.1109/OPCS.2019.8880193

R. Mussabayev, Bek Kassymzhanov, Aidos Mukashev, Viktoriya Ibrayeva, Azat Merkebayev

{"title":"创造必要的技术和专家分析条件——开发评估开放文本信息源对社会影响的信息系统","authors":"R. Mussabayev, Bek Kassymzhanov, Aidos Mukashev, Viktoriya Ibrayeva, Azat Merkebayev","doi":"10.1109/OPCS.2019.8880193","DOIUrl":null,"url":null,"abstract":"In this paper, we trained distributional models (patterns) for text preprocessing in Word2vec and Glove. Three variants of text preprocessing were used to train distributional patterns. Based on the implemented distribution model Word2Vec, a vector representation was obtained for a cluster-separated test sample of 30 news items. All variants of the weighted average calculation of the vector representation of texts were considered. Two-stage clustering was carried out. After training the Doc2Vec model on normalized documents, a vector representation was obtained for each document. The following news about the same event was selected for the test, but from different sources. A 2-dimensional “factual cube” was analyzed.","PeriodicalId":288547,"journal":{"name":"2019 15th International Asian School-Seminar Optimization Problems of Complex Systems (OPCS)","volume":"19 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"Creation of Necessary Technical and Expert- Analytical Conditions for Development of the Information System of Evaluating Open Text Information Sources’ Influence on Society\",\"authors\":\"R. Mussabayev, Bek Kassymzhanov, Aidos Mukashev, Viktoriya Ibrayeva, Azat Merkebayev\",\"doi\":\"10.1109/OPCS.2019.8880193\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In this paper, we trained distributional models (patterns) for text preprocessing in Word2vec and Glove. Three variants of text preprocessing were used to train distributional patterns. Based on the implemented distribution model Word2Vec, a vector representation was obtained for a cluster-separated test sample of 30 news items. All variants of the weighted average calculation of the vector representation of texts were considered. Two-stage clustering was carried out. After training the Doc2Vec model on normalized documents, a vector representation was obtained for each document. The following news about the same event was selected for the test, but from different sources. A 2-dimensional “factual cube” was analyzed.\",\"PeriodicalId\":288547,\"journal\":{\"name\":\"2019 15th International Asian School-Seminar Optimization Problems of Complex Systems (OPCS)\",\"volume\":\"19 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2019-08-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2019 15th International Asian School-Seminar Optimization Problems of Complex Systems (OPCS)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/OPCS.2019.8880193\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2019 15th International Asian School-Seminar Optimization Problems of Complex Systems (OPCS)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/OPCS.2019.8880193","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 1

摘要

在本文中，我们在Word2vec和Glove中训练了用于文本预处理的分布式模型(模式)。本文采用了三种文本预处理方法来训练分布模式。基于实现的分布模型Word2Vec，对30条新闻的聚类分离测试样本进行向量表示。考虑了文本向量表示的加权平均计算的所有变体。采用两阶段聚类。在规范化文档上训练Doc2Vec模型后，得到每个文档的向量表示。以下关于同一事件的新闻被选择用于测试，但来自不同的来源。分析了一个二维“事实立方体”。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Creation of Necessary Technical and Expert- Analytical Conditions for Development of the Information System of Evaluating Open Text Information Sources’ Influence on Society

In this paper, we trained distributional models (patterns) for text preprocessing in Word2vec and Glove. Three variants of text preprocessing were used to train distributional patterns. Based on the implemented distribution model Word2Vec, a vector representation was obtained for a cluster-separated test sample of 30 news items. All variants of the weighted average calculation of the vector representation of texts were considered. Two-stage clustering was carried out. After training the Doc2Vec model on normalized documents, a vector representation was obtained for each document. The following news about the same event was selected for the test, but from different sources. A 2-dimensional “factual cube” was analyzed.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

2019 15th International Asian School-Seminar Optimization Problems of Complex Systems (OPCS)

自引率

0.00%

发文量