使用Web抓取技术和大型语言模型开发自动数据收集和挖掘方法：提取技术就绪程度评估的案例研究

IF 0.5 Q4 COMPUTER SCIENCE, INFORMATION SYSTEMS

AUTOMATIC DOCUMENTATION AND MATHEMATICAL LINGUISTICS Pub Date : 2025-10-06 DOI:10.3103/S0005105525700670

F. M. Grozovskiy, I. V. Loginova

{"title":"使用Web抓取技术和大型语言模型开发自动数据收集和挖掘方法：提取技术就绪程度评估的案例研究","authors":"F. M. Grozovskiy, I. V. Loginova","doi":"10.3103/S0005105525700670","DOIUrl":null,"url":null,"abstract":"<p>The paper proposes an approach to the automated extraction and structuring of information from text, combining web scraping for data collection from online sources with a large language model for subsequent data mining. As a case study, texts from news publications on technology readiness levels from the CNews website were chosen to test the developed methodology in a specific domain. The model’s accuracy in identifying technology readiness assessments was 84–85%, which is comparable to similar results in other, less specialized tasks.</p>","PeriodicalId":42995,"journal":{"name":"AUTOMATIC DOCUMENTATION AND MATHEMATICAL LINGUISTICS","volume":"59 4","pages":"269 - 278"},"PeriodicalIF":0.5000,"publicationDate":"2025-10-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Developing an Approach for Automated Data Collection and Mining Using Web Scraping Techniques and Large Language Models: A Case Study on Extracting Technology Readiness Level Assessments\",\"authors\":\"F. M. Grozovskiy, I. V. Loginova\",\"doi\":\"10.3103/S0005105525700670\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>The paper proposes an approach to the automated extraction and structuring of information from text, combining web scraping for data collection from online sources with a large language model for subsequent data mining. As a case study, texts from news publications on technology readiness levels from the CNews website were chosen to test the developed methodology in a specific domain. The model’s accuracy in identifying technology readiness assessments was 84–85%, which is comparable to similar results in other, less specialized tasks.</p>\",\"PeriodicalId\":42995,\"journal\":{\"name\":\"AUTOMATIC DOCUMENTATION AND MATHEMATICAL LINGUISTICS\",\"volume\":\"59 4\",\"pages\":\"269 - 278\"},\"PeriodicalIF\":0.5000,\"publicationDate\":\"2025-10-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"AUTOMATIC DOCUMENTATION AND MATHEMATICAL LINGUISTICS\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://link.springer.com/article/10.3103/S0005105525700670\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q4\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"AUTOMATIC DOCUMENTATION AND MATHEMATICAL LINGUISTICS","FirstCategoryId":"1085","ListUrlMain":"https://link.springer.com/article/10.3103/S0005105525700670","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

摘要

本文提出了一种从文本中自动提取和结构化信息的方法，将从在线资源中收集数据的web抓取与后续数据挖掘的大型语言模型相结合。作为案例研究，从CNews网站上关于技术准备水平的新闻出版物中选择文本，在特定领域测试开发的方法。该模型在识别技术准备评估方面的准确性为84-85%，这与其他不太专业的任务中的类似结果相当。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

Developing an Approach for Automated Data Collection and Mining Using Web Scraping Techniques and Large Language Models: A Case Study on Extracting Technology Readiness Level Assessments

查看原文本刊更多论文

Developing an Approach for Automated Data Collection and Mining Using Web Scraping Techniques and Large Language Models: A Case Study on Extracting Technology Readiness Level Assessments

The paper proposes an approach to the automated extraction and structuring of information from text, combining web scraping for data collection from online sources with a large language model for subsequent data mining. As a case study, texts from news publications on technology readiness levels from the CNews website were chosen to test the developed methodology in a specific domain. The model’s accuracy in identifying technology readiness assessments was 84–85%, which is comparable to similar results in other, less specialized tasks.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

AUTOMATIC DOCUMENTATION AND MATHEMATICAL LINGUISTICS COMPUTER SCIENCE, INFORMATION SYSTEMS-

自引率

40.00%

发文量

期刊介绍： Automatic Documentation and Mathematical Linguistics is an international peer reviewed journal that covers all aspects of automation of information processes and systems, as well as algorithms and methods for automatic language analysis. Emphasis is on the practical applications of new technologies and techniques for information analysis and processing.