用于文档大数据定性聚类的 MapReduce 增强型模糊 K-Least 中值法

IF 1.5 4区 计算机科学 Q3 COMPUTER SCIENCE, SOFTWARE ENGINEERING
Tanvir Habib Sardar, Zahid Ahmed Ansari, Prasannavenkatesan Theerthagiri, P. Karthikeyan, Vadivel Ayyasamy, Dilip Kumar Jang Bahadur Saini
{"title":"用于文档大数据定性聚类的 MapReduce 增强型模糊 K-Least 中值法","authors":"Tanvir Habib Sardar,&nbsp;Zahid Ahmed Ansari,&nbsp;Prasannavenkatesan Theerthagiri,&nbsp;P. Karthikeyan,&nbsp;Vadivel Ayyasamy,&nbsp;Dilip Kumar Jang Bahadur Saini","doi":"10.1002/cpe.70035","DOIUrl":null,"url":null,"abstract":"<div>\n \n <p>Researchers design MapReduce-enhanced versions of traditional clustering algorithms to obtain the time performance benefit in big data clustering. However, the current literature shows that only a handful of partitioning clustering algorithms are enhanced using the MapReduce model. This work proposes a MapReduce-enhanced design of the Fuzzy K-Least Medians algorithm (MRFK-LstMdns) and its two novel variations. The purpose is to determine the best-performing MapReduce-enhanced partitioning clustering algorithms among the proposed and existing ones in terms of time performance and cluster quality. The work first preprocesses different self-crawled document datasets. Then, an optimal noise removal process is employed to make the dataset optimally noise-free. The proposed MRFK-LstMdns and its two novel variations are designed using three MapReduce job chaining. Each of the jobs performs the staged and suitable algorithmic parts. The proposed algorithms' time performance and cluster quality are compared against the existing MapReduce-enhanced partitioning algorithms. Although the proposed algorithm's time complexity is higher than that of existing algorithms as KLeast median is well known to execute in more time than other algorithms such as KMeans, KMedoids and its fuzzy versions, its efficient design, incorporating a chained MapReduce job execution, ensures that the actual execution time remains comparable to that of the existing algorithms. A majority voting technique using seven cluster quality measures shows that the MRFK-LstMdns generates better quality clusters than existing algorithms.</p>\n </div>","PeriodicalId":55214,"journal":{"name":"Concurrency and Computation-Practice & Experience","volume":"37 4-5","pages":""},"PeriodicalIF":1.5000,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"MapReduce-Enhanced Fuzzy K-Least Medians for Qualitative Clustering of Document Big Data\",\"authors\":\"Tanvir Habib Sardar,&nbsp;Zahid Ahmed Ansari,&nbsp;Prasannavenkatesan Theerthagiri,&nbsp;P. Karthikeyan,&nbsp;Vadivel Ayyasamy,&nbsp;Dilip Kumar Jang Bahadur Saini\",\"doi\":\"10.1002/cpe.70035\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div>\\n \\n <p>Researchers design MapReduce-enhanced versions of traditional clustering algorithms to obtain the time performance benefit in big data clustering. However, the current literature shows that only a handful of partitioning clustering algorithms are enhanced using the MapReduce model. This work proposes a MapReduce-enhanced design of the Fuzzy K-Least Medians algorithm (MRFK-LstMdns) and its two novel variations. The purpose is to determine the best-performing MapReduce-enhanced partitioning clustering algorithms among the proposed and existing ones in terms of time performance and cluster quality. The work first preprocesses different self-crawled document datasets. Then, an optimal noise removal process is employed to make the dataset optimally noise-free. The proposed MRFK-LstMdns and its two novel variations are designed using three MapReduce job chaining. Each of the jobs performs the staged and suitable algorithmic parts. The proposed algorithms' time performance and cluster quality are compared against the existing MapReduce-enhanced partitioning algorithms. Although the proposed algorithm's time complexity is higher than that of existing algorithms as KLeast median is well known to execute in more time than other algorithms such as KMeans, KMedoids and its fuzzy versions, its efficient design, incorporating a chained MapReduce job execution, ensures that the actual execution time remains comparable to that of the existing algorithms. A majority voting technique using seven cluster quality measures shows that the MRFK-LstMdns generates better quality clusters than existing algorithms.</p>\\n </div>\",\"PeriodicalId\":55214,\"journal\":{\"name\":\"Concurrency and Computation-Practice & Experience\",\"volume\":\"37 4-5\",\"pages\":\"\"},\"PeriodicalIF\":1.5000,\"publicationDate\":\"2025-02-25\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Concurrency and Computation-Practice & Experience\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/cpe.70035\",\"RegionNum\":4,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"COMPUTER SCIENCE, SOFTWARE ENGINEERING\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Concurrency and Computation-Practice & Experience","FirstCategoryId":"94","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/cpe.70035","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}
引用次数: 0

摘要

研究人员设计了传统聚类算法的mapreduce增强版本,以获得大数据聚类的时间性能优势。然而,目前的文献表明,只有少数分区聚类算法使用MapReduce模型得到了增强。本工作提出了模糊k -最小中值算法(MRFK-LstMdns)及其两个新变体的mapreduce增强设计。目的是在提出的和现有的算法中确定在时间性能和集群质量方面性能最好的mapreduce增强分区聚类算法。这项工作首先预处理不同的自抓取文档数据集。然后,采用最优去噪过程使数据集达到最优无噪。提出的MRFK-LstMdns及其两个新变体使用三个MapReduce作业链进行设计。每个作业执行阶段性的和合适的算法部分。将本文算法的时间性能和聚类质量与现有的mapreduce增强分区算法进行了比较。虽然该算法的时间复杂度高于现有算法,因为众所周知,KLeast median比其他算法(如KMeans、KMedoids及其模糊版本)执行时间更长,但其高效的设计,结合链式MapReduce作业执行,确保了实际执行时间与现有算法相当。使用七种聚类质量度量的多数投票技术表明,MRFK-LstMdns生成的聚类质量比现有算法更好。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
MapReduce-Enhanced Fuzzy K-Least Medians for Qualitative Clustering of Document Big Data

Researchers design MapReduce-enhanced versions of traditional clustering algorithms to obtain the time performance benefit in big data clustering. However, the current literature shows that only a handful of partitioning clustering algorithms are enhanced using the MapReduce model. This work proposes a MapReduce-enhanced design of the Fuzzy K-Least Medians algorithm (MRFK-LstMdns) and its two novel variations. The purpose is to determine the best-performing MapReduce-enhanced partitioning clustering algorithms among the proposed and existing ones in terms of time performance and cluster quality. The work first preprocesses different self-crawled document datasets. Then, an optimal noise removal process is employed to make the dataset optimally noise-free. The proposed MRFK-LstMdns and its two novel variations are designed using three MapReduce job chaining. Each of the jobs performs the staged and suitable algorithmic parts. The proposed algorithms' time performance and cluster quality are compared against the existing MapReduce-enhanced partitioning algorithms. Although the proposed algorithm's time complexity is higher than that of existing algorithms as KLeast median is well known to execute in more time than other algorithms such as KMeans, KMedoids and its fuzzy versions, its efficient design, incorporating a chained MapReduce job execution, ensures that the actual execution time remains comparable to that of the existing algorithms. A majority voting technique using seven cluster quality measures shows that the MRFK-LstMdns generates better quality clusters than existing algorithms.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Concurrency and Computation-Practice & Experience
Concurrency and Computation-Practice & Experience 工程技术-计算机:理论方法
CiteScore
5.00
自引率
10.00%
发文量
664
审稿时长
9.6 months
期刊介绍: Concurrency and Computation: Practice and Experience (CCPE) publishes high-quality, original research papers, and authoritative research review papers, in the overlapping fields of: Parallel and distributed computing; High-performance computing; Computational and data science; Artificial intelligence and machine learning; Big data applications, algorithms, and systems; Network science; Ontologies and semantics; Security and privacy; Cloud/edge/fog computing; Green computing; and Quantum computing.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信