分布式存储系统中Erasure Code数据修复的节点选择方案

Yuejin Huang, Miao Ye, Yue Cai
{"title":"分布式存储系统中Erasure Code数据修复的节点选择方案","authors":"Yuejin Huang, Miao Ye, Yue Cai","doi":"10.1145/3546000.3546003","DOIUrl":null,"url":null,"abstract":"The explosion of massive data poses a severe challenge to the storage cost and data storage reliability of traditional storage systems. As the number of storage nodes increases in a distributed storage system, the node failure probability increases. Erasure code technology as a redundancy strategy can greatly save the cost of storage space while providing security for data compared with multi-copy technology. In real large-scale data centers, the repair costs vary due to differences in storage node attributes and link status changes between nodes. In the repair process based on the erasure code mechanism, it is required to select a certain number of provider nodes from the surviving nodes to transmit data to the newcomer nodes, which leads to the problem of selecting nodes to participate in the repair process. In this paper, a tree-type repair scheme considering node selection (TR-NS) is designed, including an algorithm for selecting two types of nodes according to nodes performance and an algorithm for establishing the optimal repair tree to complete the reconstruction of the failure data. The experimental results show that the repair time of the TR-RS scheme proposed in this paper is significantly reduced.","PeriodicalId":196955,"journal":{"name":"Proceedings of the 6th International Conference on High Performance Compilation, Computing and Communications","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2022-06-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"A Node Selection Scheme for Data Repair Using Erasure Code in Distributed Storage System\",\"authors\":\"Yuejin Huang, Miao Ye, Yue Cai\",\"doi\":\"10.1145/3546000.3546003\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The explosion of massive data poses a severe challenge to the storage cost and data storage reliability of traditional storage systems. As the number of storage nodes increases in a distributed storage system, the node failure probability increases. Erasure code technology as a redundancy strategy can greatly save the cost of storage space while providing security for data compared with multi-copy technology. In real large-scale data centers, the repair costs vary due to differences in storage node attributes and link status changes between nodes. In the repair process based on the erasure code mechanism, it is required to select a certain number of provider nodes from the surviving nodes to transmit data to the newcomer nodes, which leads to the problem of selecting nodes to participate in the repair process. In this paper, a tree-type repair scheme considering node selection (TR-NS) is designed, including an algorithm for selecting two types of nodes according to nodes performance and an algorithm for establishing the optimal repair tree to complete the reconstruction of the failure data. The experimental results show that the repair time of the TR-RS scheme proposed in this paper is significantly reduced.\",\"PeriodicalId\":196955,\"journal\":{\"name\":\"Proceedings of the 6th International Conference on High Performance Compilation, Computing and Communications\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2022-06-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Proceedings of the 6th International Conference on High Performance Compilation, Computing and Communications\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1145/3546000.3546003\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the 6th International Conference on High Performance Compilation, Computing and Communications","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3546000.3546003","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 1

摘要

海量数据的爆炸式增长对传统存储系统的存储成本和数据存储可靠性提出了严峻的挑战。在分布式存储系统中,随着存储节点数量的增加,节点故障的概率也随之增加。与多副本技术相比,Erasure code技术作为一种冗余策略,在保证数据安全性的同时,大大节省了存储空间成本。在实际的大规模数据中心中,由于存储节点属性的不同以及节点间链路状态的变化,维修成本会有所不同。在基于erasure code机制的修复过程中,需要从幸存节点中选择一定数量的provider节点将数据传递给新加入的节点,这就导致了选择节点参与修复过程的问题。本文设计了一种考虑节点选择的树型修复方案(TR-NS),包括根据节点性能选择两种类型节点的算法和建立最优修复树的算法,以完成故障数据的重构。实验结果表明,本文提出的TR-RS方案的修复时间明显缩短。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
A Node Selection Scheme for Data Repair Using Erasure Code in Distributed Storage System
The explosion of massive data poses a severe challenge to the storage cost and data storage reliability of traditional storage systems. As the number of storage nodes increases in a distributed storage system, the node failure probability increases. Erasure code technology as a redundancy strategy can greatly save the cost of storage space while providing security for data compared with multi-copy technology. In real large-scale data centers, the repair costs vary due to differences in storage node attributes and link status changes between nodes. In the repair process based on the erasure code mechanism, it is required to select a certain number of provider nodes from the surviving nodes to transmit data to the newcomer nodes, which leads to the problem of selecting nodes to participate in the repair process. In this paper, a tree-type repair scheme considering node selection (TR-NS) is designed, including an algorithm for selecting two types of nodes according to nodes performance and an algorithm for establishing the optimal repair tree to complete the reconstruction of the failure data. The experimental results show that the repair time of the TR-RS scheme proposed in this paper is significantly reduced.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信