{"title":"分布式存储系统中Erasure Code数据修复的节点选择方案","authors":"Yuejin Huang, Miao Ye, Yue Cai","doi":"10.1145/3546000.3546003","DOIUrl":null,"url":null,"abstract":"The explosion of massive data poses a severe challenge to the storage cost and data storage reliability of traditional storage systems. As the number of storage nodes increases in a distributed storage system, the node failure probability increases. Erasure code technology as a redundancy strategy can greatly save the cost of storage space while providing security for data compared with multi-copy technology. In real large-scale data centers, the repair costs vary due to differences in storage node attributes and link status changes between nodes. In the repair process based on the erasure code mechanism, it is required to select a certain number of provider nodes from the surviving nodes to transmit data to the newcomer nodes, which leads to the problem of selecting nodes to participate in the repair process. In this paper, a tree-type repair scheme considering node selection (TR-NS) is designed, including an algorithm for selecting two types of nodes according to nodes performance and an algorithm for establishing the optimal repair tree to complete the reconstruction of the failure data. The experimental results show that the repair time of the TR-RS scheme proposed in this paper is significantly reduced.","PeriodicalId":196955,"journal":{"name":"Proceedings of the 6th International Conference on High Performance Compilation, Computing and Communications","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2022-06-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"A Node Selection Scheme for Data Repair Using Erasure Code in Distributed Storage System\",\"authors\":\"Yuejin Huang, Miao Ye, Yue Cai\",\"doi\":\"10.1145/3546000.3546003\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The explosion of massive data poses a severe challenge to the storage cost and data storage reliability of traditional storage systems. As the number of storage nodes increases in a distributed storage system, the node failure probability increases. Erasure code technology as a redundancy strategy can greatly save the cost of storage space while providing security for data compared with multi-copy technology. In real large-scale data centers, the repair costs vary due to differences in storage node attributes and link status changes between nodes. In the repair process based on the erasure code mechanism, it is required to select a certain number of provider nodes from the surviving nodes to transmit data to the newcomer nodes, which leads to the problem of selecting nodes to participate in the repair process. In this paper, a tree-type repair scheme considering node selection (TR-NS) is designed, including an algorithm for selecting two types of nodes according to nodes performance and an algorithm for establishing the optimal repair tree to complete the reconstruction of the failure data. The experimental results show that the repair time of the TR-RS scheme proposed in this paper is significantly reduced.\",\"PeriodicalId\":196955,\"journal\":{\"name\":\"Proceedings of the 6th International Conference on High Performance Compilation, Computing and Communications\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2022-06-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Proceedings of the 6th International Conference on High Performance Compilation, Computing and Communications\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1145/3546000.3546003\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the 6th International Conference on High Performance Compilation, Computing and Communications","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3546000.3546003","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
A Node Selection Scheme for Data Repair Using Erasure Code in Distributed Storage System
The explosion of massive data poses a severe challenge to the storage cost and data storage reliability of traditional storage systems. As the number of storage nodes increases in a distributed storage system, the node failure probability increases. Erasure code technology as a redundancy strategy can greatly save the cost of storage space while providing security for data compared with multi-copy technology. In real large-scale data centers, the repair costs vary due to differences in storage node attributes and link status changes between nodes. In the repair process based on the erasure code mechanism, it is required to select a certain number of provider nodes from the surviving nodes to transmit data to the newcomer nodes, which leads to the problem of selecting nodes to participate in the repair process. In this paper, a tree-type repair scheme considering node selection (TR-NS) is designed, including an algorithm for selecting two types of nodes according to nodes performance and an algorithm for establishing the optimal repair tree to complete the reconstruction of the failure data. The experimental results show that the repair time of the TR-RS scheme proposed in this paper is significantly reduced.