Y. Won, Jongmyeong Ban, Jaehong Min, Jungpil Hur, Sangkyu Oh, Jangsun Lee
{"title":"Efficient index lookup for De-duplication backup system","authors":"Y. Won, Jongmyeong Ban, Jaehong Min, Jungpil Hur, Sangkyu Oh, Jangsun Lee","doi":"10.1109/MASCOT.2008.4770594","DOIUrl":null,"url":null,"abstract":"We minimizes fingerprint management overhead (index lookup and index insert) via introducing main memory index lookup structure and workload-aware index partitioning of the index file in the storage. Backup server maintains three data structures for redundancy elimination: Header files, chunk files and fingerprint tables. These data structures altogether enables PRUNE to effectively eliminate redundancy and to perform efficient backup. We perform various experiment to measure the overhead of each tasks in backup operation and to examine the efficiency of redundancy elimination. Incremental modulo-K reduces the file chunking latency by approximately 60%. With filter based in-memory index data structure and index partitioning, PRUNE eliminates 99.4% of disk accesses involved in fingerprint management.","PeriodicalId":323154,"journal":{"name":"2008 IEEE International Symposium on Modeling, Analysis and Simulation of Computers and Telecommunication Systems","volume":"688 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2008-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"21","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2008 IEEE International Symposium on Modeling, Analysis and Simulation of Computers and Telecommunication Systems","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/MASCOT.2008.4770594","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 21
Abstract
We minimizes fingerprint management overhead (index lookup and index insert) via introducing main memory index lookup structure and workload-aware index partitioning of the index file in the storage. Backup server maintains three data structures for redundancy elimination: Header files, chunk files and fingerprint tables. These data structures altogether enables PRUNE to effectively eliminate redundancy and to perform efficient backup. We perform various experiment to measure the overhead of each tasks in backup operation and to examine the efficiency of redundancy elimination. Incremental modulo-K reduces the file chunking latency by approximately 60%. With filter based in-memory index data structure and index partitioning, PRUNE eliminates 99.4% of disk accesses involved in fingerprint management.