Efficient index lookup for De-duplication backup system

Y. Won, Jongmyeong Ban, Jaehong Min, Jungpil Hur, Sangkyu Oh, Jangsun Lee
{"title":"Efficient index lookup for De-duplication backup system","authors":"Y. Won, Jongmyeong Ban, Jaehong Min, Jungpil Hur, Sangkyu Oh, Jangsun Lee","doi":"10.1109/MASCOT.2008.4770594","DOIUrl":null,"url":null,"abstract":"We minimizes fingerprint management overhead (index lookup and index insert) via introducing main memory index lookup structure and workload-aware index partitioning of the index file in the storage. Backup server maintains three data structures for redundancy elimination: Header files, chunk files and fingerprint tables. These data structures altogether enables PRUNE to effectively eliminate redundancy and to perform efficient backup. We perform various experiment to measure the overhead of each tasks in backup operation and to examine the efficiency of redundancy elimination. Incremental modulo-K reduces the file chunking latency by approximately 60%. With filter based in-memory index data structure and index partitioning, PRUNE eliminates 99.4% of disk accesses involved in fingerprint management.","PeriodicalId":323154,"journal":{"name":"2008 IEEE International Symposium on Modeling, Analysis and Simulation of Computers and Telecommunication Systems","volume":"688 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2008-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"21","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2008 IEEE International Symposium on Modeling, Analysis and Simulation of Computers and Telecommunication Systems","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/MASCOT.2008.4770594","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 21

Abstract

We minimizes fingerprint management overhead (index lookup and index insert) via introducing main memory index lookup structure and workload-aware index partitioning of the index file in the storage. Backup server maintains three data structures for redundancy elimination: Header files, chunk files and fingerprint tables. These data structures altogether enables PRUNE to effectively eliminate redundancy and to perform efficient backup. We perform various experiment to measure the overhead of each tasks in backup operation and to examine the efficiency of redundancy elimination. Incremental modulo-K reduces the file chunking latency by approximately 60%. With filter based in-memory index data structure and index partitioning, PRUNE eliminates 99.4% of disk accesses involved in fingerprint management.
重复数据删除备份系统的高效索引查找
我们通过引入主内存索引查找结构和存储中索引文件的工作负载感知索引分区来最小化指纹管理开销(索引查找和索引插入)。备份服务器维护三种数据结构以消除冗余:头文件、块文件和指纹表。这些数据结构使PRUNE能够有效地消除冗余并执行有效的备份。我们进行了各种实验来测量备份操作中每个任务的开销,并检验冗余消除的效率。增量模k将文件分块延迟减少了大约60%。使用基于过滤器的内存索引数据结构和索引分区,PRUNE消除了涉及指纹管理的99.4%的磁盘访问。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书