Qin Xin, E. L. Miller, T. Schwarz, D. Long, S. Brandt, W. Litwin
{"title":"Reliability mechanisms for very large storage systems","authors":"Qin Xin, E. L. Miller, T. Schwarz, D. Long, S. Brandt, W. Litwin","doi":"10.1109/MASS.2003.1194851","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194851","url":null,"abstract":"Reliability and availability are increasingly important in large-scale storage systems built from thousands of individual storage devices. Large systems must survive the failure of individual components; in systems with thousands of disks, even infrequent failures are likely in some device. We focus on two types of errors: nonrecoverable read errors and drive failures. We discuss mechanisms for detecting and recovering from such errors, introducing improved techniques for detecting errors in disk reads and fast recovery from disk failure. We show that simple RAID cannot guarantee sufficient reliability; our analysis examines the tradeoffs among other schemes between system availability and storage efficiency. Based on our data, we believe that two-way mirroring should be sufficient for most large storage systems. For those that need very high reliability, we recommend either three-way mirroring or mirroring combined with RAID.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"192 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116142763","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Design and implementation of a storage repository using commonality factoring","authors":"Jim Hamilton, E. W. Olsen","doi":"10.1109/MASS.2003.1194854","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194854","url":null,"abstract":"In this paper, we discuss the design of a data normalization system that we term commonality factoring. A real-world implementation of a storage system based upon data normalization requires design of the data normalization itself, of the storage repository for the data, and of the protocols to be used between applications performing data normalization and the server software of the repository. Each of these areas is discussed and potential applications are presented. Building on research begun in 1999, Avamar Technologies has implemented an initial application of this technology to provide a nearline, disk-based system for backup of primary storage.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"82 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132047237","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Effective management of hierarchical storage using two levels of data clustering","authors":"R. Orlandic","doi":"10.1109/MASS.2003.1194863","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194863","url":null,"abstract":"When data resides on tertiary storage, clustering is the key to achieving high retrieval performance. However, a straightforward approach to clustering massive amounts of data on this storage requires considerable computational and storage resources that usually exceed the capabilities of even the richest super-computing centers. This paper develops a new approach to hierarchical storage management in data grid environments, which calls for two levels of clustering data on tertiary storage. Applying a mix of static and dynamic decisions, this approach achieves the benefits of clustering at reasonable costs. However, an effective realization of the approach in generic data grid environments requires advances in the areas of indexing and clustering large scientific data collections on tertiary storage. The paper describes some novel indexing and clustering techniques that can cope well not only with extremely large volumes but also with very high dimensionalities of scientific data. The basic principles of a new clustering technique for large volumes of multi-dimensional data are introduced in the paper for the first time.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"101 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124136893","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A. Azagury, V. Dreizin, M. Factor, E. Henis, D. Naor, N. Rinetzky, O. Rodeh, J. Satran, A. Tavory, Lena Yerushalmi
{"title":"Towards an object store","authors":"A. Azagury, V. Dreizin, M. Factor, E. Henis, D. Naor, N. Rinetzky, O. Rodeh, J. Satran, A. Tavory, Lena Yerushalmi","doi":"10.1109/MASS.2003.1194853","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194853","url":null,"abstract":"Today's SAN architectures promise unmediated host access to storage (i.e., without going through a server). To achieve this promise, however, we must address several issues and opportunities raised by SANs, including security, scalability and management. Object storage, such as introduced by the NASD work, is a means of addressing these issues and opportunities. An object store raises the level of abstraction presented by a storage control unit from an array of 512 byte blocks to a collection of objects. The object store provides \"fine-grain,\" object-level security, improved scalability by localizing space management, and improved management by allowing end-to-end management of semantically meaningful entities. This paper presents a detailed description of how an object store works and describes the design of Antara, our prototype object store. For a cache hit workload, our pure software prototype is able to service roughly 14000 4K I/O requests per second. We also present a layered security model for an object store which separates concerns of access security and network security, leveraging existing security infrastructure.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128529852","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Stephen Aiken, D. Grunwald, A. Pleszkun, Jesse Willek
{"title":"A performance analysis of the iSCSI protocol","authors":"Stephen Aiken, D. Grunwald, A. Pleszkun, Jesse Willek","doi":"10.1109/MASS.2003.1194849","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194849","url":null,"abstract":"Fibre channel has long dominated the realm of storage area networks (SAN's). However, with increased development and refining, iSCSI is fast becoming an equal contender which is causing many companies to reconsider how future storage networks shall be implemented. In addition to reduced costs and a unified network infrastructure, iSCSI allows for the deployment of storage networks over a commodity internet. Moreover there are inexpensive software implementations of the iSCSI protocol that may provide compelling platforms for iSCSI deployment. This paper discusses findings of a performance study on the iSCSI protocol in four different configurations. The first two consist of iSCSI in a more commercial setting, where specialized hardware is used for the iSCSI target and two different configurations are examined for the initiator. These results are contrasted with the performance of fibre channel in a similar setting. The second two configurations focus on iSCSI deployed purely in software in both SAN and WAN environments. The performance results indicate that the iSCSI protocol can be severely limited by the implementation. This is due to either inefficient handling of the underlying network, or to not using sufficient system resources to take advantage of a network.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132508815","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Peabody: the time travelling disk","authors":"C. B. Morrey, D. Grunwald","doi":"10.1109/MASS.2003.1194861","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194861","url":null,"abstract":"Disk drives are now available with capacities on the order of hundreds of gigabytes. What has not become available is an easy way to manage storage. With installed machines located across the enterprise, the backup, management of application installation, and maintenance of systems have become a nightmare. An increasing trend in the storage industry is to virtualize storage resources, maintaining a central repository that can be accessed across the network. We have designed a network block storage device, Peabody, that exposes virtual disks. These virtual disks provide mechanisms to: recover any previous state of their sectors and share backend storage to improve cache utilization and reduce the total amount of storage needed. Peabody is exposed as an iSCSI target, and is mountable by any iSCSI compatible initiator Using our implementation of Peabody, we show that for our workloads, up to 84% of disk sectors written, contain identical content to previously written sectors, motivating the need for content-based coalescing. The overhead for writing in a simple implementation is only 20 percent of the total write speed. This paper describes our early experiences with the Peabody implementation. We quantify how rapidly storage is consumed, examine optimizations, such as content-based coalescing and describe how recovery is currently implemented. We conclude with future plans based on these measurements.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125299540","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Design and implementation of multiple addresses parallel transmission architecture for storage area network","authors":"Bin Meng, Patrick B. T. Khoo, T. C. Chong","doi":"10.1109/MASS.2003.1194838","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194838","url":null,"abstract":"In this paper we present a parallel transmission architecture for the storage area network (SAN). By using two schedulers on the destination and source addresses of packets, the load of multiple data flows between multiple devices can be balanced in an asymmetrical topology without using special hardware. The SAN performance could be scaled flexibly and an additional fault tolerance feature is provided. The load balancing algorithms we provide can be easily implemented and the computation is efficient enough for high-speed transmission.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"99 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116452871","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards optimal I/O scheduling for MEMS-based storage","authors":"Hailing Yu, D. Agrawal, A. E. Abbadi","doi":"10.1109/MASS.2003.1194836","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194836","url":null,"abstract":"MEMS-based storage devices are currently being developed to narrow the gap between processor and disk speeds. MEMS-based storage devices have a different architecture from disk devices, thus algorithms, such as I/O scheduling and data placement, designed for disks need to be revisited. In this paper we focus on developing an I/O scheduling algorithm for MEMS-based storage devices. Our theoretical analysis shows that this algorithm is guaranteed to perform within twice the optimal time for any workload.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"49 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114065732","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Archive management: the missing component","authors":"H. Diamond, J. Bates, D. M. Clark, R. Mairs","doi":"10.1109/MASS.2003.1194834","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194834","url":null,"abstract":"The National Oceanic and Atmospheric Administration's (NOAA) National Environmental Satellite, Data, and Information Service (NESDIS) views the area of Archive Management, as comprising three components: Information Technology Infrastructure, Customer Service, and Scientific Stewardship. This last component, Scientific Stewardship, can be characterized as the long-term preservation of the scientific integrity, monitoring and improving the quality, and the extraction of further knowledge from the data. As our data volumes increase, this component, while being recognized as being important, has suffered. Without this component, Archive Management is static, sterile, and lacks the true ability to provide meaningful information and knowledge derived from the archived data. Proper Scientific Stewardship is performed by scientists and data managers knowledgeable in the scientific assessment of a particular data type, and the practice will ensure more effective data management.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122258995","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Irina Gerasimov, A. Zhuravlev, M. Pershin, Dennis V. Gerasimov
{"title":"Design and implementation of a block storage multi-protocol converter","authors":"Irina Gerasimov, A. Zhuravlev, M. Pershin, Dennis V. Gerasimov","doi":"10.1109/MASS.2003.1194855","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194855","url":null,"abstract":"We present the Block Storage Multi-Protocol Converter (BSMC) software architecture, which is able to translate and manage SCSI commands carried by different SCSI transport protocols, such as SCSI, Fibre Channel, and iSCSI, to a variety of storage devices. We discuss the internal organization of BSMC and present some performance testing results. We describe the variety of applications in Storage Area Networking that can benefit from products based on the BSMC architecture, such as a multi-protocol storage router and a multi-protocol, multi-RAID-level disk array.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129913701","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}