{"title":"Views, objects, and persistence for accessing a high volume global data set","authors":"R. Baldwin","doi":"10.1109/MASS.2003.1194840","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194840","url":null,"abstract":"Efficient access methods are reviewed and explored in relation to the global surface hourly data set and several of its derivative products. Typical access paradigms are compared for object persistence using streamed output files, key/value databases, and object and relational databases (ODBMS/RDBMS). An overview of Java Data Objects (JDO), Enterprise Java Beans (EJB), and other alternatives to persist objects are given. Efficiencies gained by implementing RDBMS views or aggregates are also investigated. Given spatial, temporal, and analytical factors, a conservative estimate for the number of data objects from global hourly data approaches 2 billion.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"147 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115001173","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. Grossman, Yunhong Gu, D. Hanley, X. Hong, Jorge Levera, Marco Mazzucco, D. Lillethun, J. Mambretti, J. Weinberger
{"title":"A case for the global access to large distributed data sets using data webs employing photonic data services","authors":"R. Grossman, Yunhong Gu, D. Hanley, X. Hong, Jorge Levera, Marco Mazzucco, D. Lillethun, J. Mambretti, J. Weinberger","doi":"10.1109/MASS.2003.1194837","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194837","url":null,"abstract":"We argue that data webs employing specialized path services, network protocols, and data protocols can be an effective platform to analyze and access millions of distributed Gigabyte (and larger) size data sets. We have built a prototype of such a data web today and demonstrated that it can effectively access, analyze and mine distributed Gigabyte size data sets even over thousands of miles by using specialized network and data protocols. The prototype uses a server which employs the DataSpace Transfer Protocol (DSTP). Our assumption is that WSDL/SOAP/UDDI-based discovery and description services will enable this same infrastructure to scale to millions of such DSTP-servers.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"104 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116192809","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"NSM: a distributed storage architecture for data-intensive applications","authors":"Zeyad Ali, Q. Malluhi","doi":"10.1109/MASS.2003.1194842","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194842","url":null,"abstract":"Several solutions have been developed to provide data-intensive applications with the highest possible data rates. Such solutions tried to utilize the available network resources through parallel I/O and TCP/IP tuning in order to achieve a better data throughput. The focus was on achieving the highest possible data rate while other Performance enhancements factors were ignored. Furthermore, most of those solutions were point solutions and designed to work in a specific environment for a particular application. In this paper, we introduce the Network Storage Manager (NSM). NSM is a Java-based, high-performance, distributed storage system with auto reconfigurability that has been developed in the Distributed Computing Laboratory at Jackson State University. The system is designed as a framework for data-intensive distributed applications of different natures. In addition to an architecture that employs parallelism, scalability, crash recovery, and portability, NSM provides applications with full control to optimize other application-controllable features by allowing applications to fine-tune such features or even plug-in their modules and use it instead of the standard NSM implementation.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"95 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126142414","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Magnus Lübeck, D. Geppert, K. Nienartowicz, M. Nowak, A. Valassi
{"title":"An overview of a large-scale data migration","authors":"Magnus Lübeck, D. Geppert, K. Nienartowicz, M. Nowak, A. Valassi","doi":"10.1109/MASS.2003.1194835","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194835","url":null,"abstract":"We present an overview of the migration of the data collected by the COMPASS experiment at CERN. In less than three months, almost 300 TB will be migrated at a sustained data rate approaching 100 MB/s, using a distributed system with multiple nodes and data servers. This project, which is being carried out by the Database Group of CERN's IT Division and is expected to be completed by early 2003, involves both a physical media migration, from StorageTek 9940A to StorageTek 9940B tapes, and a data format conversion, from an object database management system to a hybrid persistency mechanism based on flat files referenced in a relational database management system.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117024312","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"zFS - a scalable distributed file system using object disks","authors":"O. Rodeh, A. Teperman","doi":"10.1109/MASS.2003.1194858","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194858","url":null,"abstract":"zFS is a research project aimed at building a decentralized file system that distributes all aspects of file and storage management over a set of cooperating machines interconnected by a high-speed network. zFS is designed to be a file system that scales from a few networked computers to several thousand machines and to be built from commodity off-the-shelf components. The two most prominent features of zFS are its cooperative cache and distributed transactions. zFS integrates the memory of all participating machines into one coherent cache. Thus, instead of going to the disk for a block of data already in one of the machine memories, zFS retrieves the data block from the remote machine. zFS also uses distributed transactions and leases, instead of group-communication and clustering software. This article describes the zFS high-level architecture and how its goals are achieved.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120949291","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Media stability and life expectancies of magnetic tape for use with IBM 3590 and digital linear tape systems","authors":"J. Judge, R. Schmidt, Ron D. Weiss, G. Miller","doi":"10.1109/MASS.2003.1194844","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194844","url":null,"abstract":"The purpose of this study was to determine the environmental stability and life expectancies of magnetic media used with certain high density, digital recording tape systems, specifically the IBM 3590 and both quantum's digital linear tape (DLT) and super DLT products. The goal of the study was to compare the tape stabilities and life expectancies of the corresponding tape media and tape storage products and to determine the most suitable magnetic medium for the archival preservation of electronic records.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"124 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115614555","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"High bandwidth scientific data management using storage area networking: lessons learned at the Starfire Optical Range","authors":"T. S. Duncan, Michael S. Sichler","doi":"10.1109/MASS.2003.1194846","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194846","url":null,"abstract":"Advances in sensor technology, electronic devices, digital processing hardware, and computation systems have vastly increased the volume of data available for collection at scientific research sites. The Starfire Optical Range of the Air Force Research Laboratory Directed Energy Directorate is responsible for the research and development of advanced techniques for atmospheric compensation for large telescopes. Recent experiments have been completed which encompassed more that 100TB of collected data. The collection, processing, archival, and analysis of these data required an entirely new approach for the facility staff. This paper presents the architecture, hardware and software components, integration difficulties, and performance results in a \"lessons learned\" format focused on specific implementation choices, interoperability lessons and issues, and results and capabilities.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"89 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131707529","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A centralized data access model for grid computing","authors":"P. Andrews, Tom Sherwin, B. Banister","doi":"10.1109/MASS.2003.1194864","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194864","url":null,"abstract":"Global access to storage is a common theme of grid computing, with access mechanisms often enforcing a major restriction on the distribution of significant applications across a computational grid. The established approach is to distribute the data with the jobs, sometimes requiring lengthy delays on job completion and the necessity for significant resource discovery to establish local data capabilities. For applications that are truly data intensive, this may render them either highly inefficient, or even incapable of using grid computing environments. In this paper we describe a different approach, where the opportunity to design a grid environment from scratch is used to build a tightly-coupled, data-oriented infrastructure that leverages deep investment in leading edge technology to provide very high-speed, widespread access to large data storage. Results from a geographically distributed grid established for the Supercomputing 2002 conference, using preliminary TeraGrid infrastructure, are included and show encouraging performance including data transfer rates of over 700 MB/s using eight 1 Gb/s links from a storage area network to a 10 Gb/s wide area network.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123339366","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Storage devices, local file system and crossbar network file system characteristics, and 1 terabyte file I/O benchmark on the \"Numerical Simulator III\"","authors":"N. Fujita, H. Ookawa","doi":"10.1109/MASS.2003.1194839","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194839","url":null,"abstract":"We benchmarked a mass storage system, named \"CeMSS\", on the \"Numerical Simulator III\" system. It has 80 RAID-5 disk arrays and 40 LTO tape dives, as a storage devices, and has an HSM based local file system and crossbar network file system. We also described the CeMSS design outline. In order to clear our benchmark perspective we defined the \"standard I/O characteristic\" and \"user 10 pattern\". We recognized that the disk and tape devices of CeMSS are optimized at 2MB and 128+ [KB] I/O size. Using 16-way disks, user application programs can use over 1[GB/s] I/O throughput on the NS-III. Also, under the 80-way disk condition, CeMSS could operate a 1[TB] file within I/O minutes.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"117 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122573781","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"SPIRAL: a client-transparent third-party transfer scheme for network attached disks","authors":"Xiaonan Ma, A. Reddy","doi":"10.1109/MASS.2003.1194862","DOIUrl":"https://doi.org/10.1109/MASS.2003.1194862","url":null,"abstract":"Third-party transfer is a data transfer mechanism where the party initiating the transfer is neither the source nor the sink for the data. In this paper, we present a scheme for supporting third-party transfers on storage systems with network-attached disks (NADs), called SPIRAL. SPIRAL allows NADs to send data directly to clients without going through the server It is transparent to clients and relies only on the linear block interface of the current disks (or the future iSCSI disks). SPIRAL requires no porting of file system or application-level functionality to NADs and requires only simple modifications to server applications. To illustrate our approach, we implemented a prototype system on PCs running Linux. We present experimental results for NFS and HTTP on the prototype to demonstrate the effectiveness of our approach.","PeriodicalId":285161,"journal":{"name":"20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings.","volume":"81 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122931921","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}