Impact of Metadata Server on a Large Scale File System

2018 IEEE Colombian Conference on Communications and Computing (COLCOM) Pub Date : 2018-05-16 DOI:10.1109/COLCOMCON.2018.8466729

Ripon Patgiri, Sabuzima Nayak, S. Borgohain

{"title":"Impact of Metadata Server on a Large Scale File System","authors":"Ripon Patgiri, Sabuzima Nayak, S. Borgohain","doi":"10.1109/COLCOMCON.2018.8466729","DOIUrl":null,"url":null,"abstract":"Big Data, the most popular paradigm, consists of a huge set of data. Big Data storage is built using block-store, file system, object-store and/or hybrid of these systems, cohered with metadata. In this paper, we present role of the metadata server (MDS) in a file system for Big Data. The metadata is large in size for Big Data storage system, and therefore, a standalone metadata server (MDS) cannot hold entire metadata of the storage system. Thus, the metadata servers are augmented to form a clustered metadata server. Moreover, the clustered metadata server (MDS) defines the retrieval process of data from the clustered storage media. Besides, the clustered MDS defines the scalability of Big Data storage. In addition, the clustered MDS offers high efficiency and fast accessing of data in a large scale data storage. The size of Big Data is typically Petabytes to Exabytes. Therefore, there is a strong requirement of efficient and effective clustered MDS to define the access mechanism of Big Data storage. In this paper, we sketch the effect of MDS on very large scale file system.","PeriodicalId":151973,"journal":{"name":"2018 IEEE Colombian Conference on Communications and Computing (COLCOM)","volume":"65 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2018-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2018 IEEE Colombian Conference on Communications and Computing (COLCOM)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/COLCOMCON.2018.8466729","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 1

Abstract

Big Data, the most popular paradigm, consists of a huge set of data. Big Data storage is built using block-store, file system, object-store and/or hybrid of these systems, cohered with metadata. In this paper, we present role of the metadata server (MDS) in a file system for Big Data. The metadata is large in size for Big Data storage system, and therefore, a standalone metadata server (MDS) cannot hold entire metadata of the storage system. Thus, the metadata servers are augmented to form a clustered metadata server. Moreover, the clustered metadata server (MDS) defines the retrieval process of data from the clustered storage media. Besides, the clustered MDS defines the scalability of Big Data storage. In addition, the clustered MDS offers high efficiency and fast accessing of data in a large scale data storage. The size of Big Data is typically Petabytes to Exabytes. Therefore, there is a strong requirement of efficient and effective clustered MDS to define the access mechanism of Big Data storage. In this paper, we sketch the effect of MDS on very large scale file system.

查看原文本刊更多论文

元数据服务器对大型文件系统的影响

大数据是最流行的范式，由大量数据组成。大数据存储是使用块存储、文件系统、对象存储和/或这些系统的混合构建的，与元数据相结合。本文介绍了元数据服务器(metadata server, MDS)在大数据文件系统中的作用。由于大数据存储系统的元数据规模较大，单个MDS (metadata server)无法容纳整个存储系统的元数据。因此，元数据服务器被扩充为集群元数据服务器。此外，集群元数据服务器(MDS)定义了从集群存储介质中检索数据的过程。此外，集群MDS定义了大数据存储的可扩展性。此外，在大规模数据存储中，集群MDS提供了高效率和快速的数据访问。大数据的大小通常在pb到eb之间。因此，需要高效、有效的集群MDS来定义大数据存储的访问机制。本文简要介绍了MDS在超大规模文件系统中的应用效果。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

2018 IEEE Colombian Conference on Communications and Computing (COLCOM)

自引率

0.00%

发文量