Memory-Mapped File Approach for On-Demand Data Co-allocation on Grids

2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid Pub Date : 2009-05-18 DOI:10.1109/CCGRID.2009.22

P. Chen, Jyh-Biau Chang, Yi-Chang Zhuang, C. Shieh, Tyng-Yeu Liang

{"title":"Memory-Mapped File Approach for On-Demand Data Co-allocation on Grids","authors":"P. Chen, Jyh-Biau Chang, Yi-Chang Zhuang, C. Shieh, Tyng-Yeu Liang","doi":"10.1109/CCGRID.2009.22","DOIUrl":null,"url":null,"abstract":"Grid data sharing systems usually provide a data-intensive application with either a pre-staging mechanism or an on-demand access mechanism to access shared data. Pre-staging systems simultaneously download an entire shared file from multiple data sources even when only a tiny file fragment is required. Such systems consume unnecessary data transmission time and storage space. On-demand access systems, on the other hand, download only the required fragments from a single data source. Such systems unfortunately do not fully exploit available network bandwidth. This paper presents a data sharing system, designated as the On-Demand data Co-Allocation (ODCA). ODCA facilitates an unmodified legacy applications to transparently access shared data by using native I/O system calls. ODCA transfers only the necessary fragments on user demand, thereby reducing data transmission time, avoiding wasted network bandwidth and wasted storage space. Moreover, ODCA reduces data waiting time by downloading the file fragments from multiple data sources. Experimental results show ODCA successfully reduces turnaround time in data-intensive applications.","PeriodicalId":118263,"journal":{"name":"2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid","volume":"12 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/CCGRID.2009.22","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 2

Abstract

Grid data sharing systems usually provide a data-intensive application with either a pre-staging mechanism or an on-demand access mechanism to access shared data. Pre-staging systems simultaneously download an entire shared file from multiple data sources even when only a tiny file fragment is required. Such systems consume unnecessary data transmission time and storage space. On-demand access systems, on the other hand, download only the required fragments from a single data source. Such systems unfortunately do not fully exploit available network bandwidth. This paper presents a data sharing system, designated as the On-Demand data Co-Allocation (ODCA). ODCA facilitates an unmodified legacy applications to transparently access shared data by using native I/O system calls. ODCA transfers only the necessary fragments on user demand, thereby reducing data transmission time, avoiding wasted network bandwidth and wasted storage space. Moreover, ODCA reduces data waiting time by downloading the file fragments from multiple data sources. Experimental results show ODCA successfully reduces turnaround time in data-intensive applications.

查看原文本刊更多论文

网格上按需数据协同分配的内存映射文件方法

网格数据共享系统通常为数据密集型应用程序提供预登台机制或按需访问机制来访问共享数据。预分级系统同时从多个数据源下载整个共享文件，即使只需要很小的文件片段。这种系统消耗了不必要的数据传输时间和存储空间。另一方面，按需访问系统只从单个数据源下载所需的片段。不幸的是，这样的系统不能充分利用可用的网络带宽。本文提出了一种数据共享系统，称为按需数据协同分配(ODCA)。ODCA允许未经修改的遗留应用程序通过使用本机I/O系统调用透明地访问共享数据。ODCA只根据用户需要传输必要的分片，从而减少数据传输时间，避免浪费网络带宽和存储空间。此外，ODCA通过从多个数据源下载文件片段来减少数据等待时间。实验结果表明，ODCA成功地减少了数据密集型应用程序的周转时间。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid

自引率

0.00%

发文量