I/O analysis and optimization for an AMR cosmology application

Proceedings. IEEE International Conference on Cluster Computing Pub Date : 2002-09-23 DOI:10.1109/CLUSTR.2002.1137736

Jianwei Li, W. Liao, A. Choudhary, V. Taylor

{"title":"I/O analysis and optimization for an AMR cosmology application","authors":"Jianwei Li, W. Liao, A. Choudhary, V. Taylor","doi":"10.1109/CLUSTR.2002.1137736","DOIUrl":null,"url":null,"abstract":"In this paper we investigate the data access patterns and file I/O behaviors of a production cosmology application that uses the adaptive mesh refinement (AMR) technique for its domain decomposition. This application was originally developed using Hierarchical Data Format (HDF version 4) I/O library and since HDF4 does not provide parallel I/O facilities, the global file I/O operations were carried out by one of the allocated processors. When the number of processors becomes large, the I/O performance of this design degrades significantly due to the high communication cost and sequential file access. In this work, we present two additional I/O implementations, using MPI-IO and parallel HDF version 5, and analyze their impacts to the I/O performance for this typical AMR application. Based on the I/O patterns discovered in this application, we also discuss the interaction between user level parallel I/O operations and different parallel file systems and point out the advantages and disadvantages. The performance results presented in this work are obtained from an SGI Origin2000 using XFS, an IBM SP using GPFS, and a Linux cluster using PVFS.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":"325 1","pages":"119-126"},"PeriodicalIF":0.0000,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"19","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings. IEEE International Conference on Cluster Computing","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/CLUSTR.2002.1137736","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 19

Abstract

In this paper we investigate the data access patterns and file I/O behaviors of a production cosmology application that uses the adaptive mesh refinement (AMR) technique for its domain decomposition. This application was originally developed using Hierarchical Data Format (HDF version 4) I/O library and since HDF4 does not provide parallel I/O facilities, the global file I/O operations were carried out by one of the allocated processors. When the number of processors becomes large, the I/O performance of this design degrades significantly due to the high communication cost and sequential file access. In this work, we present two additional I/O implementations, using MPI-IO and parallel HDF version 5, and analyze their impacts to the I/O performance for this typical AMR application. Based on the I/O patterns discovered in this application, we also discuss the interaction between user level parallel I/O operations and different parallel file systems and point out the advantages and disadvantages. The performance results presented in this work are obtained from an SGI Origin2000 using XFS, an IBM SP using GPFS, and a Linux cluster using PVFS.

查看原文本刊更多论文

AMR宇宙学应用程序的I/O分析和优化

在本文中，我们研究了一个使用自适应网格细化(AMR)技术进行域分解的生产宇宙学应用程序的数据访问模式和文件I/O行为。这个应用程序最初是使用分层数据格式(HDF版本4)I/O库开发的，由于HDF不提供并行I/O设施，全局文件I/O操作由分配的处理器之一执行。当处理器数量变大时，由于通信成本高和顺序文件访问，这种设计的I/O性能会显著下降。在本文中，我们介绍了另外两个使用MPI-IO和并行HDF版本5的I/O实现，并分析了它们对这个典型AMR应用程序的I/O性能的影响。基于在该应用程序中发现的I/O模式，我们还讨论了用户级并行I/O操作与不同并行文件系统之间的交互，并指出其优缺点。本文给出的性能结果来自使用XFS的SGI Origin2000、使用GPFS的IBM SP和使用PVFS的Linux集群。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Proceedings. IEEE International Conference on Cluster Computing

自引率

0.00%

发文量