S. Kuo, M. Winslett, Yong Cho, Jonghyun Lee, Ying Chen
{"title":"科学模拟的有效输入和输出","authors":"S. Kuo, M. Winslett, Yong Cho, Jonghyun Lee, Ying Chen","doi":"10.1145/301816.301828","DOIUrl":null,"url":null,"abstract":"Large simulations which run for hundreds of hours on paralle l computers often periodically generate snapshots of states, wh ich are later post-processed to visualize the simulated physical p henomenon. For many applications, fast I/O during post-processing, wh ich is dependent on an efficient organization of data on disk, is as i mportant as minimizing computation-time I/O. In this paper we pr opose optimizations to support efficient parallel I/O for scienti fic simulations and subsequent visualizations. We present an orderin g mechanism to linearize data on disk, a performance model to help t o choose a proper stripe unit size, and a scheduling algorithm to inimize communication contention. Our experiments on an IBM S P show that the combination of these strategies provides a 2025% performance boost.","PeriodicalId":442608,"journal":{"name":"Workshop on I/O in Parallel and Distributed Systems","volume":"30 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"1999-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"15","resultStr":"{\"title\":\"Efficient input and output for scientific simulations\",\"authors\":\"S. Kuo, M. Winslett, Yong Cho, Jonghyun Lee, Ying Chen\",\"doi\":\"10.1145/301816.301828\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Large simulations which run for hundreds of hours on paralle l computers often periodically generate snapshots of states, wh ich are later post-processed to visualize the simulated physical p henomenon. For many applications, fast I/O during post-processing, wh ich is dependent on an efficient organization of data on disk, is as i mportant as minimizing computation-time I/O. In this paper we pr opose optimizations to support efficient parallel I/O for scienti fic simulations and subsequent visualizations. We present an orderin g mechanism to linearize data on disk, a performance model to help t o choose a proper stripe unit size, and a scheduling algorithm to inimize communication contention. Our experiments on an IBM S P show that the combination of these strategies provides a 2025% performance boost.\",\"PeriodicalId\":442608,\"journal\":{\"name\":\"Workshop on I/O in Parallel and Distributed Systems\",\"volume\":\"30 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"1999-05-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"15\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Workshop on I/O in Parallel and Distributed Systems\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1145/301816.301828\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Workshop on I/O in Parallel and Distributed Systems","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/301816.301828","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 15
摘要
在并行计算机上运行数百小时的大型模拟通常会周期性地生成状态快照,这些快照稍后会被后处理以可视化模拟的物理现象。对于许多应用程序,后处理期间的快速I/O(这依赖于磁盘上数据的有效组织)与最小化计算时间I/O同样重要。在本文中,我们提出了优化,以支持科学模拟和后续可视化的高效并行I/O。我们提出了一个排序机制来线性化磁盘上的数据,一个性能模型来帮助选择合适的条带单元大小,以及一个调度算法来最大限度地减少通信争用。我们在IBM S - P上的实验表明,这些策略的组合提供了2025%的性能提升。
Efficient input and output for scientific simulations
Large simulations which run for hundreds of hours on paralle l computers often periodically generate snapshots of states, wh ich are later post-processed to visualize the simulated physical p henomenon. For many applications, fast I/O during post-processing, wh ich is dependent on an efficient organization of data on disk, is as i mportant as minimizing computation-time I/O. In this paper we pr opose optimizations to support efficient parallel I/O for scienti fic simulations and subsequent visualizations. We present an orderin g mechanism to linearize data on disk, a performance model to help t o choose a proper stripe unit size, and a scheduling algorithm to inimize communication contention. Our experiments on an IBM S P show that the combination of these strategies provides a 2025% performance boost.