动态:大规模科学分析的动态多分辨率数据表示

Yuan Tian, S. Klasky, Weikuan Yu, Bin Wang, H. Abbasi, N. Podhorszki, R. Grout
{"title":"动态:大规模科学分析的动态多分辨率数据表示","authors":"Yuan Tian, S. Klasky, Weikuan Yu, Bin Wang, H. Abbasi, N. Podhorszki, R. Grout","doi":"10.1109/NAS.2013.21","DOIUrl":null,"url":null,"abstract":"Fast growing large-scale systems enable scientific applications to run at a much larger scale and accordingly produce gigantic volumes of simulation output. Such data imposes a grand challenge to post-processing tasks such as visualization and data analysis, because these tasks are often performed at a host machine that is remotely located and equipped with much less memory and storage resources. During the simulation runs, it is also desirable for scientists to be able to interactively monitor and steer the progress of simulation. This requires scientific data to be represented in an efficient form for initial exploration and computation steering. In this paper, we propose DynaM a software framework that can represent scientific data in a multiresolution form, and dynamically organize data blocks into an optimized layout for efficient scientific analysis. DynaM supports a convolution-based multiresolution data representation for abstracting scientific data for visualization at a wide spectrum of resolution. To support the efficient generation and retrieval of different data granularities from such representation, a dynamic data organization in DynaM is enabled to cater distinct peculiarities of different size data blocks for efficient and balanced I/O performance. Our experimental results demonstrate that DynaM can efficiently represent large scientific dataset and speed up the visualization of multidimensional scientific data. An up to 29 times speedup is achieved on Jaguar supercomputer at Oak Ridge National Laboratory.","PeriodicalId":213334,"journal":{"name":"2013 IEEE Eighth International Conference on Networking, Architecture and Storage","volume":"71 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2013-07-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"6","resultStr":"{\"title\":\"DynaM: Dynamic Multiresolution Data Representation for Large-Scale Scientific Analysis\",\"authors\":\"Yuan Tian, S. Klasky, Weikuan Yu, Bin Wang, H. Abbasi, N. Podhorszki, R. Grout\",\"doi\":\"10.1109/NAS.2013.21\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Fast growing large-scale systems enable scientific applications to run at a much larger scale and accordingly produce gigantic volumes of simulation output. Such data imposes a grand challenge to post-processing tasks such as visualization and data analysis, because these tasks are often performed at a host machine that is remotely located and equipped with much less memory and storage resources. During the simulation runs, it is also desirable for scientists to be able to interactively monitor and steer the progress of simulation. This requires scientific data to be represented in an efficient form for initial exploration and computation steering. In this paper, we propose DynaM a software framework that can represent scientific data in a multiresolution form, and dynamically organize data blocks into an optimized layout for efficient scientific analysis. DynaM supports a convolution-based multiresolution data representation for abstracting scientific data for visualization at a wide spectrum of resolution. To support the efficient generation and retrieval of different data granularities from such representation, a dynamic data organization in DynaM is enabled to cater distinct peculiarities of different size data blocks for efficient and balanced I/O performance. Our experimental results demonstrate that DynaM can efficiently represent large scientific dataset and speed up the visualization of multidimensional scientific data. An up to 29 times speedup is achieved on Jaguar supercomputer at Oak Ridge National Laboratory.\",\"PeriodicalId\":213334,\"journal\":{\"name\":\"2013 IEEE Eighth International Conference on Networking, Architecture and Storage\",\"volume\":\"71 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2013-07-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"6\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2013 IEEE Eighth International Conference on Networking, Architecture and Storage\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/NAS.2013.21\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2013 IEEE Eighth International Conference on Networking, Architecture and Storage","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/NAS.2013.21","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 6

摘要

快速增长的大规模系统使科学应用能够在更大的规模上运行,从而产生大量的模拟输出。这些数据给可视化和数据分析等后处理任务带来了巨大的挑战,因为这些任务通常是在远程主机上执行的,并且配备的内存和存储资源要少得多。在模拟运行过程中,科学家也希望能够交互式地监控和引导模拟的进程。这就要求科学数据以一种有效的形式表示出来,以便进行初步探索和计算指导。在本文中,我们提出了一个能够以多分辨率形式表示科学数据的软件框架DynaM,并动态地将数据块组织成优化的布局,以实现高效的科学分析。DynaM支持基于卷积的多分辨率数据表示,用于在宽分辨率范围内抽象科学数据以实现可视化。为了支持从这种表示中高效地生成和检索不同的数据粒度,DynaM中的动态数据组织能够满足不同大小数据块的独特特性,从而实现高效和平衡的I/O性能。实验结果表明,DynaM可以有效地表示大型科学数据集,加快多维科学数据的可视化速度。橡树岭国家实验室的美洲虎超级计算机实现了高达29倍的加速。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
DynaM: Dynamic Multiresolution Data Representation for Large-Scale Scientific Analysis
Fast growing large-scale systems enable scientific applications to run at a much larger scale and accordingly produce gigantic volumes of simulation output. Such data imposes a grand challenge to post-processing tasks such as visualization and data analysis, because these tasks are often performed at a host machine that is remotely located and equipped with much less memory and storage resources. During the simulation runs, it is also desirable for scientists to be able to interactively monitor and steer the progress of simulation. This requires scientific data to be represented in an efficient form for initial exploration and computation steering. In this paper, we propose DynaM a software framework that can represent scientific data in a multiresolution form, and dynamically organize data blocks into an optimized layout for efficient scientific analysis. DynaM supports a convolution-based multiresolution data representation for abstracting scientific data for visualization at a wide spectrum of resolution. To support the efficient generation and retrieval of different data granularities from such representation, a dynamic data organization in DynaM is enabled to cater distinct peculiarities of different size data blocks for efficient and balanced I/O performance. Our experimental results demonstrate that DynaM can efficiently represent large scientific dataset and speed up the visualization of multidimensional scientific data. An up to 29 times speedup is achieved on Jaguar supercomputer at Oak Ridge National Laboratory.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信