Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization最新文献

筛选
英文 中文
In situ particle advection via parallelizing over particles 通过平行于粒子的原位粒子平流
R. Binyahib, D. Pugmire, H. Childs
{"title":"In situ particle advection via parallelizing over particles","authors":"R. Binyahib, D. Pugmire, H. Childs","doi":"10.1145/3364228.3364235","DOIUrl":"https://doi.org/10.1145/3364228.3364235","url":null,"abstract":"We extend the method for particle advection that parallelizes over particles to work in an in situ setting. We then compare our method with the typical method for in situ, parallelizing over data. Our experiments consist of parallelism at 512 cores, a data set with 67 million cells, and ten billion total advection steps. Our findings show that parallelizing over particles can be more than ten times faster for some workloads, for reasonable memory cost. Overall, the significance of these findings is to demonstrate that moving data can be worthwhile in some in situ settings.","PeriodicalId":400017,"journal":{"name":"Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125404368","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
The challenges of elastic in situ analysis and visualization 弹性原位分析和可视化的挑战
Matthieu Dorier, Orcun Yildiz, T. Peterka, R. Ross
{"title":"The challenges of elastic in situ analysis and visualization","authors":"Matthieu Dorier, Orcun Yildiz, T. Peterka, R. Ross","doi":"10.1145/3364228.3364234","DOIUrl":"https://doi.org/10.1145/3364228.3364234","url":null,"abstract":"In situ analysis and visualization have been proposed in high-performance computing (HPC) to enable executing analysis tasks while a simulation is running, bypassing the parallel file system and avoiding the need for storing massive amounts of data. One aspect of in situ analysis that has not been extensively researched to date, however, is elasticity. Current in situ analysis frameworks use a fixed amount of resources and can hardly be scaled up or down dynamically throughout the simulation's run time as a response to changes in the requirements. In this paper, we present the challenges posed by elastic in situ analysis and visualization. We emphasize that elasticity can take various forms. We show the difficulties of supporting each form of elasticity with the state-of-the-art HPC technologies, and we suggest solutions to overcome these difficulties. The resulting four-way classification can be seen as a taxonomy for future elastic in situ analysis and visualization systems.","PeriodicalId":400017,"journal":{"name":"Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132156258","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 11
The impact of work distribution on in situ visualization: a case study 工作分布对现场可视化的影响:一个案例研究
Tobias Rau, P. Gralka, Oliver Fernandes, G. Reina, S. Frey, T. Ertl
{"title":"The impact of work distribution on in situ visualization: a case study","authors":"Tobias Rau, P. Gralka, Oliver Fernandes, G. Reina, S. Frey, T. Ertl","doi":"10.1145/3364228.3364233","DOIUrl":"https://doi.org/10.1145/3364228.3364233","url":null,"abstract":"Large-scale computer simulations generate data at rates that necessitate visual analysis tools to run in situ. The distribution of work on and across nodes of a supercomputer is crucial to utilize compute resources as efficiently as possible. In this paper, we study two work distribution problems in the context of in situ visualization and jointly assess the performance impact of different variants. First, especially for simulations involving heterogeneous loads across their domain, dynamic load balancing can significantly reduce simulation run times. However, the adjustment of the domain partitioning associated with this also has a direct impact on visualization performance. The exact impact of this side effect is largely unclear a priori as generally different criteria are used for balancing simulation and visualization load. Second, on node level, the adequate allocation of threads to simulation or visualization tasks minimizes the performance drain of the simulation while also enabling timely visualization results. In our case study, we jointly study both work distribution aspects with the visualization framework MegaMol coupled in situ on node level to the molecular dynamics simulation ls1 Mardyn on Stampede2 at TACC.","PeriodicalId":400017,"journal":{"name":"Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115304902","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
A new approach for in situ analysis in fully coupled earth system models 全耦合地球系统模型的原位分析新方法
U. Turuncoglu, Baris Önol, M. Ilıcak
{"title":"A new approach for in situ analysis in fully coupled earth system models","authors":"U. Turuncoglu, Baris Önol, M. Ilıcak","doi":"10.1145/3364228.3364231","DOIUrl":"https://doi.org/10.1145/3364228.3364231","url":null,"abstract":"This work focuses on enabling the use of existing in situ analysis and visualization methods within the model coupling framework, which is specially designed software layer to combine physical components of Earth system (i.e., atmosphere, ocean) through a standardized calling interface to simplify common tasks such as interpolation among different numerical grids and synchronization of the model components. The new approach enables simultaneous analysis of the vast amount of data produced by a multi-component Earth system model in an integrated way and facilitates efficient, interoperable, and optimized data analysis and visualization workflows for Earth system science applications. To demonstrate the flexibility and the capability of the new approach, a fully coupled regional atmosphere-ocean modeling application that aims to investigate air-sea interaction in a very high spatial and temporal resolution has been implemented. In the use case application, both the simulation and in situ visualization application run in parallel on a large-scale HPC platform with both software and hardware rendering support through the use of accelerators.","PeriodicalId":400017,"journal":{"name":"Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134181224","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
HDF5 as a vehicle for in transit data movement HDF5作为传输数据移动的载体
Junmin Gu, B. Loring, Kesheng Wu, E. W. Bethel
{"title":"HDF5 as a vehicle for in transit data movement","authors":"Junmin Gu, B. Loring, Kesheng Wu, E. W. Bethel","doi":"10.1145/3364228.3364237","DOIUrl":"https://doi.org/10.1145/3364228.3364237","url":null,"abstract":"For in transit processing, one of the fundamental challenges is the efficient movement of data from producers to consumers. Exploiting the flexibility offered by the SENSEI generic in situ framework, we have developed a number of different in transit data transport mechanisms. In this work, we focus on the transport mechanism that leverages the HDF5 parallel I/O library, and investigate the performance characteristics of this transport mechanism. For in transit use cases at scale on HPC platforms, one might expect that an in transit data transport mechanism that uses faster layers of the storage hierarchy, such as DRAM memory, would always outperform a transport that uses slower layers of the storage hierarchy, such as an NVRAM-based persistent storage presented as a distributed file system. However, our test results show that the performance of the transport using NVRAM is competitive with the transport that uses socket-based data movement across varying levels of producer and consumer concurrency.","PeriodicalId":400017,"journal":{"name":"Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128882598","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
In situ adaptive timestep control and visualization based on the spatio-temporal variations of the simulation results 基于仿真结果时空变化的现场自适应时间步长控制与可视化
Yoshiaki Yamaoka, Kengo Hayashi, Naohisa Sakamoto, J. Nonaka
{"title":"In situ adaptive timestep control and visualization based on the spatio-temporal variations of the simulation results","authors":"Yoshiaki Yamaoka, Kengo Hayashi, Naohisa Sakamoto, J. Nonaka","doi":"10.1145/3364228.3364230","DOIUrl":"https://doi.org/10.1145/3364228.3364230","url":null,"abstract":"Effective visualization of time-varying volumetric data is considered challenging, since in addition to the traditional visualization parameter selections, there is also the need to take care about the dynamic changings between the timesteps. In situ adaptive sampling based on the amount of change between the simulation timepsteps might be helpful for the I/O and visualization cost savings. In this paper, we propose the use of Kernel Density Estimation (KDE) and Kullback-Leibler (KL) divergence, in order to estimate the amount of internal variations between the timesteps, and use this as a parameter for controlling the sampling rate. Needless to say that these selected timesteps should be sufficient to enable the understanding of the underlying phenomena without missing important data features. We confirmed the effectiveness on reducing the number of timepsteps, by using an OpenFOAM CFD simulation with irregular volume data. However, we could also observe that the user-defined parameter selection can highly influence the amount of data size reduction as well as the smoothness of the visualization results, and an optimal parameter selection method remains as a future work.","PeriodicalId":400017,"journal":{"name":"Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114461021","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
Interactive in situ visualization and analysis using Ascent and Jupyter 使用Ascent和Jupyter的交互式原位可视化和分析
Seif Ibrahim, T. Stitt, Matthew Larsen, C. Harrison
{"title":"Interactive in situ visualization and analysis using Ascent and Jupyter","authors":"Seif Ibrahim, T. Stitt, Matthew Larsen, C. Harrison","doi":"10.1145/3364228.3364232","DOIUrl":"https://doi.org/10.1145/3364228.3364232","url":null,"abstract":"There is a gap in interactive in situ solutions for HPC simulations. While we have access to fully-featured visualization UIs through tools like ParaViewCatalyst and VisItLibSim,we lack in situ infrastructure to access more general interactive environments like Jupyter. The Jupyter ecosystem of tools provides a rich paradigm for interactive data analysis and is well-suited to help expand interactive use of in situ. Both the complexity of the Jupyter ecosystem and the security requirements of HPC centers pose challenges to developing software infrastructure that enables direct use of Jupyter in simulation codes. With this work, we developed a system enabling simulations instrumented with Ascent to connect to Jupyter. This system allows simulation users to interact with their data in situ using Jupyter Notebooks. The system combines Ascent's embedded Python filter infrastructure with a Client/Server Jupyter Bridge Kernel to simplify both deployment and security considerations on HPC systems. We describe the design of this system, demonstrate basic usage, and describe a prototype of an Ascent rendering UI built on top of this system.","PeriodicalId":400017,"journal":{"name":"Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115281703","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization 实现极端尺度分析和可视化的现场基础设施研讨会论文集
{"title":"Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization","authors":"","doi":"10.1145/3364228","DOIUrl":"https://doi.org/10.1145/3364228","url":null,"abstract":"","PeriodicalId":400017,"journal":{"name":"Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133375515","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
In-situ visualization for the large scale computing initiative milestone 大规模计算计划里程碑的现场可视化
Jeff Mauldin, Thomas J. Otahal, A. Agelastos, S. Domino
{"title":"In-situ visualization for the large scale computing initiative milestone","authors":"Jeff Mauldin, Thomas J. Otahal, A. Agelastos, S. Domino","doi":"10.1145/3364228.3364229","DOIUrl":"https://doi.org/10.1145/3364228.3364229","url":null,"abstract":"The Sandia National Laboratories (SNL) Large-Scale Computing Initiative (LSCI) milestone required running two parallel simulation codes at scale on the Trinity supercomputer at Los Alamos National Laboratory (LANL) to obtain presentation quality visualization results via in-situ methods. The two simulation codes used were Sandia Parallel Aerosciences Research Code (SPARC) and Nalu, both fluid dynamics codes developed at SNL. The codes were integrated with the ParaView Catalyst in-situ visualization library via the SNL developed Input Output SubSystem (IOSS). The LSCI milestone had a relatively short time-scale for completion of two months. During setup and execution of in-situ visualization for the milestone, there were several challenging issues in the areas of software builds, parallel startup-times, and in the a priori specification of visualizations. This paper will discuss the milestone activities and technical challenges encountered in its completion.","PeriodicalId":400017,"journal":{"name":"Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123596985","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Spack meets singularity: creating movable in-situ analysis stacks with ease Spack满足奇点:轻松创建可移动的原位分析堆栈
Sergei Shudler, N. Ferrier, J. Insley, M. Papka, S. Rizzi
{"title":"Spack meets singularity: creating movable in-situ analysis stacks with ease","authors":"Sergei Shudler, N. Ferrier, J. Insley, M. Papka, S. Rizzi","doi":"10.1145/3364228.3364682","DOIUrl":"https://doi.org/10.1145/3364228.3364682","url":null,"abstract":"In-situ data analysis and visualization is a promising technique to handle the enormous amount of data an extreme-scale application produces. One challenge users often face in adopting in-situ techniques is setting the right environment on a target machine. Platforms such as SENSEI require complex software stacks that consist of various analysis packages and visualization applications. The user has to make sure all these prerequisites exist on the target machine, which often involves compiling and setting them up from scratch. In this paper, we leverage the containers technology (e.g., light-weight virtualization images) and provide users with Singularity containers that encapsulate ready-to-use, movable in-situ software stacks. Moreover, we make use of Spack to ease the process of creating these containers. Finally, we evaluate this solution by running in-situ analysis from within a container on an HPC system.","PeriodicalId":400017,"journal":{"name":"Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization","volume":"61 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132052446","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书