{"title":"Fast Tsunami Simulations for a Real-Time Emergency Response Flow","authors":"T. Goubier, N. Rakowsky, S. Harig","doi":"10.1109/UrgentHPC51945.2020.00008","DOIUrl":"https://doi.org/10.1109/UrgentHPC51945.2020.00008","url":null,"abstract":"In the context of response to disasters, fast and accurate simulations can be integrated into a real-time flow, allowing best situation estimates to be provided at required deadlines for warnings and decisions. In such a pilot centered on earthquakes, the TsunAWI tsunami simulation code has been adapted and improved to provide both fast results with a coarse mesh and more accurate results with a fine mesh. Improvements to the TsunAWI code have allowed simulation runs to reach runtimes compatible with the needs of a workflow with real-time deadlines. A significant component of TsunAWI is the underlying mesh traversal order; the use of heuristics from the gpu rendering community can reach comparable performance to the original space filing curve ordering, and validation against historical cases is underway for the fast simulation results.","PeriodicalId":376471,"journal":{"name":"2020 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130447026","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards an HPC Service Oriented Hybrid Cloud Architecture Designed for Interactive Workflows","authors":"S. Kortas, M. Shaikh","doi":"10.1109/UrgentHPC51945.2020.00010","DOIUrl":"https://doi.org/10.1109/UrgentHPC51945.2020.00010","url":null,"abstract":"We detail Ludion, a service-oriented hybrid architecture, well-adapted to launch, monitor and steer interactive services running either on on-premise HPC resource, on user laptop and workstation or in the cloud. Based on AWS server-less components at virtually no cost, Ludion requires no special privileges. It consists of a catalog of services, a dashboard hosted in the cloud and a set of commands to install on the target resources. In the lifetime of a job, a user can register and publish new service on the dashboard. From this unique location, it is possible to trigger basic commands that are forwarded to the corresponding job. In this article, we expose several typical use cases Ludion was designed for and detail its implementation as well as the services already in place relying on this architecture.","PeriodicalId":376471,"journal":{"name":"2020 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116566831","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
G. Casini, Szymon Klarman, G. Richard, I. Varzinczak, Abhaya C. Nayak, Antonis Bikakis, A. Bochman, Nina Gierasimczuk, S. Hansson, A. Herzig, Aaron Hunter, S. Kaci, A. Kakas, G. Kern-Isberner, S. Kramer, W. Labuschagne, Michael Maher, François Schwarzentruber, Mirek Truszczynski
{"title":"Workshop Organization","authors":"G. Casini, Szymon Klarman, G. Richard, I. Varzinczak, Abhaya C. Nayak, Antonis Bikakis, A. Bochman, Nina Gierasimczuk, S. Hansson, A. Herzig, Aaron Hunter, S. Kaci, A. Kakas, G. Kern-Isberner, S. Kramer, W. Labuschagne, Michael Maher, François Schwarzentruber, Mirek Truszczynski","doi":"10.1109/PERCOMW.2006.144","DOIUrl":"https://doi.org/10.1109/PERCOMW.2006.144","url":null,"abstract":"","PeriodicalId":376471,"journal":{"name":"2020 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC)","volume":"162 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122163836","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A. Kremin, S. Bailey, J. Guy, T. Kisner, Kai Zhang
{"title":"Rapid Processing of Astronomical Data for the Dark Energy Spectroscopic Instrument","authors":"A. Kremin, S. Bailey, J. Guy, T. Kisner, Kai Zhang","doi":"10.1109/UrgentHPC51945.2020.00006","DOIUrl":"https://doi.org/10.1109/UrgentHPC51945.2020.00006","url":null,"abstract":"Dark energy remains one of the least understood areas of modern cosmology and physics. The Dark Energy Spectroscopic Instrument (DESI) was built to improve our understanding through the acquisition of data from over thirty million galaxies. DESI, located at the four meter Mayall Telescope on Kitt Peak in Arizona, is a highly multiplexed instrument capable of acquiring astronomical spectra for 5000 objects simultaneously. This multiplexing, which is larger than any multi-object spectrograph to date, is made more ambitious with an observing strategy that requires up-to-date information of data quality and completeness on a daily basis to inform the scheduling of future observations. With both an order of magnitude larger data volume than its predecessors and more stringent survey demands, novel methods have been developed to rapidly process the data in near-real-time and make inferences based on the results. The data acquired from thirty highly sensitive cameras are transferred from local storage on the mountaintop to the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory. Using an urgent queue operating at NERSC, a parallelized set of routines process the images and extract astronomically relevant information within minutes to hours of acquisition, enabling daytime analysis by the operations manager to make observing program decisions for the following night. The implementation has been run at scale and tested in the spring of 2020 during commissioning of the instrument and proven successful at giving critical feedback to the observing team about the previous night’s data quality. Quantitative and qualitative results will be discussed, along with planned improvements to the workflow before the survey begins full operation in 2021.","PeriodicalId":376471,"journal":{"name":"2020 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC)","volume":"226 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134565224","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Message from the Workshop Chairs","authors":"Xiaobo Zhou","doi":"10.1109/compsac.2017.314","DOIUrl":"https://doi.org/10.1109/compsac.2017.314","url":null,"abstract":"Welcome to the ICCCN 2009 Workshops! As an integral part of the conference, the ICCCN workshop program provides international forums for scientists and engineers to exchange and share their experiences, research results, and new ideas on hot and emerging topics on computer communications and networks. This year, ICCCN hosts seven workshops:","PeriodicalId":376471,"journal":{"name":"2020 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC)","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124151254","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Benchmarking micro-core architectures for detecting disasters at the edge","authors":"Maurice Jamieson, Nick Brown","doi":"10.1109/UrgentHPC51945.2020.00009","DOIUrl":"https://doi.org/10.1109/UrgentHPC51945.2020.00009","url":null,"abstract":"Leveraging real-time data to detect disasters such as wildfires, extreme weather, earthquakes, tsunamis, human health emergencies, or global diseases is an important opportunity. However, much of this data is generated in the field and the volumes involved mean that it is impractical for transmission back to a central data-centre for processing. Instead, edge devices are required to generate insights from sensor data streaming in, but an important question given the severe performance and power constraints that these must operate under is that of the most suitable CPU architecture. One class of device that we believe has a significant role to play here is that of micro-cores, which combine many simple low-power cores in a single chip. However, there are many to choose from, and an important question is which is most suited to what situation. This paper presents the Eithne framework, designed to simplify benchmarking of micro-core architectures. Three benchmarks, LINPACK, DFT and FFT, have been implemented atop of this framework and we use these to explore the key characteristics and concerns of common micro-core designs within the context of operating on the edge for disaster detection. The result of this work is an extensible framework that the community can use help develop and test these devices in the future.","PeriodicalId":376471,"journal":{"name":"2020 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC)","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126606905","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. Kube, R. Churchill, J. Choi, Ruonan Wang, Minjun J. Choi, S. Klasky, C. Chang
{"title":"Near real-time analysis of big fusion data on HPC systems","authors":"R. Kube, R. Churchill, J. Choi, Ruonan Wang, Minjun J. Choi, S. Klasky, C. Chang","doi":"10.1109/UrgentHPC51945.2020.00012","DOIUrl":"https://doi.org/10.1109/UrgentHPC51945.2020.00012","url":null,"abstract":"We are developing the Delta framework that aims to tackle big-data challenges specific to fusion energy sciences. Delta can be used to connect fusion experiments to remote supercomputers. Streaming measurements to distributed compute resources allows to automatically perform high-dimensional data analysis on a cadence that exceeds experimental schedules. Making data analysis results available before the next experiments allows scientists to make more informed decisions about configuration of upcoming experiments. Here we describe how Delta uses database and virtualization facilities, as well as high-performance computing, at the National Energy Research Compute Center to offer a vertically integrated near real-time data analysis and visualization. We also report on ongoing efforts to port the data analysis part of Delta to graphical processing units, which show a reduction of the analysis wall-time for a benchmark workflow by about 35% when compared to a serial implementation.","PeriodicalId":376471,"journal":{"name":"2020 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC)","volume":"49 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133194231","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
S. Cholia, L. Heagy, Matthew L. Henderson, Drew Paine, Jon Hays, L. Bianchi, D. Ghoshal, Fernando Pérez, L. Ramakrishnan
{"title":"Towards Interactive, Reproducible Analytics at Scale on HPC Systems","authors":"S. Cholia, L. Heagy, Matthew L. Henderson, Drew Paine, Jon Hays, L. Bianchi, D. Ghoshal, Fernando Pérez, L. Ramakrishnan","doi":"10.1109/UrgentHPC51945.2020.00011","DOIUrl":"https://doi.org/10.1109/UrgentHPC51945.2020.00011","url":null,"abstract":"The growth in scientific data volumes has resulted in a need to scale up processing and analysis pipelines using High Performance Computing (HPC) systems. These workflows need interactive, reproducible analytics at scale. The Jupyter platform provides core capabilities for interactivity but was not designed for HPC systems. In this paper, we outline our efforts that bring together core technologies based on the Jupyter Platform to create interactive, reproducible analytics at scale on HPC systems. Our work is grounded in a real world science use case - applying geophysical simulations and inversions for imaging the subsurface. Our core platform addresses three key areas of the scientific analysis workflow - reproducibility, scalability, and interactivity. We describe our implemention of a system, using Binder, Science Capsule, and Dask software. We demonstrate the use of this software to run our use case and interactively visualize real-time streams of HDF5 data.","PeriodicalId":376471,"journal":{"name":"2020 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC)","volume":"139 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114655576","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
G. Gibb, Nick Brown, R. Nash, M. Mendes, S. Monedero, H. Fidalgo, Joaquín Ramírez Cisneros, A. Cardil, Max Kontak
{"title":"A Bespoke Workflow Management System for Data-Driven Urgent HPC","authors":"G. Gibb, Nick Brown, R. Nash, M. Mendes, S. Monedero, H. Fidalgo, Joaquín Ramírez Cisneros, A. Cardil, Max Kontak","doi":"10.1109/UrgentHPC51945.2020.00007","DOIUrl":"https://doi.org/10.1109/UrgentHPC51945.2020.00007","url":null,"abstract":"In this paper we present a workflow management system which permits the kinds of data-driven workflows required by urgent computing, namely where new data is integrated into the workflow as a disaster progresses in order refine the predictions as time goes on. This allows the workflow to adapt to new data at runtime, a capability that most workflow management systems do not possess. The workflow management system was developed for the EU-funded VESTEC project, which aims to fuse HPC with real-time data for supporting urgent decision making. We first describe an example workflow from the VESTEC project, and show why existing workflow technologies do not meet the needs of the project. We then go on to present the design of our Workflow Management System, describe how it is implemented into the VESTEC system, and provide an example of the workflow system in use for a test case.","PeriodicalId":376471,"journal":{"name":"2020 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC)","volume":"89 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132712024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}