2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW)最新文献

GPU Direct I/O with HDF5 GPU直接I/O与HDF5

2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW) Pub Date : 2020-11-01 DOI: 10.1109/PDSW51947.2020.00010

J. Ravi, S. Byna, Q. Koziol

{"title":"GPU Direct I/O with HDF5","authors":"J. Ravi, S. Byna, Q. Koziol","doi":"10.1109/PDSW51947.2020.00010","DOIUrl":"https://doi.org/10.1109/PDSW51947.2020.00010","url":null,"abstract":"Exascale HPC systems are being designed with accelerators, such as GPUs, to accelerate parts of applications. In machine learning workloads as well as large-scale simulations that use GPUs as accelerators, the CPU (or host) memory is currently used as a buffer for data transfers between GPU (or device) memory and the file system. If the CPU does not need to operate on the data, then this is sub-optimal because it wastes host memory by reserving space for duplicated data. Furthermore, this “bounce buffer” approach wastes CPU cycles spent on transferring data. A new technique, NVIDIA GPUDirect Storage (GDS), can eliminate the need to use the host memory as a bounce buffer. Thereby, it becomes possible to transfer data directly between the device memory and the file system. This direct data path shortens latency by omitting the extra copy and enables higher-bandwidth. To take full advantage of GDS in existing applications, it is necessary to provide support with existing I/O libraries, such as HDF5 and MPI-IO, which are heavily used in applications. In this paper, we describe our effort of integrating GDS with HDF5, the top I/O library at NERSC and at DOE leadership computing facilities. We design and implement this integration using a HDF5 Virtual File Driver (VFD). The GDS VFD provides a file system abstraction to the application that allows HDF5 applications to perform I/O without the need to move data between CPUs and GPUs explicitly. We compare performance of the HDF5 GDS VFD with explicit data movement approaches and demonstrate superior performance with the GDS method.","PeriodicalId":142923,"journal":{"name":"2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW)","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114084700","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Emulating I/O Behavior in Scientific Workflows on High Performance Computing Systems 高性能计算系统中科学工作流的I/O行为仿真

2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW) Pub Date : 2020-11-01 DOI: 10.1109/PDSW51947.2020.00011

Fahim Chowdhury, Yue Zhu, F. Natale, A. Moody, Elsa Gonsiorowski, K. Mohror, Weikuan Yu

{"title":"Emulating I/O Behavior in Scientific Workflows on High Performance Computing Systems","authors":"Fahim Chowdhury, Yue Zhu, F. Natale, A. Moody, Elsa Gonsiorowski, K. Mohror, Weikuan Yu","doi":"10.1109/PDSW51947.2020.00011","DOIUrl":"https://doi.org/10.1109/PDSW51947.2020.00011","url":null,"abstract":"Scientific application workflows leverage the capabilities of cutting-edge high-performance computing (HPC) facilities to enable complex applications for academia, research, and industry communities. Data transfer and I/O dependency among different modules of modern HPC workflows can increase the complexity and hamper the overall performance of workflows. Understanding this complexity due to data-dependency and dataflow is an essential prerequisite for developing optimization strategies to improve I/O performance and, eventually, the entire workflow. In this paper, we discuss dataflow patterns for workflow applications on HPC systems. As existing I/O benchmarking tools lack in identifying and representing the dataflow in modern HPC workflows, we have implemented Wemul, an open-source workflow I/O emulation framework, to mimic different types of I/O behavior demonstrated by common and complex HPC application workflows for deeper analysis. We elaborate on the features and usage of Wemul, demonstrate its application to HPC workflows, and discuss the insights from the performance analysis results on Lassen supercomputing cluster at Lawrence Livermore National Laboratory (LLNL).","PeriodicalId":142923,"journal":{"name":"2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW)","volume":"70 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125261232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

[Title page] (标题页)

2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW) Pub Date : 2020-11-01 DOI: 10.1109/pdsw51947.2020.00001

引用次数: 0

Gauge: An Interactive Data-Driven Visualization Tool for HPC Application I/O Performance Analysis Gauge:用于HPC应用程序I/O性能分析的交互式数据驱动可视化工具

2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW) Pub Date : 2020-11-01 DOI: 10.1109/PDSW51947.2020.00008

Eliakin del Rosario, Mikaela Currier, Mihailo Isakov, S. Madireddy, Prasanna Balaprakash, P. Carns, R. Ross, K. Harms, S. Snyder, M. Kinsy

{"title":"Gauge: An Interactive Data-Driven Visualization Tool for HPC Application I/O Performance Analysis","authors":"Eliakin del Rosario, Mikaela Currier, Mihailo Isakov, S. Madireddy, Prasanna Balaprakash, P. Carns, R. Ross, K. Harms, S. Snyder, M. Kinsy","doi":"10.1109/PDSW51947.2020.00008","DOIUrl":"https://doi.org/10.1109/PDSW51947.2020.00008","url":null,"abstract":"Understanding and alleviating I/O bottlenecks in HPC system workloads is difficult due to the complex, multilayered nature of HPC I/O subsystems. Even with full visibility into the jobs executed on the system, the lack of tooling makes debugging I/O problems difficult. In this work, we introduce Gauge, an interactive, data-driven, web-based visualization tool for HPC I/O performance analysis. Gauge aids in the process of visualizing and analyzing, in an interactive fashion, large sets of HPC application execution logs. It performs a number of functions met to significantly reduce the cognitive load of navigating these sets - some worth many years of HPC logs. For instance, as its first step in many processing chains, it arranges unordered sets of collected HPC logs into a hierarchy of clusters for later analysis. This clustering step allows application developers to quickly navigate logs, find how their jobs compare to those of their peers in terms of I/O utilization, as well as how to improve their future runs. Similarly, facility operators can use Gauge to ‘get a pulse’ on the workloads running on their HPC systems, find clusters of under performing applications, and diagnose the reason for poor I/O throughput. In this work, we describe how Gauge arrives at the HPC jobs clustering, how it presents data about the jobs, and how it can be used to further narrow down and understand behavior of sets of jobs. We also provide a case study on using Gauge from the perspective of a facility operator.","PeriodicalId":142923,"journal":{"name":"2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW)","volume":"35 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130581920","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 6

Fingerprinting the Checker Policies of Parallel File Systems 对并行文件系统的检查策略进行指纹识别

2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW) Pub Date : 2020-11-01 DOI: 10.1109/PDSW51947.2020.00013

Runzhou Han, Duo Zhang, Mai Zheng

引用次数: 9

Fractional-Overlap Declustered Parity: Evaluating Reliability for Storage Systems 分数重叠聚类奇偶校验:存储系统可靠性评估

2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW) Pub Date : 2020-11-01 DOI: 10.1109/PDSW51947.2020.00009

Huan Ke, Haryadi S. Gunawi, Dominic Manno, David Bonnie, B. Settlemyer

引用次数: 0

Keeping It Real: Why HPC Data Services Don't Achieve I/O Microbenchmark Performance 保持真实:为什么HPC数据服务不能达到I/O微基准性能

2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW) Pub Date : 2020-11-01 DOI: 10.1109/PDSW51947.2020.00006

P. Carns, K. Harms, B. Settlemyer, Brian Atkinson, R. Ross

{"title":"Keeping It Real: Why HPC Data Services Don't Achieve I/O Microbenchmark Performance","authors":"P. Carns, K. Harms, B. Settlemyer, Brian Atkinson, R. Ross","doi":"10.1109/PDSW51947.2020.00006","DOIUrl":"https://doi.org/10.1109/PDSW51947.2020.00006","url":null,"abstract":"HPC storage software developers rely on benchmarks as reference points for performance evaluation. Low-level synthetic microbenchmarks are particularly valuable for isolating performance bottlenecks in complex systems and identifying optimization opportunities. The use of low-level microbenchmarks also entails risk, however, especially if the benchmark behavior does not reflect the nuances of production data services or applications. In those cases, microbenchmark measurements can lead to unrealistic expectations or misdiagnosis of performance problems. Neither benchmark creators nor software developers are necessarily at fault in this scenario, however. The underlying problem is more often a subtle disconnect between the objective of the benchmark and the objective of the developer. In this paper we investigate examples of discrepancies between microbenchmark behavior and software developer expectations. Our goal is to draw attention to these pitfalls and initiate a discussion within the community about how to improve the state of the practice in performance engineering for HPC data services.","PeriodicalId":142923,"journal":{"name":"2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW)","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115576597","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Towards On-Demand I/O Forwarding in HPC Platforms 面向HPC平台的按需I/O转发

2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW) Pub Date : 2020-11-01 DOI: 10.1109/PDSW51947.2020.00007

J. L. Bez, F. Boito, Alberto Miranda, Ramon Nou, Toni Cortes, P. Navaux

{"title":"Towards On-Demand I/O Forwarding in HPC Platforms","authors":"J. L. Bez, F. Boito, Alberto Miranda, Ramon Nou, Toni Cortes, P. Navaux","doi":"10.1109/PDSW51947.2020.00007","DOIUrl":"https://doi.org/10.1109/PDSW51947.2020.00007","url":null,"abstract":"I/O forwarding is an established and widely-adopted technique in HPC to reduce contention and improve I/O performance in the access to shared storage infrastructure. On such machines, this layer is often physically deployed on dedicated nodes, and their connection to the clients is static. Furthermore, the increasingly heterogeneous workloads entering HPC installations stress the I/O stack, requiring tuning and reconfiguration based on the applications' characteristics. Nonetheless, it is not always feasible in a production system to explore the potential benefits of this layer under different configurations without impacting clients. In this paper, we investigate the effects of I/O forwarding on performance by considering the application's I/O access patterns and system characteristics. We aim to explore when forwarding is the best choice for an application, how many I/O nodes it would benefit from, and whether not using forwarding at all might be the correct decision. To gather performance metrics, explore, and understand the impact of forwarding I/O requests of different access patterns, we implemented FORGE, a lightweight I/O forwarding layer in user-space. Using FORGE, we evaluated the optimal forwarding configurations for several access patterns on MareNostrum 4 (Spain) and Santos Dumont (Brazil) supercomputers. Our results demonstrate that shifting the focus from a static system-wide deployment to an on-demand reconfigurable I/O forwarding layer dictated by application demands can improve I/O performance on future machines.","PeriodicalId":142923,"journal":{"name":"2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128691881","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Pangeo Benchmarking Analysis: Object Storage vs. POSIX File System Pangeo基准分析:对象存储与POSIX文件系统

2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW) Pub Date : 2020-10-21 DOI: 10.1109/PDSW51947.2020.00012

Haiying Xu, Kevin Paul, Anderson Banihirwe

{"title":"Pangeo Benchmarking Analysis: Object Storage vs. POSIX File System","authors":"Haiying Xu, Kevin Paul, Anderson Banihirwe","doi":"10.1109/PDSW51947.2020.00012","DOIUrl":"https://doi.org/10.1109/PDSW51947.2020.00012","url":null,"abstract":"Pangeo is a community of scientists and software developers collaborating to enable Big Data Geoscience analysis interactively in the public cloud and on high-performance computing (HPC) systems. At the core of the Pangeo software stack is (1) Xarray, which adds labels to metadata such as dimensions, coordinates and attributes for raw array-oriented data, (2) Dask, which provides parallel computation and out-of-core memory capabilities, and (3) Jupyter Lab which offers the web-based interactive environment to the Pangeo platform. Geoscientists now have a strong candidate software stack to analyze large datasets, and they are very curious about performance differences between the Zarr and NetCDF4 data formats on both traditional file storage systems and object storage. We have written a benchmarking suite for the Pangeo stack that can measure scalability and performance information of both input/output (I/O) throughput and computation. We will describe how we performed these benchmarks, analyzed our results, and we will discuss the pros and cons of the Pangeo software stack in terms of I/O scalability on both cloud and HPC storage systems.","PeriodicalId":142923,"journal":{"name":"2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW)","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115496361","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Message from the Workshop Chairs 来自研讨会主席的信息

2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW) Pub Date : 2009-10-01 DOI: 10.1109/VIZSEC.2009.5375531

D. Frincke, C. Gates, J. Goodall

引用次数: 0