{"title":"Parallel Performance Engineering using Score-P and Vampir","authors":"William Williams, H. Brunst","doi":"10.1145/3578245.3583715","DOIUrl":"https://doi.org/10.1145/3578245.3583715","url":null,"abstract":"This tutorial will introduce participants to the Score-P measurement system and the Vampir trace visualization tool for performance analysis. We will provide examples and hands-on exercises covering the full performance engineering workflow cycle on applications that include MPI, OpenMP, and GPU parallelism. Users will learn the following concepts: 1. How to collect an initial profile of their code with Score-P. 2. Evaluation of that profile and its associated measurement overhead. 3. The concepts of scoring and filtering a profile and measurement respectively. 4. How to control the Score-P measurement system via environment variables. 5. How to collect useful traces with acceptable overhead. 6. How to understand trace visualization in Vampir.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130863534","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Core-Level Performance Engineering with the Open-Source Architecture Code Analyzer (OSACA) and the Compiler Explorer","authors":"Jan Laukemann, G. Hager","doi":"10.1145/3578245.3583716","DOIUrl":"https://doi.org/10.1145/3578245.3583716","url":null,"abstract":"While many developers put a lot of effort into optimizing large-scale parallelism, they often neglect the importance of an efficient serial code. Even worse, slow serial code tends to scale very well, hiding the fact that resources are wasted because no definite hardware performance limit (\"bottleneck\") is exhausted. This tutorial conveys the required knowledge to develop a thorough understanding of the interactions between software and hardware on the level of a single CPU core and the lowest memory hierarchy level (the L1 cache). We introduce general out-of-order core architectures and their typical performance bottlenecks using modern x86-64 (Intel Ice Lake) and ARM (Fujitsu A64FX) processors as examples. We then go into detail about x86 and AArch64 assembly code, specifically including vectorization (SIMD), pipeline utilization, critical paths, and loop-carried dependencies. We also demonstrate performance analysis and performance engineering using the Open-Source Architecture Code Analyzer (OSACA) in combination with a dedicated instance of the well-known Compiler Explorer. Various hands-on exercises allow attendees to make their own experiments and measurements and identify in-core performance bottlenecks. Furthermore, we show real-life use cases to emphasize how profitable in-core performance engineering can be.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127829235","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sebastian Frank, Julian Brott, Dominik Kesim, Heiko Holz, Matthias Eschhold, André van Hoorn
{"title":"dqualizer: Domain-Centric Runtime Quality Analysis of Business-Critical Application Systems","authors":"Sebastian Frank, Julian Brott, Dominik Kesim, Heiko Holz, Matthias Eschhold, André van Hoorn","doi":"10.1145/3578245.3584853","DOIUrl":"https://doi.org/10.1145/3578245.3584853","url":null,"abstract":"The runtime quality of application systems - e.g., performance, reliability, and resilience - directly influences companies' business success. Over the last few years, corresponding analysis measures such as load tests or monitoring have become widespread in practice, and mature commercial and open-source tools have been developed. However, these measures are all at the technical level and not interpreted at the (business) domain level. At the same time, software architecture and software development approaches such as Domain-Driven Design (DDD), which are becoming increasingly widespread, essentially do not consider runtime quality concerns despite their criticality. Our envisioned dqualizer approach aims to close the gap between the domain-specificity of application systems and the (technical) measures and findings of quality assurance utilizing a domain-centric approach. For this purpose, we integrate means to model and monitor runtime quality metrics into DDD-based techniques, e.g., Domain Story Telling, that enable domain experts to describe domain-centric runtime quality concerns. Our preliminary results comprise the prototypical extension of a domain story editor for specifying load and resilience tests and reporting test results. Using the editor, we gathered feedback from domain experts in a qualitative user study. Despite the editor's limitations regarding functionality and usability, the feedback indicated that domain experts are able to model runtime quality analyses.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"200 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132538473","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Aratz Manterola Lasa, Sacheendra Talluri, A. Iosup
{"title":"A Reference Architecture for Datacenter Scheduler Programming Abstractions: Design and Experiments (Work In Progress Paper)","authors":"Aratz Manterola Lasa, Sacheendra Talluri, A. Iosup","doi":"10.1145/3578245.3585035","DOIUrl":"https://doi.org/10.1145/3578245.3585035","url":null,"abstract":"Datacenters are the backbone of our digital society, used by the industry, academic researchers, public institutions, etc. To manage resources, data centers make use of sophisticated schedulers. Each scheduler offers a different set of capabilities and users make use of them through the APIs they offer. However, there is not a clear understanding of what programming abstractions they offer, nor why they offer some and not others. Consequently, it is difficult to understand the differences between them and the performance costs that are imposed by their APIs. In this work, we study the programming abstractions offered by industrial schedulers, their shortcomings, and the performance costs of the shortcomings. We propose a general reference architecture for scheduler programming abstractions. Specifically, we analyze the programming abstractions of five popular industrial schedulers, we analyze the differences in their APIs, we identify the missing abstractions, and finally, we carry out an exemplary experiment to demonstrate that schedulers sacrifice performance by under-implementing programming abstractions. In the experiments, we demonstrate that an API extension can improve task runtime by up to 23%. This work allows schedulers to identify their shortcomings and points of improvement in their APIs, but most importantly, provides a reference architecture for existing and future schedulers.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126508367","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Graph-Optimizer: Towards Predictable Large-Scale Graph Processing Workloads","authors":"A. Varbanescu, Andrea Bartolini","doi":"10.1145/3578245.3585340","DOIUrl":"https://doi.org/10.1145/3578245.3585340","url":null,"abstract":"Graph and hardware-specific optimisations lead to orders of magnitude improvements in performance, energy, and cost over conventional graph processing methods. Typical big data platforms, such as Apache MapReduce and Apache Spark, rely on generic primitives, exhibiting poor performance and high financial and environmental costs. Even optimised basic graph operations (BGOs) lack the tools to combine them towards real-world applications. Furthermore, graph topology and dynamics (i.e., changing the number and content of vertices and edges) lead to high variability in computational needs. Primitive predictive models demonstrate they can enable algorithm selection and advanced auto-scaling techniques to ensure better performance, but no such models exist for energy consumption. In this work, we present the Graph-Optimizer tool. Graph-Optimizer uses optimised BGOs and composition rules to capture and model the workload. It combines the workload model with hardware and infrastructure models, predicting performance and energy consumption. Combined with design space exploration, such predictions select codesigned workload implementations to fit a requested performance objective and guarantee their performance bounds during execution.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126158416","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Josu Díaz-de-Arcaya, E. Osaba, Gorka Benguria, I. Etxaniz, Jesus L. Lobo, Juncal Alonso, Ana I. Torre-Bastida, Aitor Almeida
{"title":"IEM: A Unified Lifecycle Orchestrator for Multilingual IaC Deployments","authors":"Josu Díaz-de-Arcaya, E. Osaba, Gorka Benguria, I. Etxaniz, Jesus L. Lobo, Juncal Alonso, Ana I. Torre-Bastida, Aitor Almeida","doi":"10.1145/3578245.3584938","DOIUrl":"https://doi.org/10.1145/3578245.3584938","url":null,"abstract":"Over the last few years, DevOps methodologies have promoted a more streamlined operationalization of software components in production environments. Infrastructure as Code (IaC) technologies play a key role in the lifecycle management of applications, as they promote the delivery of the infrastructural elements alongside the application components. This way, IaC technologies aspire to minimize the problems associated with the environment by providing a repeatable and traceable process. However, there are a large variety of IaC frameworks, each of them focusing on a different phase of the operationalization lifecycle, hence the necessity to master numerous technologies. In this research, we present the IaC Execution Manager (IEM), a tool devoted to providing a unified framework for the operationalization of software components that encompasses the various stages and technologies involved in the application lifecycle. We analyze an industrial use case to improve the current approach and conclude the IEM is a suitable tool for solving the problem as it promotes automation, while reducing the learning curve associated with the required IaC technologies.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125819108","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards Solving the Challenge of Minimal Overhead Monitoring","authors":"D. Reichelt, Stefan Kühne, W. Hasselbring","doi":"10.1145/3578245.3584851","DOIUrl":"https://doi.org/10.1145/3578245.3584851","url":null,"abstract":"The examination of performance changes or the performance behavior of a software requires the measurement of the performance. This is done via probes, i.e., pieces of code which obtain and process measurement data, and which are inserted into the examined application. The execution of those probes in a singular method creates overhead, which deteriorates performance measurements of calling methods and slows down the measurement process. Therefore, an important challenge for performance measurement is the reduction of the measurement overhead. To address this challenge, the overhead should be minimized. Based on an analysis of the sources of performance overhead, we derive the following four optimization options: (1) Source instrumentation instead of AspectJ instrumentation, (2) reduction of measurement data, (3) change of the queue and (4) aggregation of measurement data. We evaluate the effect of these optimization options using the MooBench benchmark. Thereby, we show that these optimizations options reduce the monitoring overhead of the monitoring framework Kieker. For MooBench, the execution duration could be reduced from 4.77 µs to 0.39 µs per method invocation on average.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"87 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114540428","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Enhancing Trace Visualizations for Microservices Performance Analysis","authors":"J. Leone, L. Traini","doi":"10.1145/3578245.3584729","DOIUrl":"https://doi.org/10.1145/3578245.3584729","url":null,"abstract":"Performance analysis of microservices can be a challenging task, as a typical request to these systems involves multiple Remote Procedure Calls (RPC) spanning across independent services and machines. Practitioners primarily rely on distributed tracing tools to closely monitor microservices performance. These tools enable practitioners to trace, collect, and visualize RPC workflows and associated events in the context of individual end-to-end requests. While effective for analyzing individual end-to-end requests, current distributed tracing visualizations often fall short in providing a comprehensive understanding of the system's overall performance. To address this limitation, we propose a novel visualization approach that enables aggregate performance analysis of multiple end-to-end requests. Our approach builds on a previously developed technique for comparing structural differences of request pairs and extends it for aggregate performance analysis of sets of requests. This paper presents our proposal and discusses our preliminary ongoing progress in developing this innovative approach.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"154 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-02-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132374507","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Alberto Archetti, Eugenio Lomurno, Francesco Lattari, André Martin, Matteo Matteucci
{"title":"Heterogeneous Datasets for Federated Survival Analysis Simulation","authors":"Alberto Archetti, Eugenio Lomurno, Francesco Lattari, André Martin, Matteo Matteucci","doi":"10.1145/3578245.3584935","DOIUrl":"https://doi.org/10.1145/3578245.3584935","url":null,"abstract":"Survival analysis studies time-modeling techniques for an event of interest occurring for a population. Survival analysis found widespread applications in healthcare, engineering, and social sciences. However, the data needed to train survival models are often distributed, incomplete, censored, and confidential. In this context, federated learning can be exploited to tremendously improve the quality of the models trained on distributed data while preserving user privacy. However, federated survival analysis is still in its early development, and there is no common benchmarking dataset to test federated survival models. This work provides a novel technique for constructing realistic heterogeneous datasets by starting from existing non-federated datasets in a reproducible way. Specifically, we propose two dataset-splitting algorithms based on the Dirichlet distribution to assign each data sample to a carefully chosen client: quantity-skewed splitting and label-skewed splitting. Furthermore, these algorithms allow for obtaining different levels of heterogeneity by changing a single hyperparameter. Finally, numerical experiments provide a quantitative evaluation of the heterogeneity level using log-rank tests and a qualitative analysis of the generated splits. The implementation of the proposed methods is publicly available in favor of reproducibility and to encourage common practices to simulate federated environments for survival analysis.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"45 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132362133","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","authors":"","doi":"10.1145/3578245","DOIUrl":"https://doi.org/10.1145/3578245","url":null,"abstract":"","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132547614","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}