{"title":"Performance Engineering Education: A Viewpoint","authors":"Kishor S. Trivedi","doi":"10.1145/3302541.3314054","DOIUrl":"https://doi.org/10.1145/3302541.3314054","url":null,"abstract":"The successful development and marketing of commercial computer/communication systems requires the ability to quantify their performance and related metrics. Specifically, one should be able to demonstrate that projected customer requirements (QoS, QoE) are met, be able to identify bottlenecks, be able to evaluate and compare different configurations, and be able to evaluate and compare different designs. Performance engineering education should then train students to be able to carry out the above tasks. Exposure to three broad categories of approaches is necessary: Measurements aided by statistical techniques, analytic modeling and simulation. Both, the theory underlying these approaches and software packages that aid such analyses should be exposed. Besides failure-free performance, attention should also be devoted to reliability, availability, performability and survivability. In the current context, power consumption and security have gained importance as well. In this talk, we will take a journey through these issues.","PeriodicalId":231712,"journal":{"name":"Companion of the 2019 ACM/SPEC International Conference on Performance Engineering","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134013586","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Practices in Model Component Reuse for Efficient Dependability Analysis","authors":"F. Machida","doi":"10.1145/3302541.3311525","DOIUrl":"https://doi.org/10.1145/3302541.3311525","url":null,"abstract":"Model-based dependability analysis provides an effective manner to evaluate and design the dependability of critical IT systems by abstracting the system architecture and operations. As the size and the complexity of systems increase, however, the process to compose the dependability model becomes complicated and time-consuming. Improving the efficiency of modeling process is practically an important challenge of dependability engineering. In this paper, we review the techniques for model component reuse that makes dependability model composition and analysis more efficient. In particular, component-based modeling approaches for reliability, availability, maintainability and safety analysis presented in the literature are summarized. In order to effectively apply model component reuse, we advocate the importance of asset-based dependability analysis approach that associates the reusable model components with underlying system development process. Finally, we discuss the necessary extensions of these techniques toward efficient dependability analysis for IoT systems which are significantly affecting real world.","PeriodicalId":231712,"journal":{"name":"Companion of the 2019 ACM/SPEC International Conference on Performance Engineering","volume":"45 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133822827","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"FAB: Framework for Analyzing Benchmarks","authors":"Varun Gohil, Shreyas Singh, M. Awasthi","doi":"10.1145/3302541.3313102","DOIUrl":"https://doi.org/10.1145/3302541.3313102","url":null,"abstract":"Performance evaluation is an integral part of computer architecture research. Rigorous performance evaluation is crucial in order to evaluate novel architectures, and is often carried out using benchmark suites. Each suite has a number of workloads with varying behavior and characteristics. Most analysis is done by analyzing the novel architecture across all workloads of a single benchmark suite. However, computer architects studying optimizations of specific microarchitectural components, require evaluation of their proposals on workloads that stress the component being optimized across multiple benchmark suites. In this paper, we present the design and implementation of FAB - a framework built with Pin and Python based workflow. FAB allows user-driven analysis of benchmarks across multiple axes like instruction distributions, types of instructions etc. through an interactive Python interface to check for desired characteristics, across multiple benchmark suites. FAB aims to provide a toolkit that would allow computer architects to 1) select workloads with desired, user-specified behavior, and 2) create synthetic workloads with desired behavior that have a grounding in real benchmarks.","PeriodicalId":231712,"journal":{"name":"Companion of the 2019 ACM/SPEC International Conference on Performance Engineering","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129107065","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. V. Kistowski, Simon Eismann, Johannes Grohmann, Norbert Schmitt, A. Bauer, Samuel Kounev
{"title":"TeaStore - A Micro-Service Reference Application for Performance Engineers","authors":"J. V. Kistowski, Simon Eismann, Johannes Grohmann, Norbert Schmitt, A. Bauer, Samuel Kounev","doi":"10.1145/3302541.3311966","DOIUrl":"https://doi.org/10.1145/3302541.3311966","url":null,"abstract":"Performance engineering researchers propose and employ various methods to analyze, model, optimize and manage the performance of modern distributed applications. In order to evaluate these methods in realistic scenarios, researchers rely on reference applications. Existing testing and benchmarking applications are usually difficult to set up and either outdated, designed for specific testing scenarios, or do not offer the necessary degrees of freedom. In this paper, we present the TeaStore, a micro-service-based reference application. The TeaStore offers multiple services with various performance characteristics and a high degree of freedom regarding its deployment and configuration to be used as a cloud reference application for researchers. The TeaStore is designed for the evaluation of performance modeling and resource management techniques. We invite researchers to use the TeaStore and provide it open-source, extendable, easily deployable and monitorable.","PeriodicalId":231712,"journal":{"name":"Companion of the 2019 ACM/SPEC International Conference on Performance Engineering","volume":"441 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131956005","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Automated Multi-paradigm Analysis of Extended and Layered Queueing Models with LINE","authors":"G. Casale","doi":"10.1145/3302541.3311959","DOIUrl":"https://doi.org/10.1145/3302541.3311959","url":null,"abstract":"LINE is an open source library to analyze systems that can be modeled by means of queueing theory. Recently, a new major release of the tool (version 2.0.0) has introduced several novel features, which are the focus of this demonstration. These include, among others, an object-oriented modeling language aligned with the abstraction of the Java Modelling Tools (JMT) simulator and a set of native solvers based on state-of-the-art analytical and simulation-based solution paradigms.","PeriodicalId":231712,"journal":{"name":"Companion of the 2019 ACM/SPEC International Conference on Performance Engineering","volume":"915 ","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120939557","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"\"What Did I learn In Performance Analysis last year?\": Teaching Queuing Theory for Long-term Retention","authors":"V. Apte","doi":"10.1145/3302541.3311526","DOIUrl":"https://doi.org/10.1145/3302541.3311526","url":null,"abstract":"This paper presents experiences over thirteen years of teaching a queuing systems-based performance analysis course. We discuss how a 'mathematics first' approach resulted in students not retaining the intuitive concepts of queueing theory, which prompted us to redesign a course which would emphasize the 'common sense' principles of queuing theory as long-term takeaways. We present a sequence of syllabus topics that starts with developing and arriving at a host of queuing systems based insights and 'formulae' without going into the mathematics at all. Our key insight is that in practice, only asymptotic values - at both low and high load - are critical to (a) understand capacities of systems being studied and (2) basic sanity checking of performance measurement experiments. We also present two assignments (one measurement, and one simulation) that we now give, that help in reinforcing the practical applicability of queuing systems to modern server systems. While we do not have formal studies, anecdotally, we have reason to believe that this re-design has helped students retain for the long term, the most essential results of queuing systems, even if they do not study this subject further.","PeriodicalId":231712,"journal":{"name":"Companion of the 2019 ACM/SPEC International Conference on Performance Engineering","volume":"88 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116180098","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Markus Weninger, Lukas Makor, Elias Gander, H. Mössenböck
{"title":"AntTracks TrendViz: Configurable Heap Memory Visualization Over Time","authors":"Markus Weninger, Lukas Makor, Elias Gander, H. Mössenböck","doi":"10.1145/3302541.3313100","DOIUrl":"https://doi.org/10.1145/3302541.3313100","url":null,"abstract":"The complexity of modern applications makes it hard to fix memory leaks and other heap-related problems without tool support. Yet, most state-of-the-art tools share problems that still need to be tackled: (1) They group heap objects only based on their types, ignoring other properties such as allocation sites or data structure compositions. (2) Analyses strongly focus on a single point in time and do not show heap evolution over time. (3) Results are displayed in tables, even though more advanced visualization techniques may ease and improve the analysis. In this paper, we present a novel visualization approach that addresses these shortcomings. Heap objects can be arbitrarily classified, enabling users to group objects based on their needs. Instead of inspecting the size of those object groups at a single point in time, our approach tracks the growth of each object group over time. This growth is then visualized using time-series charts, making it easy to identify suspicious object groups. A drill-down feature enables users to investigate these object groups in more detail. Our approach has been integrated into AntTracks, a trace-based memory monitoring tool, to demonstrate its feasibility.","PeriodicalId":231712,"journal":{"name":"Companion of the 2019 ACM/SPEC International Conference on Performance Engineering","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126830548","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Reproducibility in Benchmarking Parallel Fast Fourier Transform based Applications","authors":"S. Aseeri, B. Muite, D. Takahashi","doi":"10.1145/3302541.3313105","DOIUrl":"https://doi.org/10.1145/3302541.3313105","url":null,"abstract":"An overview of concerns observed in allowing for reproducibility in parallel applications that heavily depend on the three dimensional distributed memory fast Fourier transform are summarized. Suggestions for reproducibility categories for benchmark results are given.","PeriodicalId":231712,"journal":{"name":"Companion of the 2019 ACM/SPEC International Conference on Performance Engineering","volume":"51 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115659083","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Lessons from Teaching Analytical Performance Modeling","authors":"Y. Tay","doi":"10.1145/3302541.3311527","DOIUrl":"https://doi.org/10.1145/3302541.3311527","url":null,"abstract":"This talk summarizes some lessons from teaching a course in analytical performance modeling, specifically: (1) what can an analytical model offer? (2) how a model can be decomposed into submodels,so as to decouple different forces affecting performance, and thus analyze their interaction; (3) making choices in formulating a model; (4) the role of assumptions; (5) Average Value Approximation (AVA); (6) when bottleneck analysis suffices; (7) reducing the parameter space; (8) the concept of analytic validation; and (9) analysis with an analytical model.","PeriodicalId":231712,"journal":{"name":"Companion of the 2019 ACM/SPEC International Conference on Performance Engineering","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121405482","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sraman Choudhury, Srikar Chundury, Subramaniam Kalambur, D. Sitaram
{"title":"Poster Paper Impact Of Software Stack Version On Micro-architecture","authors":"Sraman Choudhury, Srikar Chundury, Subramaniam Kalambur, D. Sitaram","doi":"10.1145/3302541.3311963","DOIUrl":"https://doi.org/10.1145/3302541.3311963","url":null,"abstract":"Open source Big Data frameworks such as Spark have been evolvingquite rapidly. Many of the changes have addressed improvement inperformance mainly focusing on the performance of the entire job executing on a distributed system. Past studies have reported micro-architectural performance characteristics of benchmarks based onthese Big Data frameworks. Given the rapid changes to these frame-works, it is expected that some of these code changes will also have amicro-architectural impact. In this paper, we present a comparativestudy of performance of Apache Spark across two major revisions and demonstrate that there are micro-architectural differences in the way the applications use the hardware.","PeriodicalId":231712,"journal":{"name":"Companion of the 2019 ACM/SPEC International Conference on Performance Engineering","volume":"127 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124611810","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}