Lukas Iffländer, J. Walter, Simon Eismann, Samuel Kounev
{"title":"The Vision of Self-aware Reordering of Security Network Function Chains","authors":"Lukas Iffländer, J. Walter, Simon Eismann, Samuel Kounev","doi":"10.1145/3185768.3186309","DOIUrl":"https://doi.org/10.1145/3185768.3186309","url":null,"abstract":"Services provided online are subject to various types of attacks. Security appliances can be chained to protect a system against multiple types of network attacks. The sequence of appliances has a significant impact on the efficiency of the whole chain. While the operation of security appliance chains is currently based on a static order, traffic-aware reordering of security appliances may significantly improve efficiency and accuracy. In this paper, we present the vision of a self-aware system to automatically reorder security appliances according to incoming traffic. To achieve this, we propose to apply a model-based learning, reasoning, and acting (LRA-M) loop. To this end, we describe a corresponding system architecture and explain its building blocks.","PeriodicalId":10596,"journal":{"name":"Companion of the 2018 ACM/SPEC International Conference on Performance Engineering","volume":"19 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79452910","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Walter, Simon Eismann, Johannes Grohmann, Dusan Okanovic, Samuel Kounev
{"title":"Tools for Declarative Performance Engineering","authors":"J. Walter, Simon Eismann, Johannes Grohmann, Dusan Okanovic, Samuel Kounev","doi":"10.1145/3185768.3185777","DOIUrl":"https://doi.org/10.1145/3185768.3185777","url":null,"abstract":"Performance is of particular relevance to software system design, operation, and evolution. However, the application of performance engineering approaches to solve a given user concern is challenging and requires expert knowledge. In this tutorial paper, we guide the reader step-by-step through the answering of performance concerns following the idea of declarative performance engineering. We explain tools available online, which can be used for automating huge parts of the software performance engineering process. In particular, we present a performance concern language, for which we provide automated answering and visualization referring to measurement-based and model-based analysis. We also detail how to derive performance models using automated extraction of architectural performance models and modeling of parametric dependencies.","PeriodicalId":10596,"journal":{"name":"Companion of the 2018 ACM/SPEC International Conference on Performance Engineering","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79824705","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards Automating Representative Load Testing in Continuous Software Engineering","authors":"Henning Schulz, Tobias Angerstein, A. Hoorn","doi":"10.1145/3185768.3186288","DOIUrl":"https://doi.org/10.1145/3185768.3186288","url":null,"abstract":"As an application's performance can significantly impact the user satisfaction and, consequently, the business success, companies need to test performance before delivery. Though load testing allows for testing the performance under representative load by simulating user behavior, it typically entails high maintenance and execution overhead, hindering application in practice. With regard to the trend of continuous software engineering with its parallel and frequently executed delivery pipelines, load testing is even harder to be applied. In this paper, we present our vision of automated, context-specific and low-overhead load testing in continuous software engineering. First, we strive for reducing the maintenance overhead by evolving manual adjustments to generated workload models over a changing environment. Early evaluation results show a seamless evolution over changing user behavior. Building on this, we intend to significantly reduce the execution time and required resources by introducing online-generated load tests that precisely address the relevant context and services under test. Finally, we investigate minimizing the amount of components to be deployed by utilizing load-test-capable performance stubs.","PeriodicalId":10596,"journal":{"name":"Companion of the 2018 ACM/SPEC International Conference on Performance Engineering","volume":"20 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"78256895","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"ABench: Big Data Architecture Stack Benchmark","authors":"Todor Ivanov, Rekha Singhal","doi":"10.1145/3185768.3186300","DOIUrl":"https://doi.org/10.1145/3185768.3186300","url":null,"abstract":"Distributed big data processing and analytics applications demand a comprehensive end-to-end architecture stack consisting of big data technologies. However, there are many possible architecture patterns (e.g. Lambda, Kappa or Pipeline architectures) to choose from when implementing the application requirements. A big data technology in isolation may be best performing for a particular application, but its performance in connection with other technologies depends on the connectors and the environment. Similarly, existing big data benchmarks evaluate the performance of different technologies in isolation, but no work has been done on benchmarking big data architecture stacks as a whole. For example, BigBench (TPCx-BB) may be used to evaluate the performance of Spark, but is it applicable to PySpark or to Spark with Kafka stack as well? What is the impact of having different programming environments and/or any other technology like Spark? This vision paper proposes a new category of benchmark, called ABench, to fill this gap and discusses key aspects necessary for the performance evaluation of different big data architecture stacks.","PeriodicalId":10596,"journal":{"name":"Companion of the 2018 ACM/SPEC International Conference on Performance Engineering","volume":"27 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76737163","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Cloud Benchmark Suite Combining Micro and Applications Benchmarks","authors":"Joel Scheuner, P. Leitner","doi":"10.1145/3185768.3186286","DOIUrl":"https://doi.org/10.1145/3185768.3186286","url":null,"abstract":"Micro and application performance benchmarks are commonly used to guide cloud service selection. However, they are often considered in isolation in a hardly reproducible setup with a flawed execution strategy. This paper presents a new execution methodology that combines micro and application benchmarks into a benchmark suite called RMIT Combined, integrates this suite into an automated cloud benchmarking environment, and implements a repeatable execution strategy. Additionally, a newly crafted Web serving benchmark called WPBench with three different load scenarios is contributed. A case study in the Amazon EC2 cloud demonstrates that choosing a cost-efficient instance type can deliver up to 40% better performance with 40% lower costs at the same time for the Web serving benchmark WPBench. Contrary to prior research, our findings reveal that network performance does not vary relevantly anymore. Our results also show that choosing a modern type of virtualization can improve disk utilization up to 10% for I/O-heavy workloads.","PeriodicalId":10596,"journal":{"name":"Companion of the 2018 ACM/SPEC International Conference on Performance Engineering","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84538219","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
T. Bures, V. Matena, R. Mirandola, Lorenzo Pagliari, Catia Trubiani
{"title":"Performance Modelling of Smart Cyber-Physical Systems","authors":"T. Bures, V. Matena, R. Mirandola, Lorenzo Pagliari, Catia Trubiani","doi":"10.1145/3185768.3186306","DOIUrl":"https://doi.org/10.1145/3185768.3186306","url":null,"abstract":"Context: the dynamic nature of complex Cyber-Physical Systems (CPS) introduces new research challenges since they need to smartly self-adapt to changing situations in their environment. This triggers the usage of methodologies that keep track of changes and raise alarms whether extra-functional requirements (e.g., safety, reliability, performance) are violated. Objective: this paper investigates the usage of software performance engineering techniques as support to provide a model-based performance evaluation of smart CPS. The goal is to understand at which extent performance models, specifically Queueing Networks (QN), are suitable to represent these dynamic scenarios. Method and Results: we evaluate the performance characteristics of a smart parking application where cars need to communicate with hot-spots to find an empty spot to park. Through QN we are able to efficiently derive performance predictions that are compared with long-run simulations, and the relative error of model-based analysis results is no larger than 10% when transient or congestion states are discarded. Conclusion: the usage of performance models is promising in this domain and our goal is to experiment further performance models in other CPS case studies to assess their effectiveness.","PeriodicalId":10596,"journal":{"name":"Companion of the 2018 ACM/SPEC International Conference on Performance Engineering","volume":"30 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89965767","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Better Early Than Never: Performance Test Acceleration by Regression Test Selection","authors":"D. Reichelt, Stefan Kühne","doi":"10.1145/3185768.3186289","DOIUrl":"https://doi.org/10.1145/3185768.3186289","url":null,"abstract":"Currently, performance tests take much time and are therefore not able to provide fast feedback. Fast feedback on performance tests would support finding performance problems. In order to accelerate performance tests we provide a regression test selection method for performance tests. It is based on test selection by (1) code analysis and (2) trace analysis. We show the efficiency of our approach by comparison with the test selection tools EKSTAZI and Infinitest.","PeriodicalId":10596,"journal":{"name":"Companion of the 2018 ACM/SPEC International Conference on Performance Engineering","volume":"26 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84394892","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"SPEC CPU2017: Next-Generation Compute Benchmark","authors":"James Bucek, K. Lange, J. V. Kistowski","doi":"10.1145/3185768.3185771","DOIUrl":"https://doi.org/10.1145/3185768.3185771","url":null,"abstract":"Description of the new features of the SPEC CPU2017 industry standard benchmark and its metric calculations.","PeriodicalId":10596,"journal":{"name":"Companion of the 2018 ACM/SPEC International Conference on Performance Engineering","volume":"123 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83873519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
V. Muttillo, G. Valente, L. Pomante, Vincenzo Stoico, Fausto D'Antonio, F. Salice
{"title":"CC4CS: an Off-the-Shelf Unifying Statement-Level Performance Metric for HW/SW Technologies","authors":"V. Muttillo, G. Valente, L. Pomante, Vincenzo Stoico, Fausto D'Antonio, F. Salice","doi":"10.1145/3185768.3186291","DOIUrl":"https://doi.org/10.1145/3185768.3186291","url":null,"abstract":"Outlining the general characteristics of embedded systems is an arduous task. In fact, the design of such kind of systems is heavily influenced by functional and non-functional requirements, and it is based on quite complex design flows. So, there is often the need to adopt a HW/SW co-design methodology able to support the designers during high-level phases so that they can perform early analysis before dealing with low-level ones. Such a methodology, to be effective, should consider also performance estimation and ESL HW/SW timing co-simulation. The goal of this paper is to introduce a novel and fast performance metric able to speed-up the early analysis and design space exploration to identify the more promising architectures for different application domains. In particular, the paper presents a framework to evaluate such a metric and to perform some preliminary analysis to evaluate its meaningfulness when exploited in the HW/SW domain.","PeriodicalId":10596,"journal":{"name":"Companion of the 2018 ACM/SPEC International Conference on Performance Engineering","volume":"38 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83749995","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Trace Checking of Streaming Applications through DICE-TraCT","authors":"M. Bersani, F. Marconi, M. Rossi","doi":"10.1145/3185768.3186287","DOIUrl":"https://doi.org/10.1145/3185768.3186287","url":null,"abstract":"This paper introduces DICE-TraCT, the tool---part of the DICE toolchain---that allows developers of Data Intensive Applications to analyze traces of executions of such applications and detect deviations from the expected behavior. The tool works in tandem with the companion formal verification tool D-VerT, to check that the parameters used for the sizing of applications and that guarantee the desired safety and timing properties are indeed correct.","PeriodicalId":10596,"journal":{"name":"Companion of the 2018 ACM/SPEC International Conference on Performance Engineering","volume":"126 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80013694","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}