{"title":"Big Data Applications Performance Assurance","authors":"B. Zibitsker","doi":"10.1145/2859889.2883586","DOIUrl":"https://doi.org/10.1145/2859889.2883586","url":null,"abstract":"Today's fast-paced businesses have to make business decisions in real-time. That creates pressure on IT leaders to develop near real-time Big Data and Data Warehouse applications that apply advance analytics against large volumes of data to deliver recommendations fast. Hardware and software used to build Big Data infrastructure is cheap, but management of complex environments is not easy In this presentation we will review role of Performance Assurance incorporating Descriptive, Diagnostic, Predictive, Prescriptive and Control Analytics during each phase of the Application and Data life cycle. We will review challenges and Performance Assurance solutions for Big Data Batch and Real Time applications based on YARN, Map/Reduce, Kafka, Spark/Storm and Cassandra Apache projects","PeriodicalId":265808,"journal":{"name":"Companion Publication for ACM/SPEC on International Conference on Performance Engineering","volume":"88 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128340518","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Challenges in Truly Scaling Services","authors":"Manish Gupta","doi":"10.1145/2859889.2883584","DOIUrl":"https://doi.org/10.1145/2859889.2883584","url":null,"abstract":"Many services, such as healthcare and education are highly human-intensive offerings that remain inaccessible (at acceptable quality level) to large numbers of people. With advances in computational power and increasing digitization of the world, there is an opportunity to apply data analytics to transform these services. This talk will describe opportunities and key challenges, both algorithmic and performance-related, to achieve truly transformational impact. We begin by describing a dire need and an opportunity to improve the healthcare system worldwide by supporting a shift from reactive treatment to more proactive action. As examples of what is possible, we present techniques to predict a class of complications in an ICU, to identify patients in a hospital who are likely to require ICU admission, and measure body vitals through remote sensing at home or workplace for wellness or to screen for diseases and reduce the need for people to visit a hospital. We then describe a system called TutorSpace to help with personalization and improved navigation of videos from massive open online courses to enable more effective learning. We describe some of the performance issues we have faced to make these systems practical, and our approach to those problems. We frame all of the above efforts as examples of using information technology to offer personalized services at massive scale.","PeriodicalId":265808,"journal":{"name":"Companion Publication for ACM/SPEC on International Conference on Performance Engineering","volume":"73 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123897292","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Monitoring-Based Testing of Elastic Cloud Computing Applications","authors":"Michel Albonico, Jean-Marie Mottu, G. Sunyé","doi":"10.1145/2859889.2859890","DOIUrl":"https://doi.org/10.1145/2859889.2859890","url":null,"abstract":"Applications that are exposed to large-scale workloads must ensure elasticity, that is the ability to scale up and down rapidly to meet the demand. Cloud infrastructures provide adaptation tasks, which allow applications to automatically scale up and down straightforwardly. These adaptation tasks drive the system to new states, which may expose implementation errors and therefore must be tested. In this paper, we focus on testing elastic applications during different elasticity-related states. This test is difficult since the elasticity states are not directly controlled by the tester. To execute the test at different elasticity-related states, we propose a monitoring-based procedure. This procedure consists in monitoring the resource status to identify the occurrences of the elasticity states at real-time, and in parallel, execute the state-related tests. To validate our test procedure, we performed experiments on Amazon EC2. These experiments successfully identified non-functional errors.","PeriodicalId":265808,"journal":{"name":"Companion Publication for ACM/SPEC on International Conference on Performance Engineering","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133965141","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Challenges with Applying Performance Testing Methods for Systems Deployed on Shared Environments with Indeterminate Competing Workloads: Position Paper","authors":"A. Bondi","doi":"10.1145/2859889.2859895","DOIUrl":"https://doi.org/10.1145/2859889.2859895","url":null,"abstract":"There is a tendency to move production environments from corporate-owned data centers to cloud-based services. Users who do not maintain a private production environment might not wish to maintain a private performance test environment either. The application of performance engineering methods to the development and delivery of software systems is complicated when the form and or parameters of the target deployment environment cannot be controlled or determined. The difficulty of diagnosing the causes of performance issues during testing or production may be increased by the presence of highly variable workloads on the target platform that compete with the application of interest for resources in ways that might be hard to determine. In particular, performance tests might be conducted in virtualized environments that introduce factors influencing customer-affecting metrics (such as transaction response time) and observed resource usage. Observed resource usage metrics in virtualized environments can have different meanings from those in a native environment. Virtual machines may suffer delays in execution. We explore factors that exacerbate these complications. We argue that these complexities reinforce the case for rigorously using software performance engineering methods rather than diminishing it. We also explore possible performance testing methods for mitigating the risk associated with these complexities.","PeriodicalId":265808,"journal":{"name":"Companion Publication for ACM/SPEC on International Conference on Performance Engineering","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125805733","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Performance Mimicking Benchmarks for Multi-tier Applications","authors":"S. Duttagupta, Mukund Kumar, V. Apte","doi":"10.1145/2859889.2859898","DOIUrl":"https://doi.org/10.1145/2859889.2859898","url":null,"abstract":"Predicting performance of multi-tier enterprise applications for a target platform is of significant importance to IT industries especially when target environment is unavailable for deployment. Performance modeling techniques depend on accurate estimation of resource demands for a specific application. This paper proposes a methodology for deriving Performance Mimicking Benchmarks (PMBs) that can predict resource demand of application server of multi-tier on-line transaction processing applications on a target environment. PMBs do not require the actual application to be deployed on the target itself. These benchmarks invoke similar method calls as the application at different layers in the technology stack that contribute significantly to CPU utilization. Further, they mimic all send and receive interactions with external servers (e.g., database server) and web clients. Ability of PMBs for service demand prediction is validated with a number of sample multi-tier applications including SPECjEnterprise2010 on disparate hardware configurations. These service demands when used in a modified version of Mean Value Analysis algorithm, can predict throughput and response time with accuracy close to 90%.","PeriodicalId":265808,"journal":{"name":"Companion Publication for ACM/SPEC on International Conference on Performance Engineering","volume":"68 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116239120","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Execution Time Compensation for Cloud Applications by Subtracting Steal Time based on Host-Level Sampling","authors":"Masao Yamamoto, Kohta Nakashima","doi":"10.1145/2859889.2859899","DOIUrl":"https://doi.org/10.1145/2859889.2859899","url":null,"abstract":"Accurate measurement of program execution time is indispensable to time-based charge systems and performance debugging in all computer systems. However, cloud application execution time cannot be measured properly because measurement in a virtual machine (VM) includes additional time called steal time. The steal time of each program in a VM is unrecognizable by existing standard operating system (OS) tools. Therefore, it is quite difficult for performance engineers to grasp the accurate execution time of each program in a VM. In this ongoing work, we show the novel point of steal in the broad sense and describe how to compensate for function-level execution time in each program in a VM. Our novel approach works by subtracting steal time, which is based on the time-series data of host-level sampling in each function. We implement our approach as a host-level kernel module based on hardware performance counters and some user-level analysis programs. Therefore, our method requires no modification of user applications, guest OSes, a virtual machine monitor (VMM) or a host OS. Finally, our results demonstrate accurate execution time of a function-level guest program, with an overhead lower than 1% for practical use.","PeriodicalId":265808,"journal":{"name":"Companion Publication for ACM/SPEC on International Conference on Performance Engineering","volume":"105 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126375552","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards the Prediction of the Performance and Energy Efficiency of Distributed Data Management Systems","authors":"Raik Niemann","doi":"10.1145/2859889.2859891","DOIUrl":"https://doi.org/10.1145/2859889.2859891","url":null,"abstract":"The ability to accurately simulate and predict the metrics (e.g. performance and energy consumption) of data management systems offers several benefits. It can save investments in both time and hardware. A prominent example is the resource planning. Given a specific use case, a datacenter operator is able to find the most performant or most energy efficient configuration without performing benchmarks or aquiring the necessary hardware. Another possibility would be to study the effects of architectural changes without having them implemented. In this paper, Queued Petri Nets were used to predict and to study the performance and energy consumption of a distributed data management system like Cassandra. The prediction accuracy was evaluated and compared to actual experimental results. On average, the predicted and experimental results differ only by 8 percent for the performance and 16 percent for the energy efficiency, respectively. In addition to this, the experimental results of the used Cassandra cluster revealed a super-linear behavior for the performance and a sub-linear one for the energy consumption.","PeriodicalId":265808,"journal":{"name":"Companion Publication for ACM/SPEC on International Conference on Performance Engineering","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124601587","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Challenges in Applying Control Theory to Software Performance Engineering for Adaptive Systems","authors":"Davide Arcelli, V. Cortellessa","doi":"10.1145/2859889.2859894","DOIUrl":"https://doi.org/10.1145/2859889.2859894","url":null,"abstract":"Control theory has recently started to be applied to software engineering domain, mostly for managing the behavior of adaptive software systems under external disturbances. In general terms, the main advantage of control theory is that it can be formally proven that controllers achieve their goals (with certain characteristics), whereas the price to pay is that controllers and system-to-be-controlled have to be modeled by equations. The investigation of how suited are control theory techniques to address performance problems is, however, still at the beginning. In this paper we devise the main challenges behind the adoption of control theory in the context of Software Performance Engineering applied to adaptive software systems.","PeriodicalId":265808,"journal":{"name":"Companion Publication for ACM/SPEC on International Conference on Performance Engineering","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127983996","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Reference Architecture for Online Performance Model Extraction in Virtualized Environments","authors":"Simon Spinner, J. Walter, Samuel Kounev","doi":"10.1145/2859889.2859893","DOIUrl":"https://doi.org/10.1145/2859889.2859893","url":null,"abstract":"Performance models can support decisions throughout the life-cycle of a software system. However, the manual construction of such performance models is a complex and time-consuming task requiring deep system knowledge. Therefore, automatic approaches for creating and updating performance models of a running system are necessary. Existing work focuses on single aspects of model extraction or proposes approaches specifically designed for a certain technology stack. In virtualized environments, we often see different applications based on diverse technology stacks sharing the same infrastructure. In order to enable online performance model extraction in such environments, we describe a new reference architecture for integrating different specialized model extraction solutions.","PeriodicalId":265808,"journal":{"name":"Companion Publication for ACM/SPEC on International Conference on Performance Engineering","volume":"616 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127525764","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"DiffLQN: Differential Equation Analysis of Layered Queuing Networks","authors":"Tabea Waizmann, M. Tribastone","doi":"10.1145/2859889.2859896","DOIUrl":"https://doi.org/10.1145/2859889.2859896","url":null,"abstract":"Layered queuing networks are a popular technique in software performance engineering. In this paper we present DiffLQN, a tool for the analysis of networks using ordinary differential equations. It estimates average performance indices such as throughput, utilization, and response time of software and hardware devices. The complexity of computing the solution is independent of the concurrency levels in the model (i.e., thread multiplicities and processing units) and the estimates are theoretically guaranteed to be asymptotically correct for large enough concurrency levels. DiffLQN is designed having in mind compatibility with other tools that support state-of-the-art methods based on mean value analysis.","PeriodicalId":265808,"journal":{"name":"Companion Publication for ACM/SPEC on International Conference on Performance Engineering","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133641888","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}