Matija Cankar, Nenad Petrovic, Joao Pita Costa, Ales Cernivec, Jan Antic, Tomaz Martincic, Dejan Štepec
{"title":"Security in DevSecOps: Applying Tools and Machine Learning to Verification and Monitoring Steps","authors":"Matija Cankar, Nenad Petrovic, Joao Pita Costa, Ales Cernivec, Jan Antic, Tomaz Martincic, Dejan Štepec","doi":"10.1145/3578245.3584943","DOIUrl":"https://doi.org/10.1145/3578245.3584943","url":null,"abstract":"Security represents one of the crucial concerns when it comes to DevOps methodology-empowered software development and service delivery process. Considering the adoption of Infrastructure as Code (IaC), even minor flaws could potentially cause fatal consequences, especially in sensitive domains such as healthcare and maritime applications. However, most of the existing solutions tackle either Static Application Security Testing (SAST) or run-time behavior analysis distinctly. In this paper, we propose a) IaC Scan Runner, an open-source solution developed in Python for inspecting a variety of state-of-the-art IaC languages in application design time and b) the run time anomaly detection tool called LOMOS. Both tools work in synergy and provide a valuable contribution to a DevSecOps tool set. The proposed approach is demonstrated and their results will be demonstrated on various case studies showcasing the capabilities of static analysis tool IaC Scan Runner combined with LOMOS - log analysis artificial intelligence-enabled framework.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"15 12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127652276","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Case of Multi-Resource Fairness for Serverless Workflows (Work In Progress Paper)","authors":"Amit Samanta, Ryan Stutsman","doi":"10.1145/3578245.3585033","DOIUrl":"https://doi.org/10.1145/3578245.3585033","url":null,"abstract":"Serverless platforms have exploded in popularity in recent years, but, today, these platforms are still unsuitable for large classes of applications. They perform well for batch-oriented workloads that perform coarse transformations over data asynchronously, but their lack of clear service level agreements (SLAs), high per-invocation overheads, and interference make deploying online applications with stringent response time demands impractical. Our assertion is that beyond the glaring issues like cold start costs, a more fundamental shift is needed in how serverless function invocations are provisioned and scheduled in order to support these more demanding applications. Specifically, we propose a platform that leverages the observability and predictability of serverless functions to enforce multi-resource fairness. We explain why we believe interference across a spectrum of resources (CPU, network, and storage) contributes to lower resource utilization and poor response times for latency-sensitive and high-fanout serverless application patterns. Finally, we propose a new distributed and hierarchical function scheduling architecture that combines lessons from multi-resource fair scheduling, hierarchical scheduling, batch-analytics resource scheduling, and statistics to create an approach that we believe will enable tighter SLAs on serverless platforms than has been possible in the past.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128984814","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A. Bauer, Martin Straesser, Mark Leznik, Lukas Beierlieb, Marius Hadry, Nathaniel Hudson, K. Chard, Samuel Kounev, I. Foster
{"title":"Searching for the Ground Truth: Assessing the Similarity of Benchmarking Runs","authors":"A. Bauer, Martin Straesser, Mark Leznik, Lukas Beierlieb, Marius Hadry, Nathaniel Hudson, K. Chard, Samuel Kounev, I. Foster","doi":"10.1145/3578245.3584693","DOIUrl":"https://doi.org/10.1145/3578245.3584693","url":null,"abstract":"Stable and repeatable measurements are essential for comparing the performance of different systems or applications, and benchmarks are used to ensure accuracy and replication. However, if the corresponding measurements are not stable and repeatable, wrong conclusions can be drawn. To facilitate the task of determining whether the measurements are similar, we used a data set of 586 micro-benchmarks to (i) analyze the data set itself, (ii) examine our previous approach, and (iii) propose and evaluate a heuristic. To evaluate the different approaches, we perform a peer review to assess the dissimilarity of the benchmark runs. Our results show that this task is challenging even for humans and that our heuristic exhibits a sensitivity of 92%.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122661530","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nuria de Lama Sanchez, P. Haase, D. Roman, R.-C. Prodan
{"title":"Boosting the Impact of Extreme and Sustainable Graph Processing for Urgent Societal Challenges in Europe Graph-Massivizer: A Horizon Europe Project","authors":"Nuria de Lama Sanchez, P. Haase, D. Roman, R.-C. Prodan","doi":"10.1145/3578245.3585334","DOIUrl":"https://doi.org/10.1145/3578245.3585334","url":null,"abstract":"We explore the potential of the Graph-Massivizer project funded by the Horizon Europe research and innovation program of the European Union to boost the impact of extreme and sustainable graph processing for mitigating existing urgent societal challenges. Current graph processing platforms do not support diverse workloads, models, languages, and algebraic frameworks. Existing specialized platforms are difficult to use by non-experts and suffer from limited portability and interoperability, leading to redundant efforts and inefficient resource and energy consumption due to vendor and even platform lock-in. While synthetic data emerged as an invaluable resource overshadowing actual data for developing robust artificial intelligence analytics, graph generation remains a challenge due to extreme dimensionality and complexity. On the European scale, this practice is unsustainable and, thus, threatens the possibility of creating a climate-neutral and sustainable economy based on graph data. Making graph processing sustainable is essential but needs credible evidence. The grand vision of the Graph-Massivizer project is a technological solution, coupled with field experiments and experience-sharing, for a high-performance and sustainable graph processing of extreme data with a proper response for any need and organizational size by 2030.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126323123","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Challenges and Possible Approaches for Sustainable Digital Twinning","authors":"Stijn Bellis, J. Denil","doi":"10.1145/3578245.3584927","DOIUrl":"https://doi.org/10.1145/3578245.3584927","url":null,"abstract":"The advance in digital twin technology is creating value for lots of companies. We look at the digital twin design and operation from a sustainability perspective. We identify some challenges related to a digital twin's sustainable design and operation. Finally, we look at some possible approaches, grounded in multi-paradigm modelling to help us create and deploy more sustainable twins.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125160541","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Heuristic Derivation of a Fluid Model from a Layered Queueing Network","authors":"M. Woodside","doi":"10.1145/3578245.3584852","DOIUrl":"https://doi.org/10.1145/3578245.3584852","url":null,"abstract":"Fluid approximations are useful for representing transient behaviour of queueing systems. For layered queues a fluid model has previously been derived indirectly via transformation first to a PEPA model, or via recursive neural networks. This paper presents a derivation directly from the layered queueing mechanisms, starting from a transformation to a context-sensitive layered form. The accuracy of predictions, compared to transient simulations and steady-state solutions, is evaluated and appears to be useful.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130256053","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Design-time Performability Evaluation of Runtime Adaptation Strategies (Work In Progress Paper)","authors":"Martina Rapp, Max Scheerer, Ralf H. Reussner","doi":"10.1145/3578245.3585028","DOIUrl":"https://doi.org/10.1145/3578245.3585028","url":null,"abstract":"Performability is the classic metric for performance evaluation of static systems in case of failures. Compared to static systems, Self-Adaptive Systems (SASs) are inherently more complex due to their constantly changing nature. Thus software architects are facing more complex design decisions which are preferably evaluated at design-time. Model-Based Quality Analysis (MBQA) provides valuable support by putting software architects in a position to take well-founded design decisions about software system quality attributes over the whole development phase of a system. We claim that combining methods from MBQA and established performability concepts support software architects in this decision making process to design effective fault-tolerant adaptation strategies. Our contribution is a model-based approach to evaluate performability-oriented adaptation strategies of SAS at design-time. We demonstrate the applicability of our approach by a proof-of-concept.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114227818","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"ICPE'23 Fast Continuum Workshop Chairs' Welcome","authors":"D. Ardagna, E. Di Nitto, Lorenzo Blasi, F. Lordan","doi":"10.1145/3578245.3584932","DOIUrl":"https://doi.org/10.1145/3578245.3584932","url":null,"abstract":"It is our great pleasure to welcome you to the first FastContinuum Workshop held on April 16th 2023. The goal of the workshop is to foster discussion and collaboration among researchers from cloud/edge/fog computing and performance analysis communities, to share the relevant topics and results of the current approaches proposed by industry and academia. FastContinuum solicited full papers as well as demo and short papers including reports about research activities not mature enough for a full paper as well as new ideas and vision papers. The final program includes four full papers and three short ones. They cover some of the most interesting areas of computing continua, from FaaS development and acceleration to the management of heterogeneous datasets to the development of Infrastructure as Code and the automation of deployment through the computing continuum. DevSecOps is also brought to the attendees' attention as one of the crucial ingredients for proper management of the continuum. The workshop keynote, given by Samuel Kunev, investigates further the area of serverless computing properly positioning the multiple aspects and approaches developed in this area and highlighting the main challenges related to the performance of these approaches. The keynote is held in collaboration with the eleventh International Workshop on Load Testing and Benchmarking of Software Systems (LTB 2023).","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124992412","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Scaling the Metaverse: An AI Perspective","authors":"T. Lorido-Botran","doi":"10.1145/3578245.3584920","DOIUrl":"https://doi.org/10.1145/3578245.3584920","url":null,"abstract":"When one hears the word Metaverse, it is automatically associated with millions of users, immersive experiences and its potential to change our lives. But, what enables the Metaverse to function at such a scale? This talk will present the different challenges associated with handling 55 million daily users within the Roblox Metaverse. From addressing user Quality of Experience, to distributed architecture, programming model and scheduling, we will cover the entire stack underneath. In particular, I will put special attention into the AI-Metaverse relationship. On one hand, a large proportion of workloads are based on one or multiple ML/DL models and I will present the challenges of scaling them. On the other hand, I will explore infrastructure and service model challenges that can be addressed with AI, e.g. multi-resource and multi-datacenter scheduling with Reinforcement learning.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130635122","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Matthijs Jansen, Jesse Donkervliet, A. Trivedi, A. Iosup
{"title":"Can My WiFi Handle the Metaverse? A Performance Evaluation Of Meta's Flagship Virtual Reality Hardware","authors":"Matthijs Jansen, Jesse Donkervliet, A. Trivedi, A. Iosup","doi":"10.1145/3578245.3585022","DOIUrl":"https://doi.org/10.1145/3578245.3585022","url":null,"abstract":"Extending human societies into virtual space through the construction of a metaverse has been a long-term challenge in both industry and academia. Achieving this challenge is now closer than ever due to advances in computer systems, facilitating large-scale online platforms such as Minecraft and Roblox that fulfill an increasing number of societal needs, and extended reality (XR) hardware, which provides users with state-of-the-art immersive experiences. For a metaverse to succeed, we argue that all involved systems must provide consistently good performance. However, there is a lack of knowledge on the performance characteristics of extended reality devices. In this paper, we address this gap and focus on extended- and virtual-reality hardware. We synthesize a user-centered system model that models common deployments of XR hardware and their trade-offs. Based on this model, we design and conduct real-world experiments with Meta's flagship virtual reality device, the Quest Pro. We highlight two surprising results from our findings which show that (i) under our workload, the battery drains 15% faster when using wireless offloading compared to local execution, and (ii) the outdated 2.4 GHz WiFi4 gives surprisingly good performance, with 99% of samples achieving a frame rate of at least 65 Hz, compared to the 72 Hz performance target. Our experimental setup and data are available at https://github.com/atlarge-research/measuring-the-metaverse.","PeriodicalId":190503,"journal":{"name":"Companion of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129585908","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}