Walter Willinger, Arpit Gupta, Arthur S. Jacobs, Roman Beltiukov, Ronaldo A. Ferreira, Lisandro Granville
{"title":"A NetAI Manifesto (Part I): Less Explorimentation, More Science","authors":"Walter Willinger, Arpit Gupta, Arthur S. Jacobs, Roman Beltiukov, Ronaldo A. Ferreira, Lisandro Granville","doi":"10.1145/3626570.3626609","DOIUrl":"https://doi.org/10.1145/3626570.3626609","url":null,"abstract":"The application of the latest techniques from artificial intelligence (AI) and machine learning (ML) to improve and automate the decision-making required for solving realworld network security and performance problems (NetAI, for short) has generated great excitement among networking researchers. However, network operators have remained very reluctant when it comes to deploying NetAI-based solutions in their production networks, mainly because the black-box nature of the underlying learning models forces operators to blindly trust these models without having any understanding of how they work, why they work, or when they don't work (and why not). Paraphrasing [1], we argue that to overcome this roadblock and ensure its future success in practice, NetAI \"has to get past its current stage of explorimentation, or the practice of poking around to see what happens, and has to start employing tools of the scientific method.\"","PeriodicalId":35745,"journal":{"name":"Performance Evaluation Review","volume":"71 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135426417","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Role of Advanced Math in Teaching Performance Modeling","authors":"Ziv Scully","doi":"10.1145/3626570.3626591","DOIUrl":"https://doi.org/10.1145/3626570.3626591","url":null,"abstract":"How should we teach performance modeling without assuming a deep mathematical background? One approach is to focus on rigorously studying relatively simple stochastic models that do not require too much math background. But this may leave students underprepared to reason about systems in practice. They have multiple servers, bursty arrivals, heavy tails, and other features that demand more complex stochastic models. Reasoning about these phenomena calls for advanced tools from performance modeling theory, but rigorously learning such tools requires more math background than many computer science and engineering students have.","PeriodicalId":35745,"journal":{"name":"Performance Evaluation Review","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135426418","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Teaching Software Performance Evaluation to Undergrads: Lessons Learned and Challenges","authors":"Diwakar Krishnamurthy","doi":"10.1145/3626570.3626589","DOIUrl":"https://doi.org/10.1145/3626570.3626589","url":null,"abstract":"Recent high-profile performance-related outages and problems in industry clearly establish the importance of imparting performance evaluation skills to students at the undergrad level.","PeriodicalId":35745,"journal":{"name":"Performance Evaluation Review","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135426424","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Report on the Second International Workshop on Teaching Performance Analysis of Computer Systems 2023","authors":"Vittoria de Nitto Persone, Y.C. Tay","doi":"10.1145/3626570.3626587","DOIUrl":"https://doi.org/10.1145/3626570.3626587","url":null,"abstract":"The First International Workshop on Teaching Performance Analysis of Computer Systems (TeaPACS1) in 2021 came to the conclusion among participants that there is a need to continue and regularly update the discussion on teaching strategies, changing the curriculum, reaching out to the systems community, facing the new generation and their new learning habits, etc.","PeriodicalId":35745,"journal":{"name":"Performance Evaluation Review","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135426568","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Insensitivity for Loss Systems with Compatibilities","authors":"Runhan Xie, Kristen Gardner, Rhonda Righter","doi":"10.1145/3626570.3626576","DOIUrl":"https://doi.org/10.1145/3626570.3626576","url":null,"abstract":"In the study of queueing systems, we are often interested in finding the stationary distribution of the system state, which in turn can be used to compute various performance measures of interest. Under the assumption that the arrival process is Poisson and the service time distribution is exponential, many queueing systems have elegant productform stationary distributions. However, the exponentiality assumption is often unrealistic. Although relaxing this assumption typically makes the analysis of the stationary distribution less tractable, a number of queueing systems are insensitive to the service time distribution, meaning that they have the same stationary distribution under all service time distributions. Below we describe several examples.","PeriodicalId":35745,"journal":{"name":"Performance Evaluation Review","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135426571","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Enabling Perception-Driven Optimization in Networking","authors":"Yihua Cheng, Xu Zhang, Junchen Jiang","doi":"10.1145/3626570.3626608","DOIUrl":"https://doi.org/10.1145/3626570.3626608","url":null,"abstract":"Service providers struggle to catch up with the rapid growth in bandwidth and latency demand of Internet videos and other applications. An essential contributor to this resource contention is the assumption that users are equally sensitive to service quality everywhere, so any low-quality incidents must be avoided. However, this assumption is not true. For example, our work and other parallel efforts have shown that more video users can be served with better quality of experience (QoE) if we embrace the fact that the QoE's sensitivity to video quality varies greatly with the video content. To unleash such benefits, the application systems must be driven by not only system measurement data but also user feedback data that capture users' perceptions of service quality. In this short paper, I will highlight some of our recent efforts toward the efficient collection of user feedback and enabling perception-driven optimization for Internet applications.","PeriodicalId":35745,"journal":{"name":"Performance Evaluation Review","volume":"134 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135426572","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Toward Fast Query Serving in Key-Value Store Migration with Approximate Telemetry","authors":"Alexander Braverman, Zaoxing Liu","doi":"10.1145/3626570.3626604","DOIUrl":"https://doi.org/10.1145/3626570.3626604","url":null,"abstract":"Distributed key-value stores scale data analytical processing by spreading data across nodes. Frequent migration of key-value shards between online nodes is a key technique to react to dynamic workload changes for load balancing and service elasticity. During migration, the data is split between a source and a destination, making it difficult to query the exact location. Existing solutions aiming to provide real-time read and write query capabilities during migration may require querying both source and destination servers, doubling the compute/network resources. In this paper, we explore a simple yet effective measurement approach to track the key-value migration status, in order to improve the query-serving performance under migration. In our preliminary prototype, we use a Bloom filter on the destination server to keep track of individual key-value pairs that have been successfully migrated. For key-value pairs that have yet migrated, the information stored in the Bloom filter enables fast forwarding to the source server without the need to check the database. We prototype this design on a local cluster with Redis deployments. Our preliminary results show that this approximate measurement-based design minimizes query losses during migration.","PeriodicalId":35745,"journal":{"name":"Performance Evaluation Review","volume":"73 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135426574","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Julianna Bor, Giuliano Casale, William Knottenbelt, Evgenia Smirni, Andreas Stathopoulos
{"title":"Fitting with matrix exponential mixtures generated by discrete probabilistic scaling","authors":"Julianna Bor, Giuliano Casale, William Knottenbelt, Evgenia Smirni, Andreas Stathopoulos","doi":"10.1145/3626570.3626577","DOIUrl":"https://doi.org/10.1145/3626570.3626577","url":null,"abstract":"Matrix exponential (ME) distributions generalize phase-type distributions; however, their use in queueing theory is hampered by the difficulty of checking their feasibility. We propose a novel ME fitting algorithm that produces a valid distribution by construction. The ME distribution used during the fitting is a product of independent random variables that are easy to control in isolation. Consequently, the calculation of the CDF and the Mellin transform factorizes, making it possible to use these measures for the fitting without significant restriction on the distribution order. Trace-driven queueing simulations indicate that the resulting distributions yield highly accurate results.","PeriodicalId":35745,"journal":{"name":"Performance Evaluation Review","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135426421","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Switching Constrained Online Convex Optimization with Predictions and Feedback Delays","authors":"Weici Pan, Zhenhua Liu","doi":"10.1145/3626570.3626573","DOIUrl":"https://doi.org/10.1145/3626570.3626573","url":null,"abstract":"In various applications such as smart grids, the online player is allowed a limited number of switches among decisions. Additionally, real-world scenarios often involve feedback delays or access to near-future predictions. Motivated by this, we study Online Convex Optimization with a switching limit, incorporating feedback delays and predictions. In this extended abstract, we established a near-optimal regret of O(T/S) for delayed feedbacks and a bound of O(T/S - t ) for predictions of t rounds even though the player is only allowed to move at most S times, in expectation, across T rounds. We developed an algorithm which achieves the bounds in both cases and still works when there are both delays and predictions.","PeriodicalId":35745,"journal":{"name":"Performance Evaluation Review","volume":"71 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135426575","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Exponential Tail Bounds on Queues","authors":"Prakirt Jhunjhunwala, Daniela Hurtado-Lange, Siva Theja Maguluri","doi":"10.1145/3626570.3626580","DOIUrl":"https://doi.org/10.1145/3626570.3626580","url":null,"abstract":"A popular approach to computing performance measures of queueing systems (such as delay and queue length) is studying the system in an asymptotic regime. However, these results are only valid in the limit and often provide bounds for the pre-limit systems that are not optimized and, hence, give loose bounds for the tail probabilities. In this paper, we provide optimized bounds for the tail probabilities of the scaled total queue length in a load-balancing system under Join the Shortest Queue (JSQ). Our bounds characterize the rate of convergence of the tail probabilities to the corresponding heavy traffic values. For the tail probability of the JSQ system, our bounds yield a multiplicative error that arises from three factors: pre-limit tail, pre-exponent error, and State-Space Collapse (SSC). As an immediate corollary of our main theorem, we provide a bound to the tail probabilities of a single-server queue. In this case, the multiplicative error only consists of pre-limit tail and pre-exponent error, since there is no state-space collapse.","PeriodicalId":35745,"journal":{"name":"Performance Evaluation Review","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135426270","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}