Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering最新文献

筛选

英文中文

Hunter: Using Change Point Detection to Hunt for Performance Regressions 猎人:使用变化点检测来寻找性能回归

Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering Pub Date : 2023-01-08 DOI: 10.1145/3578244.3583719

Matt Fleming, Piotr Kolaczkowski, Ishita Kumar, Shaunak Das, Sean McCarthy, Pushkala Pattabhiraman, Henrik Ingo

{"title":"Hunter: Using Change Point Detection to Hunt for Performance Regressions","authors":"Matt Fleming, Piotr Kolaczkowski, Ishita Kumar, Shaunak Das, Sean McCarthy, Pushkala Pattabhiraman, Henrik Ingo","doi":"10.1145/3578244.3583719","DOIUrl":"https://doi.org/10.1145/3578244.3583719","url":null,"abstract":"Change point detection has recently gained popularity as a method of detecting performance changes in software due to its ability to cope with noisy data. In this paper we present Hunter, an open source tool that automatically detects performance regressions and improvements in time-series data. Hunter uses a modified E-divisive means algorithm to identify statistically significant changes in normally-distributed performance metrics. We describe the changes we made to the E-divisive means algorithm along with their motivation. The main change we adopted was to replace the significance test using randomized permutations with a Student's t-test, as we discovered that the randomized approach did not produce deterministic results, at least not with a reasonable number of iterations. In addition we've made tweaks that allow us to find change points the original algorithm would not, such as two nearby changes. For evaluation, we developed a method to generate real timeseries, but with artificially injected changes in latency. We used these data sets to compare Hunter against two other well known algorithms, PELT and DYNP. Finally, we conclude with lessons we've learned supporting Hunter across teams with individual responsibility for the performance of their project.","PeriodicalId":160204,"journal":{"name":"Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering","volume":"110 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124685582","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering 2023 ACM/SPEC性能工程国际会议论文集

Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering Pub Date : 1900-01-01 DOI: 10.1145/3578244

引用次数: 0

首页上一页