Anton Strand, Markus Gunnarson, Ricardo Britto, M. Usman
{"title":"Using a Context-Aware Approach to Recommend Code Reviewers: Findings from an Industrial Case Study","authors":"Anton Strand, Markus Gunnarson, Ricardo Britto, M. Usman","doi":"10.1145/3377813.3381365","DOIUrl":"https://doi.org/10.1145/3377813.3381365","url":null,"abstract":"Code review is a commonly used practice in software development. It refers to the process of reviewing new code changes before they are merged with the code base. However, to perform the review, developers are mostly assigned manually to code changes. This may lead to problems such as: a time-consuming selection process, limited pool of known candidates and risk of over-allocation of a few reviewers. To address the above problems, we developed Carrot, a machine learning-based tool to recommend code reviewers. We conducted an improvement case study at Ericsson. We evaluated Carrot using a mixed approach. we evaluated the prediction accuracy using historical data and the metrical Mean Reciprocal Rank (MRR). Furthermore, we deployed the tool in one Ericsson project and evaluated how adequate the recommendations were from the point of view of the tool users and the recommended reviewers. We also asked the opinion of senior developers about the usefulness of the tool. The results show that Carrot can help identify relevant non-obvious reviewers and be of great assistance to new developers. However, there were mixed opinions on Carrot’s ability to assist with workload balancing and the decrease code review lead time.","PeriodicalId":111497,"journal":{"name":"2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114798383","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Anders Fischer-Nielsen, Zhoulai Fu, Ting Su, A. Wąsowski
{"title":"The Forgotten Case of the Dependency Bugs : On the Example of the Robot Operating System","authors":"Anders Fischer-Nielsen, Zhoulai Fu, Ting Su, A. Wąsowski","doi":"10.1145/3377813.3381364","DOIUrl":"https://doi.org/10.1145/3377813.3381364","url":null,"abstract":"A dependency bug is a software fault that manifests itself when accessing an unavailable asset. Dependency bugs are pervasive and we all hate them. This paper presents a case study of dependency bugs in the Robot Operating System (ROS), applying mixed methods: a qualitative investigation of 78 dependency bug reports, a quantitative analysis of 1354 ROS bug reports against 19553 reports in the top 30 GitHub projects, and a design of three dependency linters evaluated on 406 ROS packages.The paper presents a definition and a taxonomy of dependency bugs extracted from data. It describes multiple facets of these bugs and estimates that as many as 15% (!) of all reported bugs are dependency bugs. We show that lightweight tools can find dependency bugs efficiently, although it is challenging to decide which tools to build and difficult to build general tools. We present the research problem to the community, and posit that it should be feasible to eradicate it from software development practice.","PeriodicalId":111497,"journal":{"name":"2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)","volume":"112 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131842885","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Chungha Sung, Shuvendu K. Lahiri, Mike Kaufman, Pallavi Choudhury, Chao Wang
{"title":"Towards Understanding and Fixing Upstream Merge Induced Conflicts in Divergent Forks: An Industrial Case Study","authors":"Chungha Sung, Shuvendu K. Lahiri, Mike Kaufman, Pallavi Choudhury, Chao Wang","doi":"10.1145/3377812.3390800","DOIUrl":"https://doi.org/10.1145/3377812.3390800","url":null,"abstract":"Divergent forks are a common practice in open-source software development to perform long-term, independent and diverging development on top of a popular source repository. However, keeping such divergent downstream forks in sync with the upstream source evolution poses engineering challenges in terms of frequent merge conflicts. In this paper, we conduct the first industrial case study of the implications of frequent merges from upstream and the resulting merge conflicts, in the context of Microsoft Edge development. The study consists of two parts. First, we describe the nature of merge conflicts that arise due to merges from upstream and classify them into textual conflicts, build breaks, and test failures. Second, we investigate the feasibility of automatically fixing a class of merge conflicts related to build breaks that consume a significant amount of developer time to rootcause and fix. Towards this end, we have implemented a tool MrgBldBrkFixer and evaluate it on three months of real Microsoft Edge Beta development data, and report encouraging results.","PeriodicalId":111497,"journal":{"name":"2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114309165","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Software Development Data for Architecture Analysis: Expectations, Reality, and Future Directions","authors":"Yuanfang Cai, R. Kazman","doi":"10.1145/3377813.3381357","DOIUrl":"https://doi.org/10.1145/3377813.3381357","url":null,"abstract":"Recently we have worked with a dozen industrial collaborators to pinpoint and quantify architecture debts, from multi-national corporations to startup companies. Our technology leverages a wide range of project data, from source file dependencies to issue records, and we interacted with projects of various sizes and characteristics. Crossing the border between research and practice, we have observed significant gaps in terms of data availability and quality among projects of different kinds. Compared with successful open source projects, data from proprietary projects are rarely complete or well-organized. Consequently, not all projects can benefit from all the features and analyses we provide. This, in turn, made them realize they needed to improve their development processes. In this talk, we categorize the commonly observed differences between open source and proprietary project data, analyze the reasons for such differences, and propose suggestions to minimize the gaps, to facilitate advances to both software research and practice.","PeriodicalId":111497,"journal":{"name":"2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)","volume":"126 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133481382","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jens Meinicke, Chu-Pan Wong, Bogdan Vasilescu, C. Kaestner
{"title":"Exploring Differences and Commonalities between Feature Flags and Configuration Options","authors":"Jens Meinicke, Chu-Pan Wong, Bogdan Vasilescu, C. Kaestner","doi":"10.1145/3377813.3381366","DOIUrl":"https://doi.org/10.1145/3377813.3381366","url":null,"abstract":"Feature flags for continuous deployment and configuration options for customizing software share many similarities, both conceptually and technically. However, neither academic nor practitioner publications seem to clearly compare these two concepts. We argue that a distinction is valuable, as applications, goals, and challenges differ fundamentally between feature flags and configuration options. In this work, we explore the differences and commonalities of both concepts to help understand practices and challenges, and to help transfer existing solutions (e.g., for testing). To better understand feature flags and how they relate to configuration options, we performed nine semi-structured interviews with feature-flag experts. We discovered several distinguishing characteristics but also opportunities for knowledge and technology transfer across both communities. Overall, we think that both communities can learn from each other.","PeriodicalId":111497,"journal":{"name":"2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)","volume":"77 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124106778","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"How to reduce risk effectively in fixed price software development","authors":"V. Gruhn, Niklas Spitczok von Brisinski","doi":"10.1145/3377813.3381361","DOIUrl":"https://doi.org/10.1145/3377813.3381361","url":null,"abstract":"In this paper we consider 20 years of software development at a medium-sized European software company (called adesso). We identify changes and trends in software project management and we name typical risks inherent to fixed price projects (beyond the common sense that unclear requirements, missing domain knowledge and breakdown of project communication are permanent risks). Fixed price projects and their related derivatives can have a huge economical upside for companies like adesso, which goes along with sometimes even larger risks. The goal of this research is to identify both risk drivers and risk reduction or even elimination strategies. On this basis, we introduce the notion of a Project Management Office (PMO) and supporting tools and mechanisms which help to identify project risks early. Experience reported is based on eight years of monitoring software development projects with these tools and mechanisms. We also give some insights how PMO at adesso has changed over time and why these changes were implemented. As key message of this paper we show the correlation between the systematic usage of the PMO tools and a decreasing overspend rate over eight years and more than 320 projects.CCS CONCEPTS• General and reference → Metrics; • Software and its engineering → Software development process management; Risk management.","PeriodicalId":111497,"journal":{"name":"2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)","volume":"154 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115194952","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Automated Identification of Libraries from Vulnerability Data","authors":"Yang Chen, A. Santosa, Asankhaya Sharma, David Lo","doi":"10.1145/3377813.3381360","DOIUrl":"https://doi.org/10.1145/3377813.3381360","url":null,"abstract":"Software Composition Analysis (SCA) has gained traction in recent years with a number of commercial offerings from various companies. SCA involves vulnerability curation process where a group of security researchers, using various data sources, populate a database of open-source library vulnerabilities, which is used by a scanner to inform the end users of vulnerable libraries used by their applications. One of the data sources used is the National Vulnerability Database (NVD). The key challenge faced by the security researchers here is in figuring out which libraries are related to each of the reported vulnerability in NVD. In this article, we report our design and implementation of a machine learning system to help identify the libraries related to each vulnerability in NVD. The problem is that of extreme multi-label learning (XML), and we developed our system using the state-of-the-art FastXML algorithm. Our system is iteratively executed, improving the performance of the model over time. At the time of writing, it achieves F1 @1 score of 0.53 with average F1 @k score for $k=1,2,3$ of 0.51 $(F_{1}$ @k is the harmonic mean of precision@k and recall@k). It has been deployed in Veracode as part of a machine learning system that helps the security researchers identify the likelihood of web data items to be vulnerability-related. In addition, we present evaluation results of our feature engineering and the FastXML tree number used. Our work formulates and solves for the first time library name identification from NVD data as XML, and deploys the solution in a complete production system.CCS CONCEPTS• Security and privacy → Software security engineering; • Software and its engineering → Software maintenance tools.","PeriodicalId":111497,"journal":{"name":"2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129004077","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nengwen Zhao, Junjie Chen, Xiao Peng, Honglin Wang, Xinya Wu, Yuanzong Zhang, Zikai Chen, Xiangzhong Zheng, Xiaohui Nie, Gang Wang, Yong Wu, F. Zhou, Wenchi Zhang, Kaixin Sui, Dan Pei
{"title":"Understanding and Handling Alert Storm for Online Service Systems","authors":"Nengwen Zhao, Junjie Chen, Xiao Peng, Honglin Wang, Xinya Wu, Yuanzong Zhang, Zikai Chen, Xiangzhong Zheng, Xiaohui Nie, Gang Wang, Yong Wu, F. Zhou, Wenchi Zhang, Kaixin Sui, Dan Pei","doi":"10.1145/3377812.3390809","DOIUrl":"https://doi.org/10.1145/3377812.3390809","url":null,"abstract":"Alert is a kind of key data source in monitoring system for online service systems, which is used to record the anomalies in service components and report to engineers. In general, the occurrence of a service failure tends to be along with a large number of alerts, which is called alert storm. However, alert storm brings great challenges to diagnose the failure, because it is time-consuming and tedious for engineers to investigate such an overwhelming number of alerts manually. To help understand alert storm in practice, we conduct the first empirical study of alert storm based on large-scale real-world alert data and gain some valuable insights. Based on the findings obtained from the study, we propose a novel approach to handling alert storm. Specifically, this approach includes alert storm detection which aims to identify alert storm accurately, and alert storm summary which aims to recommend a small set of representative alerts to engineers for failure diagnosis. Our experimental study on real-world dataset demonstrates that our alert storm detection can achieve high F1-score (larger than 0.9). Besides, our alert storm summary can reduce the number of alerts that need to be examined by more than 98% and discover representative alerts accurately. We have successfully applied our approach to the service maintenance of a large commercial bank (China EverBright Bank), and we also share our success stories and lessons learned in industry. CCS CONCEPTS • Software and its engineering $rightarrow$ Maintaining software.","PeriodicalId":111497,"journal":{"name":"2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125889744","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nathan Chong, B. Cook, Konstantinos Kallas, Kareem Khazem, Felipe R. Monteiro, Daniel Schwartz-Narbonne, S. Tasiran, Michael Tautschnig, M. Tuttle
{"title":"Code-Level Model Checking in the Software Development Workflow","authors":"Nathan Chong, B. Cook, Konstantinos Kallas, Kareem Khazem, Felipe R. Monteiro, Daniel Schwartz-Narbonne, S. Tasiran, Michael Tautschnig, M. Tuttle","doi":"10.1145/3377813.3381347","DOIUrl":"https://doi.org/10.1145/3377813.3381347","url":null,"abstract":"This experience report describes a style of applying symbolic model checking developed over the course of four years at Amazon Web Services (AWS). Lessons learned are drawn from proving properties of numerous C-based systems, e.g., custom hypervisors, encryption code, boot loaders, and an IoT operating system. Using our methodology, we find that we can prove the correctness of industrial low-level C-based systems with reasonable effort and predictability. Furthermore, AWS developers are increasingly writing their own formal specifications. All proofs discussed in this paper are publicly available on GitHub.CCS CONCEPTS• Software and its engineering →Formal software verification; Model checking; Correctness; • Theory of computation →Program reasoning.","PeriodicalId":111497,"journal":{"name":"2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133131695","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Emily Kowalczyk, Karan Nair, Zebao Gao, Leo Silberstein, Teng Long, A. Memon
{"title":"Modeling and Ranking Flaky Tests at Apple","authors":"Emily Kowalczyk, Karan Nair, Zebao Gao, Leo Silberstein, Teng Long, A. Memon","doi":"10.1145/3377813.3381370","DOIUrl":"https://doi.org/10.1145/3377813.3381370","url":null,"abstract":"Test flakiness-inability to reliably repeat a test’s Pass/Fail out come-continues to be a significant problem in Industry, adversely impacting continuous integration and test pipelines. Completely eliminating flaky tests is not a realistic option as a significant fraction of system tests (typically non-hermetic) for services-based implementations exhibit some level of flakiness. In this paper, we view the flakiness of a test as a rankable value, which we quantify, track and assign a confidence. We develop two ways to model flakiness, capturing the randomness of test results via entropy, and the temporal variation via flipRate, and aggregating these over time. We have implemented our flakiness scoring service and discuss how its adoption has impacted test suites of two large services at Apple. We show how flakiness is distributed across the tests in these services, including typical score ranges and outliers. The flakiness scores are used to monitor and detect changes in flakiness trends. Evaluation results demonstrate near perfect accuracy in ranking, identification and alignment with human interpretation. The scores were used to identify 2 causes of flakiness in the dataset evaluated, which have been confirmed, and where fixes have been implemented or are underway. Our models reduced flakiness by 44% with less than 1% loss in fault detection.","PeriodicalId":111497,"journal":{"name":"2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)","volume":"174 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132859911","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}