{"title":"Detecting Runtime Exceptions by Deep Code Representation Learning with Attention-Based Graph Neural Networks","authors":"Rongfang Li, Bihuan Chen, Fengyi Zhang, Chao Sun, Xin Peng","doi":"10.1109/saner53432.2022.00053","DOIUrl":"https://doi.org/10.1109/saner53432.2022.00053","url":null,"abstract":"Uncaught runtime exceptions have been recognized as one of the commonest root causes of real-life exception bugs in Java applications. However, existing runtime exception detection techniques rely on symbolic execution or random testing, which may suffer the scalability or coverage problem. Rule-based bug detectors (e.g., SpotBugs) provide limited rule support for runtime exceptions. Inspired by the recent successes in applying deep learning to bug detection, we propose a deep learning-based technique, named Drex, to identify not only the types of runtime exceptions that a method might signal but also the statement scopes that might signal the detected runtime exceptions. It is realized by graph-based code representation learning with (i) a lightweight analysis to construct a joint graph of CFG, DFG and AST for each method without requiring a build environment so as to comprehensively characterize statement syntax and semantics and (ii) an attention-based graph neural network to learn statement embeddings in order to distinguish different types of potentially signaled runtime exceptions with interpretability. Our evaluation on 54,255 methods with caught runtime exceptions and 54,255 methods without caught runtime exceptions from 5,996 GitHub Java projects has indicated that Drex improves baseline approaches by up to 18.2% in exact accuracy and 41.6% in F1-score. Drex detects 20 new uncaught runtime exceptions in 13 real-life pro-jects, 7 of them have been fixed, while none of them is detected by rule-based bug detectors (i.e., SpotBugs and PMD).","PeriodicalId":437520,"journal":{"name":"2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","volume":"282 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133229903","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"What Do We Know About Readability of Test Code? - A Systematic Mapping Study","authors":"D. Winkler, Pirmin Urbanke, R. Ramler","doi":"10.1109/saner53432.2022.00135","DOIUrl":"https://doi.org/10.1109/saner53432.2022.00135","url":null,"abstract":"The readability of software code is a key success criterion for understanding and maintaining software systems and tests. In industry practice, a limited number of guidelines aim for improving and assessing the readability of software (test) code. Although several studies focus on investigating the readability of software code, we observed limited research work that focuses on the readability of software test code. In this paper we focus on systematically investigating the characteristics, factors, and assessment criteria that have an impact on the readability of test code. We build on a Systematic Mapping Study (SMS) to identify key characteristics, factors, and assessment criteria that have an impact on test code readability, legibility, and understandability to support and improve maintenance tasks. The result set includes 16 studies for further analysis. The majority of publications focuses on readability investigations of automatically generated test code (88%), often evaluated with surveys to access the readability of test code (44 %). Although several approaches aim at assessing the readability with focus on isolated factors, a combination of different readability aspects within an assessment framework can help to better assess and justify the readability of test code with focus on improving software and system test maintenance.","PeriodicalId":437520,"journal":{"name":"2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133865095","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Static Analysis Warnings and Automatic Fixing: A Replication for C# Projects","authors":"Martin Odermatt, Diego Marcilio, Carlo A. Furia","doi":"10.1109/saner53432.2022.00098","DOIUrl":"https://doi.org/10.1109/saner53432.2022.00098","url":null,"abstract":"Static analyzers have become increasingly popular both as developer tools and as subjects of empirical studies. Whereas static analysis tools exist for disparate programming languages, the bulk of the empirical research has focused on the popular Java programming language. In this paper, we investigate to what extent some known results about using static analyzers for Java change when considering C#-another popular object-oriented language. To this end, we combine two replications of previous Java studies. First, we study which static analysis tools are most widely used among C# developers, and which warnings are more commonly reported by these tools on open-source C# projects. Second, we develop and empirically evaluate EagleRepair: a technique to automatically fix code in response to static analysis warnings; this is a replication of our previous work for Java [20]. Our replication indicates, among other things, that 1) static code analysis is fairly popular among C# developers too; 2) Re-Sharper is the most widely used static analyzer for C#; 3) several static analysis rules are commonly violated in both Java and C# projects; 4) automatically generating fixes to static code analysis warnings with good precision is feasible in C#. The EagleRepair tool developed for this research is available as open source.","PeriodicalId":437520,"journal":{"name":"2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","volume":"137 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115791335","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"On the rise and fall of CI services in GitHub","authors":"M. Golzadeh, Alexandre Decan, T. Mens","doi":"10.1109/saner53432.2022.00084","DOIUrl":"https://doi.org/10.1109/saner53432.2022.00084","url":null,"abstract":"Continuous integration (CI) services are used in collaborative open source projects to automate parts of the development workflow. Such services have been in widespread use for over a decade, with new CIs being introduced over the years, sometimes overtaking other CIs in popularity. We conducted a longitudinal empirical study over a period of nine years, aiming to better understand this rapidly evolving CI landscape. By analysing the development history of 91,810 GitHub repositories of active npm packages having used at least one CI service, we quantitatively studied the evolution of seven popular CIs, specifically focusing on their co-usage and migration in the considered repositories. We provide statistical evidence of the rise of GitHub Actions, that has become the dominant CI service in less than 18 months time. This coincides with the fall of Travis that has seen an important decrease in usage, likely due to a combination of policy changes and migrations to GitHub Actions.","PeriodicalId":437520,"journal":{"name":"2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133268498","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"iTest: Using coverage measurements to improve test efficiency","authors":"Stefan Fischer, Denise Rigoni, N. Obrenović","doi":"10.1109/saner53432.2022.00133","DOIUrl":"https://doi.org/10.1109/saner53432.2022.00133","url":null,"abstract":"Many testing tasks in practice are still done manually. Which test cases are developed, automated and selected for execution is done by experience and instinct, rather than facts and data from the current system. To change this automated tool support requires lots of different data from the system under test. One very important information is what parts of the system are reached by a given test. This coverage information can be used to select, prioritize, or remove tests for the tested system revision. In this paper, we discuss our work on recording code coverage for individual system level tests. The discussed approach has shown to work in industry and is already in use. Nonetheless, we still have several open questions that require answers to improve coverage recordings. We highlight the lessons learned from our ongoing work and discuss the open questions to encourage further research on these problems.","PeriodicalId":437520,"journal":{"name":"2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116381423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Enrique A. da Roza, Jackson A. Prado Lima, Rogério C. Silva, S. Vergilio
{"title":"Machine Learning Regression Techniques for Test Case Prioritization in Continuous Integration Environment","authors":"Enrique A. da Roza, Jackson A. Prado Lima, Rogério C. Silva, S. Vergilio","doi":"10.1109/saner53432.2022.00034","DOIUrl":"https://doi.org/10.1109/saner53432.2022.00034","url":null,"abstract":"Test Case Prioritization (TCP) techniques are a key factor in reducing the regression testing costs even more when Continuous Integration (CI) practices are adopted. TCP approaches based on failure history have been adopted in this context because they are more suitable for CI environment constraints: test budget and test case volatility, that is, test cases may be added or removed over the CI cycles. Promising approaches are based on Reinforcement Learning (RL), which learns with past prioritization, guided by a reward function. In this work, we introduce a TCP approach for CI environments based on the sliding window method, which can be instantiated with different Machine Learning (ML) algorithms. Unlike other ML approaches, it does not require retraining the model to perform the prioritization and any code analysis. As an alternative for the RL approaches, we apply the Random Forest (RF) algorithm and a Long Short Term Memory (LSTM) deep learning network in our evaluation. We use three time budgets and eleven systems. The results show the applicability of the approach considering the prioritization time and the time between the CI cycles. Both algorithms take just a few seconds to execute. The RF algorithm obtained the best performance for more restrictive budgets compared to the RL approaches described in the literature. Considering all systems and budgets, RF reaches Normalized Average Percentage of Faults Detected (NAPFD) values that are the best or statistically equivalent to the best ones in around 72% of the cases, and the LSTM network in 55% of them. Moreover, we discuss some implications of our results for the usage of the algorithms evaluated.","PeriodicalId":437520,"journal":{"name":"2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","volume":"84 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116558765","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Alejandra Duque-Torres, Dietmar Pfahl, Claus Klammer, Stefan Fischer
{"title":"Using Source Code Metrics for Predicting Metamorphic Relations at Method Level","authors":"Alejandra Duque-Torres, Dietmar Pfahl, Claus Klammer, Stefan Fischer","doi":"10.1109/SANER53432.2022.0013","DOIUrl":"https://doi.org/10.1109/SANER53432.2022.0013","url":null,"abstract":"Metamorphic testing (TM) examines the relations between inputs and outputs of test runs. These relations are known as metamorphic relations (MR). Currently, MRs are handpicked and require in-depth knowledge of the System Under Test (SUT), as well as its problem domain. As a result, the identification and selection of high-quality MRs is a challenge. Kanewala et al. suggested the Predicting Metamorphic Relations (PMR) approach for automatic prediction of applicable MRs picked from a predefined list. PMR is based on a Support Vector Machine (SVM) model using features derived from the Control Flow Graphs (CFGs) of 100 Java methods. The original study of Kanewala et al. showed encouraging results, but developing classification models from CFG-related features is costly. In this paper, we aim at developing a PMR approach that is less costly without losing performance. We complement the original PMR approach by considering other than CFG-related features. We define 21 features that can be directly extracted from source code and build several classifiers, including SVM models. Our results indicate that using the original CFG-based method-level features, in particular for a SVM with random walk kernel (RWK), achieve better predictions in terms of AUC-ROC for most of the candidate MRs than our models. However, for one of the candidate MRs, using source code features achieved the best AUC-ROC result (greater than 0.8).","PeriodicalId":437520,"journal":{"name":"2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121160386","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Accessing the Presentation of Causal Graphs and an Application of Gestalt Principles with Eye Tracking","authors":"Lisa Grabinger, Florian Hauser, J. Mottok","doi":"10.1109/saner53432.2022.00153","DOIUrl":"https://doi.org/10.1109/saner53432.2022.00153","url":null,"abstract":"The discipline of causal inference uses so-called causal graphs to model cause and effect relations of random variables. As those graphs only encode a relation structure there is no hard rule concerning their alignment. The present paper presents a study with the aim of working out the optimal alignment of causal graphs with respect to comprehensibility and interestingness. In addition, the study examines whether the central gestalt principles of psychology apply for causal graphs. Data from 29 participants is acquired by triangulating eye tracking with a questionnaire. The results of the study suggest that causal graphs should be aligned downwards. Moreover, the gestalt principles proximity, similarity and closure are shown to hold true for causal graphs.","PeriodicalId":437520,"journal":{"name":"2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122320203","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sergio Moreschini, Francesco Lomio, David Hästbacka, D. Taibi
{"title":"MLOps for evolvable AI intensive software systems","authors":"Sergio Moreschini, Francesco Lomio, David Hästbacka, D. Taibi","doi":"10.1109/saner53432.2022.00155","DOIUrl":"https://doi.org/10.1109/saner53432.2022.00155","url":null,"abstract":"DevOps practices are the de facto sandard when developing software. The increased adoption of machine learning (ML) to solve problems urges us to adapt all the current approaches to developing a new standard that can take full benefit from the new solution. In this work we propose a graphical representation for DevOps for ML-based applications, namely MLOps, and also outline open research challenges. The pipeline aims to get the best of both worlds by maintaining the simple and iconic pipeline of DevOps, yet improving it by adding new circular steps for ML incorporation. This aims to create an ML-based development subsystem that can be self-maintained, and is capable of evolving side-by-side with the software development.","PeriodicalId":437520,"journal":{"name":"2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122817859","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Message from the General Chair and Program Co-Chairs SANER 2022","authors":"","doi":"10.1109/saner53432.2022.00006","DOIUrl":"https://doi.org/10.1109/saner53432.2022.00006","url":null,"abstract":"","PeriodicalId":437520,"journal":{"name":"2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126527791","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}