{"title":"Exploiting the Unique Expression for Improved Sentiment Analysis in Software Engineering Text","authors":"Kexin Sun, Hui Gao, Hongyu Kuang, Xiaoxing Ma, Guoping Rong, Dong Shao, He Zhang","doi":"10.1109/ICPC52881.2021.00023","DOIUrl":"https://doi.org/10.1109/ICPC52881.2021.00023","url":null,"abstract":"Sentiment analysis on software engineering (SE) texts has been widely used in the SE research, such as evaluating app reviews or analyzing developers’ sentiments in commit messages. To better support the use of automated sentiment analysis for SE tasks, researchers built an SE-domain-specified sentiment dictionary to further improve the accuracy of the results. Unfortunately, recent work reported that current mainstream tools for sentiment analysis still cannot provide reliable results when analyzing the sentiments in SE texts. We suggest that the reason for this situation is because the way of expressing sentiments in SE texts is largely different from the way in social network or movie comments. In this paper, we propose to improve sentiment analysis in SE texts by using sentence structures, a different perspective from building a domain dictionary. Specifically, we use sentence structures to first identify whether the author is expressing her sentiment in a given clause of an SE text, and to further adjust the calculation of sentiments which are confirmed in the clause. An empirical evaluation based on four different datasets shows that our approach can outperform two dictionary-based baseline approaches, and is more generalizable compared to a learning-based baseline approach.","PeriodicalId":288873,"journal":{"name":"2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116539902","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Bruno Henrique Pachulski Camara, Marco Aurélio Graciotto Silva, A. T. Endo, S. Vergilio
{"title":"What is the Vocabulary of Flaky Tests? An Extended Replication","authors":"Bruno Henrique Pachulski Camara, Marco Aurélio Graciotto Silva, A. T. Endo, S. Vergilio","doi":"10.1109/ICPC52881.2021.00052","DOIUrl":"https://doi.org/10.1109/ICPC52881.2021.00052","url":null,"abstract":"Software systems have been continuously evolved and delivered with high quality due to the widespread adoption of automated tests. A recurring issue hurting this scenario is the presence of flaky tests, a test case that may pass or fail non-deterministically. A promising, but yet lacking more empirical evidence, approach is to collect static data of automated tests and use them to predict their flakiness. In this paper, we conducted an empirical study to assess the use of code identifiers to predict test flakiness. To do so, we first replicate most parts of the previous study of Pinto et al. (MSR 2020). This replication was extended by using a different ML Python platform (Scikit-learn) and adding different learning algorithms in the analyses. Then, we validated the performance of trained models using datasets with other flaky tests and from different projects. We successfully replicated the results of Pinto et al. (2020), with minor differences using Scikit-learn; different algorithms had performance similar to the ones used previously. Concerning the validation, we noticed that the recall of the trained models was smaller, and classifiers presented a varying range of decreases. This was observed in both intra-project and inter-projects test flakiness prediction.","PeriodicalId":288873,"journal":{"name":"2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115415168","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Comprehensive Integration of API Usage Patterns","authors":"Qi Shen, Shijun Wu, Yanzhen Zou, Bing Xie","doi":"10.1109/ICPC52881.2021.00017","DOIUrl":"https://doi.org/10.1109/ICPC52881.2021.00017","url":null,"abstract":"Nowadays, developers often reuse existing APIs to implement their programming tasks. A lot of API usage patterns are mined to help developers learn API usage rules. However, there are still many missing variables to be synthesized when developers integrate the patterns into their programming context. To deal with this issue, we propose a comprehensive approach to integrate API usage patterns in this paper. We first perform an empirical study by analyzing how API usage patterns are integrated in real-world projects. We find the expressions for variable synthesis is often non-trivial and can be divided into 5 syntax types. Based on the observation, we promote an approach to help developers interactively complete API usage patterns. Compared to the existing code completion techniques, our approach can recommend infrequent expressions accompanied with their real-world usage examples according to the user intent. The evaluation shows that our approach could assist users to integrate APIs more efficiently and complete the programming tasks faster than existing works.","PeriodicalId":288873,"journal":{"name":"2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC)","volume":"255 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115773590","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zengyang Li, Xiaoxiao Qi, Qinyi Yu, Peng Liang, Ran Mo, Chen Yang
{"title":"Multi-Programming-Language Commits in OSS: An Empirical Study on Apache Projects","authors":"Zengyang Li, Xiaoxiao Qi, Qinyi Yu, Peng Liang, Ran Mo, Chen Yang","doi":"10.1109/ICPC52881.2021.00029","DOIUrl":"https://doi.org/10.1109/ICPC52881.2021.00029","url":null,"abstract":"Modern software systems, such as Spark, are usually written in multiple programming languages (PLs). Besides benefiting from code reuse, such systems can also take advantages of specific PLs to implement certain features, to meet various quality needs, and to improve development efficiency. In this context, a change to such systems may need to modify source files written in different PLs. We define a multi-programming-language commit (MPLC) in a version control system (e.g., Git) as a commit that involves modified source files written in two or more PLs. To our knowledge, the phenomenon of MPLCs in software development has not been explored yet. In light of the potential impact of MPLCs on development difficulty and software quality, we performed an empirical study to understand the state of MPLCs, their change complexity, as well as their impact on open time of issues and bug proneness of source files in real-life software projects. By exploring the MPLCs in 20 non-trivial Apache projects with 205,994 commits, we obtained the following findings: (1) 9% of the commits from all the projects are MPLCs, and the proportion of MPLCs in 80% of the projects goes to a relatively stable level; (2) more than 90% of the MPLCs from all the projects involve source files written in two PLs; (3) the change complexity of MPLCs is significantly higher than that of non-MPLCs in all projects; (4) issues fixed in MPLCs take significantly longer to be resolved than issues fixed in non-MPLCs in 80% of the projects; and (5) source files that have been modified in MPLCs tend to be more bug-prone than source files that have never been modified in MPLCs. These findings provide practitioners with useful insights on the architecture design and quality management of software systems written in multiple PLs.","PeriodicalId":288873,"journal":{"name":"2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC)","volume":"87 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124930646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Quentin Perez, Pierre-Antoine Jean, C. Urtado, S. Vauttier
{"title":"Bug or not bug? That is the question","authors":"Quentin Perez, Pierre-Antoine Jean, C. Urtado, S. Vauttier","doi":"10.1109/ICPC52881.2021.00014","DOIUrl":"https://doi.org/10.1109/ICPC52881.2021.00014","url":null,"abstract":"Nowadays, development teams often rely on tools such as Jira or Bugzilla to manage backlogs of issues to be solved to develop or maintain software. Although they relate to many different concerns (e.g., bug fixing, new feature development, architecture refactoring), few means are proposed to identify and classify these different kinds of issues, except for non mandatory labels that can be manually associated to them. This may lead to a lack of issue classification or to issue misclassification that may impact automatic issue management (planning, assignment) or issue-derived metrics. Automatic issue classification thus is a relevant topic for assisting backlog management. This paper proposes a binary classification solution for discriminating bug from non bug issues. This solution combines natural language processing (TF-IDF) and classification (multi-layer perceptron) techniques, selected after comparing commonly used solutions to classify issues. Moreover, hyper-parameters of the neural network are optimized using a genetic algorithm. The obtained results, as compared to existing works on a commonly used benchmark, show significant improvements on the F1 measure for all datasets.","PeriodicalId":288873,"journal":{"name":"2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC)","volume":"58 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126642380","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"RefactorHub: A Commit Annotator for Refactoring","authors":"Ryo Kuramoto, M. Saeki, Shinpei Hayashi","doi":"10.1109/ICPC52881.2021.00058","DOIUrl":"https://doi.org/10.1109/ICPC52881.2021.00058","url":null,"abstract":"It is necessary to gather real refactoring instances while conducting empirical studies on refactoring. However, existing refactoring detection approaches are insufficient in terms of their accuracy and coverage. Reducing the manual effort of curating refactoring data is challenging in terms of obtaining various refactoring data accurately. This paper proposes a tool named RefactorHub, which supports users to manually annotate potential refactoring-related commits obtained from existing refactoring detection approaches to make their refactoring information more accurate and complete with rich details. In the proposed approach, the parameters of each refactoring operation are defined as a meaningful set of code elements in the versions before or after refactoring. RefactorHub provides interfaces and supporting features to annotate each parameter, such as the automated filling of dependent parameters, thereby avoiding wrong or uncertain selections. A preliminary user study showed that RefactorHub reduced annotation effort and improved the degree of agreement among users. Source code and demo video are available at https://github.com/salab/RefactorHub","PeriodicalId":288873,"journal":{"name":"2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132724961","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Shamsujjoha, J. Grundy, Li Li, Hourieh Khalajzadeh, Qinghua Lu
{"title":"Checking App Behavior Against App Descriptions: What If There are No App Descriptions?","authors":"M. Shamsujjoha, J. Grundy, Li Li, Hourieh Khalajzadeh, Qinghua Lu","doi":"10.1109/ICPC52881.2021.00050","DOIUrl":"https://doi.org/10.1109/ICPC52881.2021.00050","url":null,"abstract":"Classifying mobile apps based on their description is beneficial for several purposes. However, many app descriptions do not reflect app functionalities, whether accidentally or on purpose. Most importantly, these app classification methods do not work if the app description is unavailable. This paper investigates a Reverse Engineering-based Approach to Classify mobile apps using The data that exists in the app, called REACT. To validate the proposed REACT method, we use a large set of Android apps (24,652 apps in total). We also show REACTs’ extendibility for malware/anomaly detection and prove its reliability and scalability. However, our analysis shows some limitations in REACT procedure and implementation, especially for similar feature based app grouping. We discuss the root cause of these failures, our key lessons learned, and some future enhancement ideas. We also share our REACT tools and reproduced datasets for the app market analyst, mobile app developers and software engineering research communities for further research purposes.","PeriodicalId":288873,"journal":{"name":"2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC)","volume":"340 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133037294","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Project-Level Encoding for Neural Source Code Summarization of Subroutines","authors":"Aakash Bansal, S. Haque, Collin McMillan","doi":"10.1109/ICPC52881.2021.00032","DOIUrl":"https://doi.org/10.1109/ICPC52881.2021.00032","url":null,"abstract":"Source code summarization of a subroutine is the task of writing a short, natural language description of that subroutine. The description usually serves in documentation aimed at programmers, where even brief phrase (e.g. \"compresses data to a zip file\") can help readers rapidly comprehend what a subroutine does without resorting to reading the code itself. Techniques based on neural networks (and encoder-decoder model designs in particular) have established themselves as the state-of-the-art. Yet a problem widely recognized with these models is that they assume the information needed to create a summary is present within the code being summarized itself – an assumption which is at odds with program comprehension literature. Thus a current research frontier lies in the question of encoding source code context into neural models of summarization. In this paper, we present a project-level encoder to improve models of code summarization. By project-level, we mean that we create a vectorized representation of selected code files in a software project, and use that representation to augment the encoder of state-of-the-art neural code summarization techniques. We demonstrate how our encoder improves several existing models, and provide guidelines for maximizing improvement while controlling time and resource costs in model size.","PeriodicalId":288873,"journal":{"name":"2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129440684","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Shulin Zhou, Xiaodong Liu, Shanshan Li, Zhouyang Jia, Yuanliang Zhang, Teng Wang, Wang Li, Xiangke Liao
{"title":"ConfInLog: Leveraging Software Logs to Infer Configuration Constraints","authors":"Shulin Zhou, Xiaodong Liu, Shanshan Li, Zhouyang Jia, Yuanliang Zhang, Teng Wang, Wang Li, Xiangke Liao","doi":"10.1109/ICPC52881.2021.00018","DOIUrl":"https://doi.org/10.1109/ICPC52881.2021.00018","url":null,"abstract":"Misconfigurations have become the dominant causes of software failures in recent years, drawing tremendous attention for their increasing prevalence and severity. Configuration constraints can preemptively avoid misconfiguration by defining the conditions that configuration options should satisfy. Documentation is the main source of configuration constraints, but it might be incomplete or inconsistent with the source code. In this regard, prior researches have focused on obtaining configuration constraints from software source code through static analysis. However, the difficulty in pointer analysis and context comprehension prevents them from collecting accurate and comprehensive constraints. In this paper, we observed that software logs often contain configuration constraints. We conducted an empirical study and summarized patterns of configuration-related log messages. Guided by the study, we designed and implemented ConfInLog, a static tool to infer configuration constraints from log messages. ConfInLog first selects configuration-related log messages from source code by using the summarized patterns, then infers constraints from log messages based on the summarized natural language patterns. To evaluate the effectiveness of ConfInLog, we applied our tool on seven popular open-source software systems. ConfInLog successfully inferred 22∼163 constraints, in which 59.5%∼ 61.6% could not be inferred by the state-of-the-art work. Finally, we submitted 67 documentation patches regarding the constraints inferred by ConfInLog. The constraints in 29 patches have been confirmed by the developers, among which 10 patches have been accepted.","PeriodicalId":288873,"journal":{"name":"2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC)","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114754302","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Alejandro Mazuera-Rozo, Anamaria Mojica-Hanke, Mario Linares-V'asquez, G. Bavota
{"title":"Shallow or Deep? An Empirical Study on Detecting Vulnerabilities using Deep Learning","authors":"Alejandro Mazuera-Rozo, Anamaria Mojica-Hanke, Mario Linares-V'asquez, G. Bavota","doi":"10.1109/ICPC52881.2021.00034","DOIUrl":"https://doi.org/10.1109/ICPC52881.2021.00034","url":null,"abstract":"Deep learning (DL) techniques are on the rise in the software engineering research community. More and more approaches have been developed on top of DL models, also due to the unprecedented amount of software-related data that can be used to train these models. One of the recent applications of DL in the software engineering domain concerns the automatic detection of software vulnerabilities. While several DL models have been developed to approach this problem, there is still limited empirical evidence concerning their actual effectiveness especially when compared with shallow machine learning techniques. In this paper, we partially fill this gap by presenting a large-scale empirical study using three vulnerability datasets and five different source code representations (i.e., the format in which the code is provided to the classifiers to assess whether it is vulnerable or not) to compare the effectiveness of two widely used DL-based models and of one shallow machine learning model in (i) classifying code functions as vulnerable or non-vulnerable (i.e., binary classification), and (ii) classifying code functions based on the specific type of vulnerability they contain (or \"clean\", if no vulnerability is there). As a baseline we include in our study the AutoML utility provided by the Google Cloud Platform. Our results show that the experimented models are still far from ensuring reliable vulnerability detection, and that a shallow learning classifier represents a competitive baseline for the newest DL-based models.","PeriodicalId":288873,"journal":{"name":"2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115465781","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}