{"title":"Local-based active classification of test report to assist crowdsourced testing","authors":"Junjie Wang, Song Wang, Qiang Cui, Qing Wang","doi":"10.1145/2970276.2970300","DOIUrl":"https://doi.org/10.1145/2970276.2970300","url":null,"abstract":"In crowdsourced testing, an important task is to identify the test reports that actually reveal fault - true fault, from the large number of test reports submitted by crowd workers. Most existing approaches towards this problem utilized supervised machine learning techniques, which often require users to manually label a large amount of training data. Such process is time-consuming and labor-intensive. Thus, reducing the onerous burden of manual labeling while still being able to achieve good performance is crucial. Active learning is one potential technique to address this challenge, which aims at training a good classifier with as few labeled data as possible. Nevertheless, our observation on real industrial data reveals that existing active learning approaches generate poor and unstable performances on crowdsourced testing data. We analyze the deep reason and find that the dataset has significant local biases. To address the above problems, we propose LOcal-based Active ClassiFication (LOAF) to classify true fault from crowdsourced test reports. LOAF recommends a small portion of instances which are most informative within local neighborhood, and asks user their labels, then learns classifiers based on local neighborhood. Our evaluation on 14,609 test reports of 34 commercial projects from one of the Chinese largest crowdsourced testing platforms shows that our proposed LOAF can generate promising results. In addition, its performance is even better than existing supervised learning approaches which built on large amounts of labelled historical data. Moreover, we also implement our approach and evaluate its usefulness using real-world case studies. The feedbacks from testers demonstrate its practical value.","PeriodicalId":230074,"journal":{"name":"2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE)","volume":"574 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123039963","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The power of probabilistic thinking","authors":"David S. Rosenblum","doi":"10.1145/2970276.2985780","DOIUrl":"https://doi.org/10.1145/2970276.2985780","url":null,"abstract":"Traditionally, software engineering has dealt in absolutes. For instance, we talk about a system being “correct” or “incorrect”, with the shades of grey in between occasionally acknowledged but rarely dealt with explicitly. And we typically employ logical, algebraic, relational and other representations and techniques that help us reason about software in such absolute terms. There of course have been notable exceptions to this, such as the use of statistical techniques in testing and debugging. But by and large, both researchers and practitioners have favored the relative comfort of an absolutist viewpoint in all aspects of development. In this talk, I will argue the benefits of taking a more thoroughly probabilistic approach in software engineering. Software engineering is rife with stochastic phenomena, and the vast majority of software systems operate in an environment of uncertain, random behavior, which suits an explicit probabilistic characterization. Furthermore, this uncertainty is becoming ever more pronounced in new software systems and platforms, such as the Internet of Things and autonomous vehicles, with their frequent imprecise outputs and heavy reliance on machine learning. To illustrate more deeply some of the considerations involved in taking a probabilistic approach, I will talk about some recent research I have been doing in probabilistic verification.","PeriodicalId":230074,"journal":{"name":"2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE)","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127176367","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Salome Maro, Anthony Anjorin, Rebekka Wohlrab, Jan-Philipp Steghöfer
{"title":"Traceability maintenance: Factors and guidelines","authors":"Salome Maro, Anthony Anjorin, Rebekka Wohlrab, Jan-Philipp Steghöfer","doi":"10.1145/2970276.2970314","DOIUrl":"https://doi.org/10.1145/2970276.2970314","url":null,"abstract":"Traceability is an important concern for numerous software engineering activities. Establishing traceability links is a challenging and cost-intensive task, which is uneconomical without suitable strategies for maintaining high link quality. Current approaches to Traceability Management (TM), however, often make important assumptions and choices without ensuring that the consequences and implications for trace-ability maintenance are feasible and desirable in practice. In this paper, therefore, we identify a set of core factors that influence how the quality of traceability links can be maintained. For each factor, we discuss relevant challenges and provide guidelines on how best to ensure viable traceability maintenance in a practical TM approach. Our guidelines are meant to be used by tool developers and users to select the most appropriate TM approach for their needs. Our results are based on and supported by data collected from interviews conducted with: (i) 9 of our industrial and academic project partners to elicit requirements for a TM tool, and (ii) 24 software development stakeholders from 15 industrial cases to provide a broader overview of the current state of the practice on TM. To evaluate the feasibility of our guidelines, we investigate a set of existing TM approaches used in industry with respect to our guidelines.","PeriodicalId":230074,"journal":{"name":"2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE)","volume":"79 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126191934","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Vesal Vojdani, Kalmer Apinis, Vootele Rõtov, H. Seidl, V. Vene, Ralf Vogler
{"title":"Static race detection for device drivers: The Goblint approach","authors":"Vesal Vojdani, Kalmer Apinis, Vootele Rõtov, H. Seidl, V. Vene, Ralf Vogler","doi":"10.1145/2970276.2970337","DOIUrl":"https://doi.org/10.1145/2970276.2970337","url":null,"abstract":"Device drivers rely on fine-grained locking to ensure safe access to shared data structures. For human testers, concurrency makes such code notoriously hard to debug; for automated reasoning, dynamically allocated memory and low-level pointer manipulation poses significant challenges. We present a flexible approach to data race analysis, implemented in the open source Goblint static analysis framework that combines different pointer and value analyses in order to handle a wide range of locking idioms, including locks allocated dynamically as well as locks stored in arrays. To the best of our knowledge, this is the most ambitious effort, having lasted well over ten years, to create a fully automated static race detection tool that can deal with most of the intricate locking schemes found in Linux device drivers. Our evaluation shows that these analyses are sufficiently precise, but practical use of these techniques requires inferring environmental and domain-specific assumptions.","PeriodicalId":230074,"journal":{"name":"2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE)","volume":"65 3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126770766","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Owolabi Legunsen, Wajih Ul Hassan, Xinyue Xu, Grigore Roşu, D. Marinov
{"title":"How good are the specs? A study of the bug-finding effectiveness of existing Java API specifications","authors":"Owolabi Legunsen, Wajih Ul Hassan, Xinyue Xu, Grigore Roşu, D. Marinov","doi":"10.1145/2970276.2970356","DOIUrl":"https://doi.org/10.1145/2970276.2970356","url":null,"abstract":"Runtime verification can be used to find bugs early, during software development, by monitoring test executions against formal specifications (specs). The quality of runtime verification depends on the quality of the specs. While previous research has produced many specs for the Java API, manually or through automatic mining, there has been no large-scale study of their bug-finding effectiveness. We present the first in-depth study of the bug-finding effectiveness of previously proposed specs. We used JavaMOP to monitor 182 manually written and 17 automatically mined specs against more than 18K manually written and 2.1M automatically generated tests in 200 open-source projects. The average runtime overhead was under 4.3x. We inspected 652 violations of manually written specs and (randomly sampled) 200 violations of automatically mined specs. We reported 95 bugs, out of which developers already fixed 74. However, most violations, 82.81% of 652 and 97.89% of 200, were false alarms. Our empirical results show that (1) runtime verification technology has matured enough to incur tolerable runtime overhead during testing, and (2) the existing API specifications can find many bugs that developers are willing to fix; however, (3) the false alarm rates are worrisome and suggest that substantial effort needs to be spent on engineering better specs and properly evaluating their effectiveness.","PeriodicalId":230074,"journal":{"name":"2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE)","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129537244","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sven Peldszus, Géza Kulcsár, Malte Lochau, Sandro Schulze
{"title":"Continuous detection of design flaws in evolving object-oriented programs using incremental multi-pattern matching","authors":"Sven Peldszus, Géza Kulcsár, Malte Lochau, Sandro Schulze","doi":"10.1145/2970276.2970338","DOIUrl":"https://doi.org/10.1145/2970276.2970338","url":null,"abstract":"Design flaws in object-oriented programs may seriously corrupt code quality thus increasing the risk for introducing subtle errors during software maintenance and evolution. Most recent approaches identify design flaws in an ad-hoc manner, either focusing on software metrics, locally restricted code smells, or on coarse-grained architectural anti-patterns. In this paper, we utilize an abstract program model capturing high-level object-oriented code entities, further augmented with qualitative and quantitative design-related information such as coupling/cohesion. Based on this model, we propose a comprehensive methodology for specifying object-oriented design flaws by means of compound rules integrating code metrics, code smells and anti-patterns in a modular way. This approach allows for efficient, automated design-flaw detection through incremental multi-pattern matching, by facilitating systematic information reuse among multiple detection rules as well as between subsequent detection runs on continuously evolving programs. Our tool implementation comprises well-known anti-patterns for Java programs. The results of our experimental evaluation show high detection precision, scalability to real-size programs, as well as a remarkable gain in efficiency due to information reuse.","PeriodicalId":230074,"journal":{"name":"2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE)","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130381724","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sebastian Proksch, Sven Amann, Sarah Nadi, M. Mezini
{"title":"Evaluating the evaluations of code recommender systems: A reality check","authors":"Sebastian Proksch, Sven Amann, Sarah Nadi, M. Mezini","doi":"10.1145/2970276.2970330","DOIUrl":"https://doi.org/10.1145/2970276.2970330","url":null,"abstract":"While researchers develop many new exciting code recommender systems, such as method-call completion, code-snippet completion, or code search, an accurate evaluation of such systems is always a challenge. We analyzed the current literature and found that most of the current evaluations rely on artificial queries extracted from released code, which begs the question: Do such evaluations reflect real-life usages? To answer this question, we capture 6,189 fine-grained development histories from real IDE interactions. We use them as a ground truth and extract 7,157 real queries for a specific method-call recommender system. We compare the results of such real queries with different artificial evaluation strategies and check several assumptions that are repeatedly used in research, but never empirically evaluated. We find that an evolving context that is often observed in practice has a major effect on the prediction quality of recommender systems, but is not commonly reflected in artificial evaluations.","PeriodicalId":230074,"journal":{"name":"2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE)","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125494269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Factoring requirement dependencies in software requirement selection using graphs and integer programming","authors":"D. Mougouei","doi":"10.1145/2970276.2975936","DOIUrl":"https://doi.org/10.1145/2970276.2975936","url":null,"abstract":"Software requirement selection is to find a subset of requirements (so-called optimal set) that gives the highest customer value for a release of software while keeping the cost within the budget. Several industrial studies however, have demonstrated that requirements of software projects are intricately interdependent and these interdependencies impact the values of requirements. Furthermore, the strengths of dependency relations among requirements vary in the context of real-world projects. For instance, requirements can be strongly or weakly interdependent. Therefore, it is important to consider both the existence and the strengths of dependency relations during requirement selection. The existing selection models however, have ignored either requirement dependencies altogether or the strengths of those dependencies. This research proposes an Integer programming model for requirement selection which considers both the existence and strengths of requirement dependencies. We further contribute a graph-based dependency modeling technique for capturing requirement dependencies and the their corresponding strengths. Automated/semi-automated techniques will also be devised to identify requirement dependencies and the strengths of those dependencies.","PeriodicalId":230074,"journal":{"name":"2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE)","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132373169","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Mining revision histories to detect cross-language clones without intermediates","authors":"Xiao Cheng, Zhiming Peng, Lingxiao Jiang, Hao Zhong, Haibo Yu, Jianjun Zhao","doi":"10.1145/2970276.2970363","DOIUrl":"https://doi.org/10.1145/2970276.2970363","url":null,"abstract":"To attract more users on different platforms, many projects release their versions in multiple programming languages (e.g., Java and C#). They typically have many code snippets that implement similar functionalities, i.e., cross-language clones. Programmers often need to track and modify cross-language clones consistently to maintain similar functionalities across different language implementations. In literature, researchers have proposed approaches to detect cross-language clones, mostly for languages that share a common intermediate language (such as the .NET language family) so that techniques for detecting single-language clones can be applied. As a result, those approaches cannot detect cross-language clones for many projects that are not implemented in a .NET language. To overcome the limitation, in this paper, we propose a novel approach, CLCMiner, that detects cross-language clones automatically without the need of an intermediate language. Our approach mines such clones from revision histories, which reflect how programmers maintain cross-language clones in practice. We have implemented a prototype tool for our approach and conducted an evaluation on five open source projects that have versions in Java and C#. The results show that CLCMiner achieves high accuracy and point to promising future work.","PeriodicalId":230074,"journal":{"name":"2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130175535","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Developer targeted analytics: Supporting software development decisions with runtime information","authors":"Jürgen Cito","doi":"10.1145/2970276.2975939","DOIUrl":"https://doi.org/10.1145/2970276.2975939","url":null,"abstract":"Runtime information of deployed software has been used by business and operations units to make informed decisions under the term “analytics”. However, decisions made by software engineers in the course of evolving software have, for the most part, been based on personal belief and gut-feeling. This could be attributed to software development being, for the longest time, viewed as an activity that is detached from the notion of operating software in a production environment. In recent years, this view has been challenged with the emergence of the DevOps movement, which aim is to promote cross-functional capabilities of development and operations activities within teams. This shift in process and mindset requires analytics tools that specifically target software developers. In this research, I investigate approaches to support developers in their decision-making by incorporating runtime information in source code and provide live feedback in IDEs by predicting the impact of code changes.","PeriodicalId":230074,"journal":{"name":"2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133561686","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}