R. Ferenc, Z. Tóth, Gergely Ladányi, István Siket, T. Gyimóthy
{"title":"A Public Unified Bug Dataset for Java","authors":"R. Ferenc, Z. Tóth, Gergely Ladányi, István Siket, T. Gyimóthy","doi":"10.1145/3273934.3273936","DOIUrl":"https://doi.org/10.1145/3273934.3273936","url":null,"abstract":"Background: Bug datasets have been created and used by many researchers to build bug prediction models. Aims: In this work we collected existing public bug datasets and unified their contents. Method: We considered 5 public datasets which adhered to all of our criteria. We also downloaded the corresponding source code for each system in the datasets and performed their source code analysis to obtain a common set of source code metrics. This way we produced a unified bug dataset at class and file level that is suitable for further research (e.g. to be used in the building of new bug prediction models). Furthermore, we compared the metric definitions and values of the different bug datasets. Results: We found that (i) the same metric abbreviation can have different definitions or metrics calculated in the same way can have different names, (ii) in some cases different tools give different values even if the metric definitions coincide because (iii) one tool works on source code while the other calculates metrics on bytecode, or (iv) in several cases the downloaded source code contained more files which influenced the afferent metric values significantly. Conclusions: Apart from all these imprecisions, we think that having a common metric set can help in building better bug prediction models and deducing more general conclusions. We made the unified dataset publicly available for everyone. By using a public dataset as an input for different bug prediction related investigations, researchers can make their studies reproducible, thus able to be validated and verified.","PeriodicalId":425863,"journal":{"name":"Proceedings of the 14th International Conference on Predictive Models and Data Analytics in Software Engineering","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126469877","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Modeling Relationship between Post-Release Faults and Usage in Mobile Software","authors":"Tapajit Dey, A. Mockus","doi":"10.1145/3273934.3273941","DOIUrl":"https://doi.org/10.1145/3273934.3273941","url":null,"abstract":"Background: The way post-release usage of a software affects the number of faults experienced by users is scarcely explored due to the proprietary nature of such data. The commonly used quality measure of post-release faults may, therefore, reflect usage instead of the quality of the software development process. Aim: To determine how software faults and software use are related in a post-deployment scenario and, based on that, derive post-deployment quality measure that reflects developers' performance more accurately. Method: We analyze Google Analytics data counting daily new users, visits, time-on-site, visits per user, and release start date and duration for 169 releases of a complex communication application for Android OS. We utilize Linear Regression, Bayesian Network, and Random Forest models to explain the interrelationships and to derive release quality measure that is relatively stable with respect to variations in software usage. Results: We found the number of new users and release start date to be the determining factors for the number of exceptions, and found no direct link between the intensity and frequency of software usage and software faults. Furthermore, the relative increase in the number of crashes was found to be stably associated with a power of 1.3 relative increase in the number of new users. Based on the findings we propose a release quality measure: number of crashes per user for a release of the software, which was seen to be independent of any other usage variables, providing us with a usage independent measure of software quality. Conclusions: We expect our result and our proposed quality measure will help gauge release quality of a software more accurately and inspire further research in this area.","PeriodicalId":425863,"journal":{"name":"Proceedings of the 14th International Conference on Predictive Models and Data Analytics in Software Engineering","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121657871","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"How Effectively Is Defective Code Actually Tested?: An Analysis of JUnit Tests in Seven Open Source Systems","authors":"Jean Petrić, T. Hall, David Bowes","doi":"10.1145/3273934.3273939","DOIUrl":"https://doi.org/10.1145/3273934.3273939","url":null,"abstract":"Background: Newspaper headlines still regularly report latent software defects. Such defects have often evaded testing for many years. It remains difficult to identify how well a system has been tested. It also remains difficult to assess how successful at finding defects particular tests are. Coverage and mutation testing are frequently used to asses test effectiveness. We look more deeply at the performance of commonly used JUnit testing by assessing how much JUnit testing was done and how effective that testing was at detecting defects in seven open source systems. Aim: We aim to identify whether defective code has been effectively tested by JUnit tests as non-defective code. We also aim to identify the characteristics of JUnit tests that are related to identifying defects. Methodology: We first extract the defects from seven open source projects using the SZZ algorithm. We match those defects with JUnit tests to identify the proportion of defects that were covered by JUnit tests. We also do the same for non-defective code. We then use Principal Component Analysis and machine learning to investigate the characteristics of JUnit tests that were successful in identifying defects. Results: Our findings suggest that most of the open source systems we investigated are under-tested. On average over 66% of defective methods were not linked to any JUnit tests. We show that the number of methods touched by a JUnit test is strongly related to that test uncovering a defect. Conclusion: More JUnit tests need to be produced for the seven open source systems that we investigate. JUnit tests need to be relatively sophisticated, in particular they should touch more than just one method during the test.","PeriodicalId":425863,"journal":{"name":"Proceedings of the 14th International Conference on Predictive Models and Data Analytics in Software Engineering","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133053343","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mohammad D. Alahmadi, Jonathan Hassel, Biswas Parajuli, S. Haiduc, Piyush Kumar
{"title":"Accurately Predicting the Location of Code Fragments in Programming Video Tutorials Using Deep Learning","authors":"Mohammad D. Alahmadi, Jonathan Hassel, Biswas Parajuli, S. Haiduc, Piyush Kumar","doi":"10.1145/3273934.3273935","DOIUrl":"https://doi.org/10.1145/3273934.3273935","url":null,"abstract":"Background: Video programming tutorials are becoming a popular resource for developers looking for quick answers to a specific programming problem or trying to learn a programming topic in more depth. Since the most important source of information for developers in many such videos is source code, it is important to be able to accurately extract this code from the screen, such that developers can easily integrate it into their programs. Aims: Our main goal is to facilitate the accurate and noise-free extraction of code appearing in programming video tutorials. In particular, in this paper we aim to accurately predict the location of source code in video frames. This will allow for the dramatic reduction of noise when using extraction techniques such as Optical Character Recognition, which could otherwise extract a large amount of irrelevant text (e.g., text found in menu items, package hierarchy, etc.). Method: We propose an approach using a deep Convolutional Neural Network (CNN) to predict the bounding box of fully-visible code sections in video frames. To evaluate our approach, we collected a set of 150 Java programming tutorials, having more than 82K frames in total. A sample of 4,000 frames from these videos were then manually annotated with the code bounding box location and used as the ground truth in an experiment evaluating our approach. Results: The results of the evaluation show that our approach is able to successfully predict the code bounding box in a given frame with 92% accuracy. Conclusions: Our CNN-based approach is able to accurately predict the location of source code within the frames of programming video tutorials.","PeriodicalId":425863,"journal":{"name":"Proceedings of the 14th International Conference on Predictive Models and Data Analytics in Software Engineering","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126990089","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Giuseppe Destefanis, S. Qaderi, David Bowes, Jean Petrić, Marco Ortu
{"title":"A Longitudinal Study of Anti Micro Patterns in 113 versions of Tomcat","authors":"Giuseppe Destefanis, S. Qaderi, David Bowes, Jean Petrić, Marco Ortu","doi":"10.1145/3273934.3273945","DOIUrl":"https://doi.org/10.1145/3273934.3273945","url":null,"abstract":"Background: Micro patterns represent design decisions in code. They are similar to design patterns and can be detected automatically. These micro structures can be helpful in identifying portions of code which should be improved (anti-micro patterns), or other well-designed parts which need to be preserved. The concepts expressed in these design decisions are defined at class-level; therefore the primary goal is to detect and provide information related to a specific granularity level. Aim: this paper aims to present preliminary results about a longitudinal study performed on anti-micro pattern distributions over 113 versions of Tomcat. Method: we first extracted the micro patterns from the 113 versions of Tomcat, then found the percentage of classes matching each of the six anti-micro pattern considered for this analysis, and studied correlations among the obtained time series after testing for stationarity, randomness and seasonality. Results: results show that the time series are stationary, not random (except for Function Pointer), and that additional studied are needed for studying seasonality. Regarding correlations, only the Pool and Record time series presented a correlation of 0.69, while moderate correlation has been found between Function Pointer and Function Object (0.58) and between Cobol Like and Pool (0.44).","PeriodicalId":425863,"journal":{"name":"Proceedings of the 14th International Conference on Predictive Models and Data Analytics in Software Engineering","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128516771","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Cross-Version Defect Prediction using Cross-Project Defect Prediction Approaches: Does it work?","authors":"S. Amasaki","doi":"10.1145/3273934.3273938","DOIUrl":"https://doi.org/10.1145/3273934.3273938","url":null,"abstract":"Background: Specifying and removing defects before release deserve extra cost for the success of software projects. Long-running projects experience multiple releases, and it is a natural choice to adopt cross-version defect prediction (CVDP) that uses information from older versions. A past study shows that feeding multi older versions data may have a positive influence on the performance. The study also suggests that cross-project defect prediction (CPDP) may fit the situation but one CPDP approach was only examined. Aims: To investigate whether feeding multiple older versions data is effective for CVDP using CPDP approaches. The investigation also involves performance comparisons of the CPDP approaches under CVDP situation. Method: We chose a style of replication of the comparative study on CPDP approaches by Herbold et al. under CVDP situation. Results: Feeding multiple older versions had a positive effect for more than a half CPDP approaches. However, almost all of the CPDP approaches did not perform significantly better than a simple rule-based prediction. Although the best CPDP approach could work better than it and with-in project defect prediction, we found no effect of feeding multiple older versions for it. Conclusions: Feeding multiple older versions could improve CPDP approaches under CVDP situation. However, it did not work for the best CPDP approach in the study.","PeriodicalId":425863,"journal":{"name":"Proceedings of the 14th International Conference on Predictive Models and Data Analytics in Software Engineering","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131690630","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Francis Palma, T. Abdou, A. Bener, J. Maidens, Stella Liu
{"title":"An Improvement to Test Case Failure Prediction in the Context of Test Case Prioritization","authors":"Francis Palma, T. Abdou, A. Bener, J. Maidens, Stella Liu","doi":"10.1145/3273934.3273944","DOIUrl":"https://doi.org/10.1145/3273934.3273944","url":null,"abstract":"Aim: In this study, we aim to re-evaluate research questions on the ability of a logistic regression model proposed in a previous work to predict and prioritize the failing test cases based on some test quality metrics. Background: The process of prioritizing test cases aims to come up with a ranked test suite where test cases meeting certain criteria are prioritized. One criterion may be the ability of test cases to find faults that can be predicted a priori. Ranking test cases and executing the top-ranked test cases is particularly beneficial when projects have tight schedules and budgets. Method: We performed the comparison by first rebuilding the predictive models using the features from the original study and then we extended the original work to improve the predictive models using new features by combining with the existing ones. Results: The results of our study, using a dataset of five open-source systems, confirm that the findings from the original study hold and that our predictive models with new features outperform the original models in predicting and prioritizing the failing test cases. Conclusions: We plan to apply this method to a large-scale dataset from a large commercial enterprise project, to better demonstrate the improvement that our modified features provide and to explore the model's performance at scale.","PeriodicalId":425863,"journal":{"name":"Proceedings of the 14th International Conference on Predictive Models and Data Analytics in Software Engineering","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126896944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An Empirical Study of Metric-based Comparisons of Software Libraries","authors":"Fernando López de la Mora, Sarah Nadi","doi":"10.1145/3273934.3273937","DOIUrl":"https://doi.org/10.1145/3273934.3273937","url":null,"abstract":"BACKGROUND: Software libraries provide a set of reusable functionality, which helps developers write code in a systematic and timely manner. However, selecting the appropriate library to use is often not a trivial task. AIMS: In this paper, we investigate the usefulness of software metrics in helping developers choose libraries. Different developers care about different aspects of a library and two developers looking for a library in a given domain may not necessarily choose the same library. Thus, instead of directly recommending a library to use, we provide developers with a metric-based comparison of libraries in the same domain to empower them with the information they need to make an informed decision. METHOD: We use software data analytics from several sources of information to create quantifiable metric-based comparisons of software libraries. For evaluation, we select 34 open-source Java libraries from 10 popular domains and extract nine metrics related to these libraries. We then conduct a survey of 61 developers to evaluate whether our proposed metric-based comparison is useful, and to understand which metrics developers care about. RESULTS: Our results show that developers find that the proposed technique provides useful information when selecting libraries. We observe that developers care the most about metrics related to the popularity, security, and performance of libraries. We also find that the usefulness of some metrics may vary according to the domain. CONCLUSIONS: Our survey results showed that our proposed technique is useful. We are currently building a public website for metric-based library comparisons, while incorporating the feedback we obtained from our survey participants.","PeriodicalId":425863,"journal":{"name":"Proceedings of the 14th International Conference on Predictive Models and Data Analytics in Software Engineering","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126910912","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Are Software Dependency Supply Chain Metrics Useful in Predicting Change of Popularity of NPM Packages?","authors":"Tapajit Dey, A. Mockus","doi":"10.1145/3273934.3273942","DOIUrl":"https://doi.org/10.1145/3273934.3273942","url":null,"abstract":"Background: As software development becomes more interdependent, unique relationships among software packages arise and form complex software ecosystems. Aim: We aim to understand the behavior of these ecosystems better through the lens of software supply chains and model how the effects of software dependency network affect the change in downloads of Javascript packages. Method: We analyzed 12,999 popular packages in NPM, between 01-December-2017 and 15-March-2018, using Linear Regression and Random Forest models and examined the effects of predictors representing different aspects of the software dependency supply chain on changes in numbers of downloads for a package. Result: Preliminary results suggest that the count and downloads of upstream and downstream runtime dependencies have a strong effect on the change in downloads, with packages having fewer, more popular packages as dependencies (upstream or downstream) likely to see an increase in downloads. This suggests that in order to interpret the number of downloads for a package properly, it is necessary to take into account the peculiarities of the supply chain (both upstream and downstream) of that package. Conclusion: Future work is needed to identify the effects of added, deleted, and unchanged dependencies for different types of packages, e.g. build tools, test tools.","PeriodicalId":425863,"journal":{"name":"Proceedings of the 14th International Conference on Predictive Models and Data Analytics in Software Engineering","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115883497","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Manzano, E. Mendes, Cristina Gómez, Claudia P. Ayala, Xavier Franch
{"title":"Using Bayesian Networks to estimate Strategic Indicators in the context of Rapid Software Development","authors":"M. Manzano, E. Mendes, Cristina Gómez, Claudia P. Ayala, Xavier Franch","doi":"10.1145/3273934.3273940","DOIUrl":"https://doi.org/10.1145/3273934.3273940","url":null,"abstract":"Background: During Rapid Software Development, a large amount of project and development data can be collected from different and heterogeneous data sources. Aims: Design a methodology to process these data and turn it into relevant strategic indicators to help companies make meaningful decisions. Method: We adapt an existing methodology to create and estimate strategic indicators using Bayesian Networks in the context of Rapid Software Development, and applied it to a use case. Results: Applying the methodology in the use case, we create a model to predict product quality based on software factors and metrics, using companies' business knowledge and collected data. Conclusions: We proved the methodology's feasibility and obtained positive feedback from the company's use case.","PeriodicalId":425863,"journal":{"name":"Proceedings of the 14th International Conference on Predictive Models and Data Analytics in Software Engineering","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134635309","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}