Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering最新文献

{"title":"Effort estimation in agile software development: a survey on the state of the practice","authors":"M. Usman, E. Mendes, J. Börstler","doi":"10.1145/2745802.2745813","DOIUrl":"https://doi.org/10.1145/2745802.2745813","url":null,"abstract":"Context: There are numerous studies on effort estimation in Agile Software Development (ASD) and the state of the art in this area has been recently documented in a Systematic Literature Review (SLR). However, to date there are no studies on the state of the practice in this area, focusing on similar issues to those investigated in the above-mentioned SLR. Objectives: The aim of this paper is to report on the state of the practice on effort estimation in ASD, focusing on a wide range of aspects such as the estimation techniques and effort predictors used, to name a few. Method: A survey was carried out using as instrument an on-line questionnaire answered by agile practitioners who have experience in effort estimation. Results: Data was collected from 60 agile practitioners from 16 different countries, and the main findings are: 1) Planning poker (63%), analogy (47%) and expert judgment (38%) are frequently practiced estimation techniques in ASD; 2) Story points is the most frequently (62%) employed size metric, used solo or in combination with other metrics (e.g., function points); 3) Team's expertise level and prior experience are most commonly used cost drivers; 4) 52% of the respondents believe that their effort estimates on average are under/over estimated by an error of 25% or more; 5) Most agile teams take into account implementation and testing activities during effort estimation; and 6) Estimation is mostly performed at sprint and release planning levels in ASD. Conclusions: Estimation techniques that rely on experts' subjective assessment are the ones used the most in ASD, with effort underestimation being the dominant trend. Further, the use of multiple techniques in combination and story points seem to present a positive association with estimation accuracy, and team-related cost drivers are the ones used by most agile teams. Finally, requirements and management related issues are perceived as the main reasons for inaccurate estimates.","PeriodicalId":324836,"journal":{"name":"Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering","volume":"75 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121470624","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"An empirical analysis of the utilization of multiple programming languages in open source projects","authors":"Philip Mayer, Alexander Bauer","doi":"10.1145/2745802.2745805","DOIUrl":"https://doi.org/10.1145/2745802.2745805","url":null,"abstract":"Background: Anecdotal evidence suggests that software applications are usually implemented using a combination of (programming) languages. Aim: We want to provide empirical evidence on the phenomenon of multi-language programming. Methods: We use data mining of 1150 open source projects selected for diversity from a public repository to a) investigate the projects for number and type of languages found and the relative sizes of the languages; b) report on associations between the number of languages found and the size, age, number of contributors, and number of commits of a project using a (Quasi-)Poisson regression model, and c) discuss concrete associations between the general-purpose languages and domain-specific languages found using frequent item set mining. Results: We found a) a mean number of 5 languages per project with a clearly dominant main general-purpose language and 5 often-used DSL types, b) a significant influence of the size, number of commits, and the main language on the number of languages as well as no significant influence of age and number of contributors, and c) three language ecosystems grouped around XML, Shell/Make, and HTML/CSS. Conclusions: Multi-language programming seems to be common in open-source projects and is a factor which must be dealt with in tooling and when assessing development and maintenance of such software systems.","PeriodicalId":324836,"journal":{"name":"Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126895445","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Customer satisfaction feedback in an IT outsourcing company: a case study on the insigma Hengtian company","authors":"Xin Xia, D. Lo, Jingfan Tang, Shanping Li","doi":"10.1145/2745802.2745834","DOIUrl":"https://doi.org/10.1145/2745802.2745834","url":null,"abstract":"To reduce budget and improve competitive power, some companies would outsource their information technology (IT) functions to a third-party company referred to as an IT outsourcing company. After an outsourcing company completes a project, it would collect feedback from the customer. Analyzing this feedback could help to further improve the service of the outsourcing company. To our best knowledge, there are limited studies on customer satisfaction feedback. In this paper, we perform a case study of customer satisfaction feedback in an IT outsourcing company. We analyze 3 years of customer satisfaction feedback reports in Insigma Hengtian, which is the one of the well-known IT outsourcing companies in China with more than 2,000 employees. Each report specifies the overall satisfaction of a customer, and feedback on factors that contribute to customer satisfaction including: business knowledge and technical skills, work attitude, communication skills, on-time delivery, cost-effectiveness, etc. In total, we investigate 8 factors which are related to customer satisfaction. Next, we build a logistic regression model and analyze the statistical significance and collinearity characteristics of the independent factors used to build the model. We find that among the 8 factors, business knowledge and technical skills, on-time delivery, and cost-effectiveness are the 3 most important factors, and all of them have positive impact to customer satisfaction.","PeriodicalId":324836,"journal":{"name":"Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering","volume":"91 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126169731","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Case consistency: a necessary data quality property for software engineering data sets","authors":"Passakorn Phannachitta, Akito Monden, J. Keung, Ken-ichi Matsumoto","doi":"10.1145/2745802.2745820","DOIUrl":"https://doi.org/10.1145/2745802.2745820","url":null,"abstract":"Data quality is an essential aspect in any empirical study, because the validity of models and/or analysis results derived from an empirical data is inherently influenced by its quality. In this empirical study, we focus on data consistency as a critical factor influencing the accuracy of prediction models in software engineering. We propose a software metric called Cases Inconsistency Level (CIL) for analyzing conflicts within software engineering data sets by leveraging probability statistics on project cases and counting the number of conflicting pairs. The result demonstrated that CIL is able to be used as a metric to identify either consistent data sets or inconsistent data sets, which are valuable for building robust prediction models. In addition to measuring the level of consistency, CIL is proved to be applicable to predict whether or not an effort model built from data set can achieve higher accuracy, an important indicator for empirical experiments in software engineering.","PeriodicalId":324836,"journal":{"name":"Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130878388","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Support mechanisms to conduct empirical studies in software engineering: a systematic mapping study","authors":"A. Borges, Waldemar Ferreira, E. Barreiros, Adauto Almeida, Liliane Fonseca, Eudis Teixeira, Diogo Silva, Aline Alencar, S. Soares","doi":"10.1145/2745802.2745823","DOIUrl":"https://doi.org/10.1145/2745802.2745823","url":null,"abstract":"Context: Empirical studies are gaining recognition in the Software Engineering (SE) research community, allowing improved quality of research and accelerating the adoption of new technologies in the software market. However, empirical studies in this area are still limited. In order to foster empirical research in SE, it is essential to understand the resources available to aid these studies. Goal: Identify support mechanisms (methodology, tool, guideline, process, etc.) used to conduct empirical studies in the Empirical Software Engineering (ESE) community. Method: We performed a systematic mapping study that included all full papers published at EASE, ESEM and ESEJ since their first editions. Were selected 891 studies between 1996 and 2013. Results: A total of 375 support mechanisms were identified. We provide the full list of mechanisms and the strategies that uses them. Despite this, we identified a high number of studies that do not cite any mechanism to support their empirical strategies: 433 studies (48%). Experiment is the strategy that has more resources to support their activities. And guideline was the most used type of mechanism. Moreover we observed that the most mechanisms used as reference to empirical studies are not specific to SE area. And some mechanisms were used only in specific activities of empirical research, such as statistical and qualitative data analysis. Experiment and case studies are the strategies most applied. Conclusions: The use of empirical methods in SE has increased over the years. Despite this, many studies did not apply these methods and do not cite any resource to guide their research. Therefore, the list of support mechanisms, where and how they were applied is a major asset to the SE community. Such asset can encourage empirical studies aiding the choice regarding which strategies and mechanisms to use in a research, as well as pointing out examples where they were used, mainly to novice researchers. We also identified new perspectives and gaps that foster other research for the improvement of empirical research in this area.","PeriodicalId":324836,"journal":{"name":"Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133311912","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Robust statistical methods: why, what and how: keynote","authors":"B. Kitchenham","doi":"10.1145/2745802.2747956","DOIUrl":"https://doi.org/10.1145/2745802.2747956","url":null,"abstract":"This keynote discusses the need for more robust statistical methods. For visualizing data I suggest using Kernel density plots rather than box plots. For parametric analysis, I propose more robust measures of central location such as trimmed means, which can support reliable tests of the differences between the central location of two or more samples. In addition, I also recommend non-parametric effect sizes such as Cliff's δ and Brunner and Munzel's p-hat that avoid some of the problems with rank-based non-parametric methods.","PeriodicalId":324836,"journal":{"name":"Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering","volume":"317 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123491354","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Applying clustering to analyze opinion diversity","authors":"Mohammad Mahdi Hassan, M. Blom","doi":"10.1145/2745802.2745809","DOIUrl":"https://doi.org/10.1145/2745802.2745809","url":null,"abstract":"In empirical software engineering research there is an increased use of questionnaires and surveys to collect information from practitioners. Typically, such data is then analyzed based on overall, descriptive statistics. Even though this can capture the general trends there is a risk that the opinions of different (minority) sub-groups are lost. Here we propose the use of clustering to segment the respondents so that a more detailed analysis can be achieved. Our findings suggest that it can give a better insight about the survey population and the participants' opinions. This partitioning approach can show more precisely the extent of opinion differences between different groups. This approach also gives an opportunity for the minorities to be heard. Through the process significant new findings may also be obtained. In our example study regarding the state of testing and requirement activities in industry, we found several significant groups that showed significant opinion differences from the overall conclusion.","PeriodicalId":324836,"journal":{"name":"Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124039132","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Understanding differences in process perspectives between developers and acquirers in off-the-shelf-based custom software projects undertaken in Indonesia","authors":"D. S. Kusumo, Ariadi Nugroho","doi":"10.1145/2745802.2745828","DOIUrl":"https://doi.org/10.1145/2745802.2745828","url":null,"abstract":"In Off-the-shelf (OTS)-based custom software projects there are common and different processes between software developers and acquirers. Better understanding of these differences may help to initiate reconciling perspectives between these stakeholders for managing the software processes. An online survey was conducted to explore the involvement of two main stakeholders in OTS-based software project, developer and acquirer, and also their roles in decision making in make vs. buy, and in OTS selection. The respondents were 35 software developers and 34 software acquirers of Indonesian background. Then, a follow-up study involving 4 respondents was conducted to investigate differences in process perspective between both stakeholders. Our study found that for common and unique processes, both stakeholders perceive to have high and medium involvement in their stakeholder's processes. Only for detailed software design and coding, acquirer involvements in OTS-based custom software development were low. This study also found that each stakeholder perceives either themselves or both to make final decision on make vs. buy. In OTS-selection, both stakeholders agreed that developers had higher involvement. Based on the follow-up study findings, we proposed a model comparing and discussing stakeholder involvements, roles and responsibilities to reconcile these differences. A real example from the follow-up study findings was given to demonstrate the proposed model.","PeriodicalId":324836,"journal":{"name":"Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering","volume":"307 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123762922","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The adoption of capture-recapture in software engineering: a systematic literature review","authors":"Gaoxuan Liu, Guoping Rong, He Zhang, Qi Shan","doi":"10.1145/2745802.2745816","DOIUrl":"https://doi.org/10.1145/2745802.2745816","url":null,"abstract":"Context: Capture-recapture method has long been adopted in software engineering as a relatively objective way for defect estimation. While many relevant studies have been carried out to evaluate various capture-recapture models and estimators, there still lacks common understanding on the adoption status of the method in software engineering. It is necessary to systematically collect empirical evidence of Capture-recapture adoption hence form necessary understanding on the method. Objective: This study aims to synthesize relevant primary studies on the adoption of capture-recapture method in software engineering, and try to identify possible gaps between the state-of-practice and the state-of-art so as to provide clues for future research. Method: By following the guidelines of Kitchenham, we conducted a Systematic Literature Review(SLR) on studies of the adoption of capture-recapture method in software engineering. Results: From 5 common digital libraries, we retrieved 506 published articles, among them 44 were identified as relevant primary studies. We identified 18 capture-recapture estimators under 4 basic models. Types of the currently existing studies as well as the relevant influencing factors to adoption of the capture-recapture method are also discussed. Conclusion: Results show that there are no conclusive decisions on the best capture-recapture models and estimators. Besides, the number of inspectors and their capability to detect defects as well as the difficulty to detect defects are most critical influencing factors. In addition, lacking of industrial application may be the major issue of current adoption status of capture-recapture method in software engineering.","PeriodicalId":324836,"journal":{"name":"Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114255348","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A controlled experiment for the empirical evaluation of safety analysis techniques for safety-critical software","authors":"Asim Abdulkhaleq, S. Wagner","doi":"10.1145/2745802.2745817","DOIUrl":"https://doi.org/10.1145/2745802.2745817","url":null,"abstract":"Context: Today's safety critical systems are increasingly reliant on software. Software becomes responsible for most of the critical functions of systems. Many different safety analysis techniques have been developed to identify hazards of systems. FTA and FMEA are most commonly used by safety analysts. Recently, STPA has been proposed with the goal to better cope with complex systems including software. Objective: This research aimed at comparing quantitatively these three safety analysis techniques with regard to their effectiveness, applicability, understandability, ease of use and efficiency in identifying software safety requirements at the system level. Method: We conducted a controlled experiment with 21 master and bachelor students applying these three techniques to three safety-critical systems: train door control, anti-lock braking and traffic collision and avoidance. Results: The results showed that there is no statistically significant difference between these techniques in terms of applicability, understandability and ease of use, but a significant difference in terms of effectiveness and efficiency is obtained. Conclusion: We conclude that STPA seems to be an effective method to identify software safety requirements at the system level. In particular, STPA addresses more different software safety requirements than the traditional techniques FTA and FMEA, but STPA needs more time to carry out by safety analysts with little or no prior experience.","PeriodicalId":324836,"journal":{"name":"Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114249136","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}