S. Bauersfeld, T. Vos, Nelly Condori-Fernández, A. Bagnato, E. Brosse
{"title":"Evaluating the TESTAR tool in an industrial case study","authors":"S. Bauersfeld, T. Vos, Nelly Condori-Fernández, A. Bagnato, E. Brosse","doi":"10.1145/2652524.2652588","DOIUrl":"https://doi.org/10.1145/2652524.2652588","url":null,"abstract":"[Context] Automated test case design and execution at the GUI level of applications is not a fact in industrial practice. Tests are still mainly designed and executed manually. In previous work we have described TESTAR, a tool which allows to set-up fully automatic testing at the GUI level of applications to find severe faults such as crashes or non-responsiveness. [Method] This paper aims at the evaluation of TESTAR with an industrial case study. The case study was conducted at SOFTEAM, a French software company, while testing their Modelio SaaS system, a cloud-based system to manage virtual machines that run their popular graphical UML editor Modelio. [Goal] The goal of the study was to evaluate how the tool would perform within the context of SOFTEAM and on their software application. On the other hand, we were interested to see how easy or difficult it is to learn and implant our academic prototype within an industrial setting. [Results] The effectiveness and efficiency of the automated tests generated with TESTAR can definitely compete with that of the manual test suite. [Conclusions] The training materials as well as the user and installation manual of TESTAR need to be improved using the feedback received during the study. Finally, the need to program Java-code to create sophisticated oracles for testing created some initial problems and some resistance. However, it became clear that this could be solved by explaining the need for these oracles and compare them to the alternative of more expensive and complex human oracles. The need to raise consciousness that automated testing means programming solved most of the initial problems.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"118 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116351508","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"VCS-analyzer for software evolution empirical analysis","authors":"F. Fontana, Matteo Rolla, M. Zanoni","doi":"10.1145/2652524.2652599","DOIUrl":"https://doi.org/10.1145/2652524.2652599","url":null,"abstract":"Version Control Systems (VCSs) provide historical information that can be used to perform deep analyses on the evolution of a software project, with the aim of enhancing the quality of the system and predicting software evolution.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127072577","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. Mello, Pedro Correa da Silva, P. Runeson, G. Travassos
{"title":"Towards a framework to support large scale sampling in software engineering surveys","authors":"R. Mello, Pedro Correa da Silva, P. Runeson, G. Travassos","doi":"10.1145/2652524.2652567","DOIUrl":"https://doi.org/10.1145/2652524.2652567","url":null,"abstract":"Context: The low quality and small size of samples in empirical studies in software engineering hamper the interpretation and generalization of their results. Therefore, enlarging sample sizes and improving their quality represent an important research challenge. Goal: We aim to define a conceptual framework, including requirements for establishing adequate sources for sampling subjects in software engineering surveys. Method: We use previous experience on applying systematic sampling strategies combined with contemporary web technologies in previously executed surveys, to organize the conceptual framework. We analyze its application to different sources of sampling. Results: The framework was observed to be feasible after its application to nine different large-scale sources of sampling. Conclusions: The analyzed crowdsourcing tools do not support essential requirements to be considered sources of sampling, while free-lancing tools and professional social network do.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"49 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129136962","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Markus Borg, P. Runeson, Jens Johansson, M. Mäntylä
{"title":"A replicated study on duplicate detection: using apache lucene to search among Android defects","authors":"Markus Borg, P. Runeson, Jens Johansson, M. Mäntylä","doi":"10.1145/2652524.2652556","DOIUrl":"https://doi.org/10.1145/2652524.2652556","url":null,"abstract":"Context: Duplicate detection is a fundamental part of issue management. Systems able to predict whether a new defect report will be closed as a duplicate, may decrease costs by limiting rework and collecting related pieces of information. Goal: Our work explores using Apache Lucene for large-scale duplicate detection based on textual content. Also, we evaluate the previous claim that results are improved if the title is weighted as more important than the description. Method: We conduct a conceptual replication of a well-cited study conducted at Sony Ericsson, using Lucene for searching in the public Android defect repository. In line with the original study, we explore how varying the weighting of the title and the description affects the accuracy. Results: We show that Lucene obtains the best results when the defect report title is weighted three times higher than the description, a bigger difference than has been previously acknowledged. Conclusions: Our work shows the potential of using Lucene as a scalable solution for duplicate detection.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134309522","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
S. Biffl, Marcos Kalinowski, F. Ekaputra, Amadeu Anderlin Neto, T. Conte, D. Winkler
{"title":"Towards a semantic knowledge base on threats to validity and control actions in controlled experiments","authors":"S. Biffl, Marcos Kalinowski, F. Ekaputra, Amadeu Anderlin Neto, T. Conte, D. Winkler","doi":"10.1145/2652524.2652568","DOIUrl":"https://doi.org/10.1145/2652524.2652568","url":null,"abstract":"[Context] Experiment planners need to be aware of relevant Threats to Validity (TTVs), so they can devise effective control actions or accept the risk. [Objective] The aim of this paper is to introduce a TTV knowledge base (KB) that supports experiment planners in identifying relevant TTVs in their research context and actions to control these TTVs. [Method] We identified requirements, designed and populated a TTV KB with data extracted during a systematic review: 63 TTVs and 149 control actions from 206 peer-reviewed published software engineering experiments. We conducted an initial proof of concept on the feasibility of using the TTV KB and analyzed its content. [Results] The proof of concept and content analysis provided indications that experiment planners can benefit from an extensible TTV KB for identifying relevant TTVs and control actions in their specific context. [Conclusions] The TTV KB should be further evaluated and evolved in a variety of software engineering contexts.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"254 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114029524","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
N. B. Moe, Darja Šmite, Aivars Sablis, Anne-Lie Börjesson, Pia Andréasson
{"title":"Networking in a large-scale distributed agile project","authors":"N. B. Moe, Darja Šmite, Aivars Sablis, Anne-Lie Börjesson, Pia Andréasson","doi":"10.1145/2652524.2652584","DOIUrl":"https://doi.org/10.1145/2652524.2652584","url":null,"abstract":"Context: In large-scale distributed software projects the expertise may be scattered across multiple locations.\u0000 Goal: We describe and discuss a large-scale distributed agile project at Ericsson, a multinational telecommunications company headquartered in Sweden. The project is distributed across four development locations (one in Sweden, one in Korea and two in China) and employs 17 teams. In such a large scale environment the challenge is to have as few dependences between teams as possible, which is one reason why Ericsson introduced cross-functional feature teams -- teams that are capable of taking the full responsibility for implementing one entire feature. To support such teams when solving problems, ensure knowledge sharing within the project and safeguard the quality Ericsson introduced a new role -- Technical Area Responsible (TAR).\u0000 Method: We conducted extensive fieldwork for 9 months at two Ericsson sites in Sweden and China. We interviewed representatives from different roles in the organization, in addition to focus groups and a survey with seven teams.\u0000 Results: We describe the TAR role, and how the TARs communicate, coordinate and support the teams. Also architects support the teams, however not as closely as the TARs. We found that the TAR is usually a senior developer working halftime or fulltime in the role. We also present measures of the actual knowledge network of three Chinese and three Swedish teams and the TARs position in it.\u0000 Conclusions: TARs are central in the knowledge network and act as the boundary spanners between the teams and between the sites. We learned that availability of the TARs across sites is lower than that with local TARs. We also found that the size of a team's knowledge network depends on how long the team members have been working in the company. Finally we discuss the advantages and the challenges of introducing experts in key roles in large scale distributed agile development.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121051251","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evolution of design patterns: a replication study","authors":"B. Rossi, B. Russo","doi":"10.1145/2652524.2652563","DOIUrl":"https://doi.org/10.1145/2652524.2652563","url":null,"abstract":"Context. In 2007, Aversano et al. [2] analysed the evolution of JHotDraw, ArgoUML, and Eclipse JDT between years 2000-2005 to understand the role of frequently changed design patterns. Goal. In this paper, we perform a replication of the study on more recent versions to control for artifactual results. In particular, we investigate whether maturity of software versions can affect the original results. Method. We perform a re-analysis of the original data to learn and correctly deploy the tools used for data collection and analysis and to control instrumental threats that typically affect a replication. Results/Conclusions. Findings confirm that patterns change more frequently when they play a crucial role in the software and when in newer releases they support more advanced features.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121396862","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"What do game developers test in their products?","authors":"Jussi Kasurinen, K. Smolander","doi":"10.1145/2652524.2652525","DOIUrl":"https://doi.org/10.1145/2652524.2652525","url":null,"abstract":"Context: Software projects often have four objectives; to produce the required functionality, in budget and in schedule, with acceptable quality. That statement may be true for most of the ordinary software development projects, but are these objectives enough for game development, where creativity and artistic aspects have a major role? Goal: We analyze how game developing organizations test their products, what are their main test objectives and how they perceive themselves in the software business. Method: We interviewed seven game development teams from different companies and studied how they test their products with grounded theory approach. Results: Our results suggest that game developers focus on soft values such as game content or user experience, instead of more traditional objectives such as reliability or efficiency. Conclusions: Game developers have similar, but not fully comparable to software industry, set of priorities in their software testing and quality assurance approaches.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117323088","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Engineering of quality requirements as perceived by near-shore development centers' architects in eastern Europe: the hole in the whole","authors":"M. Daneva, S. Marczak, A. Herrmann","doi":"10.1145/2652524.2652534","DOIUrl":"https://doi.org/10.1145/2652524.2652534","url":null,"abstract":"Context: To software architects (SAs), the quality requirements (QRs) to a software system are key to designing the software architecture. However, understanding SAs' roles in the QRs engineering activities only recently became a topic in empirical requirements engineering research and very little is still known about QRs engineering from SAs' in large and distributed projects. Goal: This exploratory study aims at explicating how SAs are involved in engineering QRs in a specific distributed development setting, namely in organizations that distribute software development activities to closely located business units, known as near-shore development centres (NDCs), and in a specific geographic zone, namely Eastern Europe. Method: Based on interviews with 16 practitioners working on large projects in NDCs, we explicate the participation and involvement of NDCs' architects in QRs tasks. Results: We found that SAs from NDCs (i) are actively involved in QRs documentation and validation, (ii) are relatively passive participants in QRs elicitation and prioritization, and (iii) are not at all involved in QRs negotiation. Perhaps, our most surprising finding is that NDCs may often have economic incentives to misalign with onshore QRs practices. Conclusions: We explicated QRs practices, compared them to previously published ones, and found implications for both researchers and practitioners. Though, our results are preliminary, as they are from an exploratory study.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114295051","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Industrial evaluation of the impact of quality-driven release planning","authors":"M. Felderer, Armin Beer, Jason Ho, G. Ruhe","doi":"10.1145/2652524.2652579","DOIUrl":"https://doi.org/10.1145/2652524.2652579","url":null,"abstract":"Context: Product release decisions are often made ad hoc or not relying on up-to-date information from systematic and analysis-driven process. Often, too much emphasis solely is put into functionality, thereby neglecting the different quality aspects being important for the success of the product. Objective: For a case study project, we quantitatively measure the impact of the analytical and systematic release planning approach Q-EVOLVE II. Method: As an explorative and retrospective case study, we perform optimized planning scenarios and compare them with the actual baseline scenario. Results: For the case study project in an Austrian public health insurance institution, we demonstrate that the analytical and systematic approach Q-EVOLVE II results in a (1) reduced release time, (2) more features implemented in the same time, and (3) better final quality through increased testing activities and less defects slipping through at the time of release. Conclusion: In the context of the studied institution, analytical and systematic planning of product releases considering quality criteria is superior to ad hoc planning ignoring quality.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114562567","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}