Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering最新文献_第4页

Choosing the Right Test Automation Tool: a Grey Literature Review of Practitioner Sources 选择正确的测试自动化工具:实践者资源的灰色文献回顾

Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering Pub Date : 2017-06-15 DOI: 10.1145/3084226.3084252

Päivi Raulamo-Jurvanen, M. Mäntylä, V. Garousi

{"title":"Choosing the Right Test Automation Tool: a Grey Literature Review of Practitioner Sources","authors":"Päivi Raulamo-Jurvanen, M. Mäntylä, V. Garousi","doi":"10.1145/3084226.3084252","DOIUrl":"https://doi.org/10.1145/3084226.3084252","url":null,"abstract":"Background: Choosing the right software test automation tool is not trivial, and recent industrial surveys indicate lack of right tools as the main obstacle to test automation. Aim: In this paper, we study how practitioners tackle the problem of choosing the right test automation tool. Method: We synthesize the \"voice\" of the practitioners with a grey literature review originating from 53 different companies. The industry experts behind the sources had roles such as \"Software Test Automation Architect\", and \"Principal Software Engineer\". Results: Common consensus about the important criteria exists but those are not applied systematically. We summarize the scattered steps from individual sources by presenting a comprehensive process for tool evaluation with 12 steps and a total of 14 different criteria for choosing the right tool. Conclusions: The practitioners tend to have general interest in and be influenced by related grey literature as about 78% of our sources had at least 20 backlinks (a reference comparable to a citation) while the variation was between 3 and 759 backlinks. There is a plethora of different software testing tools available, yet the practitioners seem to prefer and adopt the widely known and used tools. The study helps to identify the potential pitfalls of existing processes and opportunities for comprehensive tool evaluation.","PeriodicalId":192290,"journal":{"name":"Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering","volume":"12 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124912054","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 44

A systematic review on the use of Definition of Done on agile software development projects 对在敏捷软件开发项目中使用“完成定义”的系统回顾

Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering Pub Date : 2017-06-15 DOI: 10.1145/3084226.3084262

Ana Silva, Thalles Araújo, João Nunes, M. Perkusich, E. Dilorenzo, H. Almeida, A. Perkusich

{"title":"A systematic review on the use of Definition of Done on agile software development projects","authors":"Ana Silva, Thalles Araújo, João Nunes, M. Perkusich, E. Dilorenzo, H. Almeida, A. Perkusich","doi":"10.1145/3084226.3084262","DOIUrl":"https://doi.org/10.1145/3084226.3084262","url":null,"abstract":"Background: Definition of Done (DoD) is a Scrum practice that consists of a simple list of criteria that adds verifiable or demonstrable value to the product. It is one of the most popular agile practices and assures a balance between short-term delivery of features and long-term product quality, but little is known of its actual use in Agile teams. Objective: To identify possible gaps in the literature and define a starting point to define DoD for practitioners through the identification and synthesis of the DoD criteria used in agile projects as presented in the scientific literature. Method: We applied a Systematic Literature Review of studies published up to (and including) 2016 through database search and backward and forward snowballing. Results: In total, we evaluated 2326 papers, of which 8 included DoD criteria used in agile projects. We identified that some studies presented up to 4 levels of DoD, which include story, sprint, release or project. We identified 62 done criteria, which are related to software verification and validation, deploy, code inspection, test process quality, regulatory compliance, software architecture design, process management, configuration management and non-functional requirements. Conclusion: The main implication for research is a need for more and better empirical studies documenting and evaluating the use of the DoD in agile software development. For the industry, the review provides a map of how DoD is currently being used in the industry and can be used as a starting point to define or compare with their own DoD definition.","PeriodicalId":192290,"journal":{"name":"Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121178638","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 27

Experience-based guidelines for effective and efficient data extraction in systematic reviews in software engineering 在软件工程的系统审查中有效和高效的数据提取的基于经验的指南

Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering Pub Date : 2017-06-15 DOI: 10.1145/3084226.3084238

V. Garousi, M. Felderer

{"title":"Experience-based guidelines for effective and efficient data extraction in systematic reviews in software engineering","authors":"V. Garousi, M. Felderer","doi":"10.1145/3084226.3084238","DOIUrl":"https://doi.org/10.1145/3084226.3084238","url":null,"abstract":"To systematically collect evidence and to structure a given area in software engineering (SE), Systematic Literature Reviews (SLR) and Systematic Mapping (SM) studies have become common. Data extraction is one of the main phases (activities) when conducting an SM or an SLR, whose objective is to extract required data from the primary studies and to accurately record the information researchers need to answer the questions of the SM/SLR study. Based on experience in a large number of SM/SLR studies, we and many other researchers have found the data extraction in SLRs to be time consuming and error-prone, thus raising the real need for heuristics and guidelines for effective and efficient data extraction in these studies, especially to be learnt by junior and young researchers. As a 'guideline' paper, this paper contributes a synthesized list of challenges usually faced during SLRs' data extraction phase and the corresponding solutions (guidelines). For our synthesis, we consider two data sources: (1) the pool of 16 SLR studies in which the authors have been involved in, as well as (2) a review of challenges and guidelines in the existing literature. Our experience in utilizing the presented guidelines in the near past have helped our junior colleagues to conduct data extractions more effectively and efficiently.","PeriodicalId":192290,"journal":{"name":"Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering","volume":"128 13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122090461","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 21

A Two-staged Survey on Release Readiness 发布准备的两阶段调查

Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering Pub Date : 2017-06-15 DOI: 10.1145/3084226.3084254

S. Alam, Maleknaz Nayebi, Dietmar Pfahl, G. Ruhe

{"title":"A Two-staged Survey on Release Readiness","authors":"S. Alam, Maleknaz Nayebi, Dietmar Pfahl, G. Ruhe","doi":"10.1145/3084226.3084254","DOIUrl":"https://doi.org/10.1145/3084226.3084254","url":null,"abstract":"Deciding about the content and readiness when shipping a new product release can have a strong impact on the success (or failure) of the product. Having formerly analyzed the state-of-the art in this area, the objective for this paper was to better understand the process and rationale of real-world release decisions and to what extent research on release readiness is aligned with industrial needs. We designed two rounds of surveys with focus on the current (Survey-A) and the desired (Survey-B) process of how to make release readiness decisions. We received 49 and 40 valid responses for Survey-A and Survey-B, respectively. In total, we identified 12 main findings related to the process, the rationale and the tool support considered for making release readiness decisions. We found that reasons for failed releases and the factors considered for making release decisions are context specific and vary with release cycle time. Practitioners confirmed that (i) release readiness should be measured and continuously monitored during the whole release cycle, (ii) release readiness decisions are context-specific and should not be based solely on quality considerations, and iii) some of the observed reasons for failed releases such as low functionality, high cost, and immature service are not adequately studied in research where there is dominance on investigating quality and testing only. In terms of requested tool support, dashboards covering multidimensional aspects of the status of release development were articulated as key requirements.","PeriodicalId":192290,"journal":{"name":"Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering","volume":"94 4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133821835","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Revisiting Assert Use in GitHub Projects 重新审视Assert在GitHub项目中的使用

Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering Pub Date : 2017-06-15 DOI: 10.1145/3084226.3084259

Pavneet Singh Kochhar, D. Lo

{"title":"Revisiting Assert Use in GitHub Projects","authors":"Pavneet Singh Kochhar, D. Lo","doi":"10.1145/3084226.3084259","DOIUrl":"https://doi.org/10.1145/3084226.3084259","url":null,"abstract":"Assertions are often used to test the assumptions that developers have about a program. An assertion contains a boolean expression which developers believe to be true at a particular program point. It throws an error if the expression is not satisfied, which helps developers to detect and correct bugs. Since assertions make developer assumptions explicit, assertions are also believed to improve under-standability of code. Recently, Casalnuovo et al. analyse C and C++ programs to understand the relationship between assertion usage and defect occurrence. Their results show that asserts have a small effect on reducing the density of bugs and developers often add asserts to methods they have prior knowledge of and larger ownership. In this study, we perform a partial replication of the above study on a large dataset of Java projects from GitHub (185 projects, 20 million LOC, 4 million commits, 0.2 million files and 1 million methods). We collect metrics such as number of asserts, number of defects, number of developers and number of lines changed to a method, and examine the relationship between asserts and defect occurrence. We also analyse relationship between developer experience and ownership and the number of asserts. Furthermore, we perform a study of what are different types of asserts added and why they are added by developers. We find that asserts have a small yet significant relationship with defect occurrence and developers who have added asserts to methods often have higher ownership of and experience with the methods than developers who did not add asserts.","PeriodicalId":192290,"journal":{"name":"Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering","volume":"150 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132323088","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 14

What Do We Know about Alignment of Requirements Engineering and Software Testing? 关于需求工程和软件测试的一致性我们知道些什么?

Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering Pub Date : 2017-06-15 DOI: 10.1145/3084226.3084265

Pertti Karhapää, Alireza Haghighatkhah, M. Oivo

{"title":"What Do We Know about Alignment of Requirements Engineering and Software Testing?","authors":"Pertti Karhapää, Alireza Haghighatkhah, M. Oivo","doi":"10.1145/3084226.3084265","DOIUrl":"https://doi.org/10.1145/3084226.3084265","url":null,"abstract":"Context: The alignment of different software engineering activities for coordinated functioning and optimized product development is of great importance, particularly in industrial-scale development. The link between intermediate activities has been researched extensively, but the link between requirements engineering (RE) and software testing (ST) is a relatively less explored area. Objective: The objective of this study is to aggregate, structure, and classify all existing research regarding alignment of RE and ST published by the end of 2015. Method: We conducted a systematic mapping study (SMS) and aggregated all studies relevant to our scope. The primary studies are analyzed in terms of publication trend, focus area, i.e., how alignment is supported, the application domain and benefits and challenges, methodological data, and scientific rigor and industrial relevance. Results: There is a growing interest towards the topic. Several different techniques have been identified to improve RE and ST alignment. Test generation from requirements specification has received most attention. Alignment of RE and ST is particularly important for large safety-critical domains. While many challenges have been reported, the supporting evidence for benefits is scarce. Frameworks/methods/techniques is the most frequent contribution type. Solution proposal and evaluation research were the most frequently applied research type. Case study research was the most frequently applied research method, however, almost half of the studies did not clearly report any research method. Conclusion: Despite the numerous approaches that are proposed, it is not clear what approach is suitable in what context and why. To support industry in RE and ST alignment, guidelines and tool support are needed. The supporting evidence for claimed benefits is very limited. Overall, the research area is in its early stages and an increase in both the number and rigor of empirical studies are required.","PeriodicalId":192290,"journal":{"name":"Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering","volume":"102 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133752076","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 6

Reporting Statistical Validity and Model Complexity in Machine Learning based Computational Studies 基于机器学习的计算研究报告统计有效性和模型复杂性

Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering Pub Date : 2017-06-15 DOI: 10.1145/3084226.3084283

B. Olorisade, P. Brereton, Péter András

{"title":"Reporting Statistical Validity and Model Complexity in Machine Learning based Computational Studies","authors":"B. Olorisade, P. Brereton, Péter András","doi":"10.1145/3084226.3084283","DOIUrl":"https://doi.org/10.1145/3084226.3084283","url":null,"abstract":"Background:: Statistical validity and model complexity are both important concepts to enhanced understanding and correctness assessment of computational models. However, information about these are often missing from publications applying machine learning. Aim: The aim of this study is to show the importance of providing details that can indicate statistical validity and complexity of models in publications. This is explored in the context of citation screening automation using machine learning techniques. Method: We built 15 Support Vector Machine (SVM) models, each developed using word2vec (average word) features --- and data for 15 review topics from the Drug Evaluation Review Program (DERP) of the Agency for Healthcare Research and Quality (AHRQ). Results: The word2vec features were found to be sufficiently linearly separable by the SVM and consequently we used the linear kernels. In 11 of the 15 models, the negative (majority) class used over 80% of its training data as support vectors (SVs) and approximately 45% of the positive training data. Conclusions: In this context, exploring the SVs revealed that the models are overly complex against ideal expectations of not more than 2%-5% (and preferably much less) of the training vectors.","PeriodicalId":192290,"journal":{"name":"Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120968407","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

The Links Between Agile Practices, Interpersonal Conflict, and Perceived Productivity 敏捷实践、人际冲突和感知生产力之间的联系

Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering Pub Date : 2017-06-15 DOI: 10.1145/3084226.3084269

Lucas Gren

{"title":"The Links Between Agile Practices, Interpersonal Conflict, and Perceived Productivity","authors":"Lucas Gren","doi":"10.1145/3084226.3084269","DOIUrl":"https://doi.org/10.1145/3084226.3084269","url":null,"abstract":"Agile processes explicitly focus more on team-work than more traditional management techniques when building software. With high velocity and responsiveness on team-level come the risk of interpersonal conflict in the agile organizations. Through a survey with 68 software developers from three large Swedish companies, I found that the presence of interpersonal conflict was negatively connected to the agile practices Iterative Development and Customer Access. The agile practices Iteration Planning and Iterative Development were positively linked to the measurement of the developers' perceived team productivity. However, Continuous Integration & Testing was negatively connected to productivity. These results show which agile practices are directly linked to team productivity, but also, and more importantly, indicate which of the agile practices that might be more prone to not work as intended, when the team struggles with interpersonal conflict. Therefore, I argue that members of agile teams need training in conflict resolution techniques in order to lower the risk of interpersonal conflict negatively affecting team productivity.","PeriodicalId":192290,"journal":{"name":"Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering","volume":"72 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127363007","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 18

On the Use of Ontologies in Software Process Assessment: A Systematic Literature Review 本体论在软件过程评估中的应用:系统文献综述

Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering Pub Date : 2017-06-15 DOI: 10.1145/3084226.3084261

Ayça Tarhan, G. Giray

{"title":"On the Use of Ontologies in Software Process Assessment: A Systematic Literature Review","authors":"Ayça Tarhan, G. Giray","doi":"10.1145/3084226.3084261","DOIUrl":"https://doi.org/10.1145/3084226.3084261","url":null,"abstract":"Software process assessment (SPA) is the foundation step for software process improvement. ISO/IEC 15504 defines the term process assessment as \"the systematic evaluation of an organization's processes against a process reference model (PRM)\". In process assessment, there is a need to set and maintain a mapping between an organization's processes and a PRM, where process experts transform the gap between the two into opportunities for process improvement. To maintain such a mapping requires a continuous tracking and alignment between the organization's processes and the PRM(s). The use of ontologies might be a suitable solution to provide computerized tool support for SPA that becomes erroneous and time-consuming if done manually. With an aim to understand the use and usefulness of ontologies in SPA, in this study, we have performed a systematic literature review (SLR). We have searched the most known digital libraries and selected 14 studies out of 54 initially selected and 571 initially retrieved. We analyzed the selected studies with respect to a number of research questions that address; contribution facet, targeted software processes, research facet, process improvement model used, process assessment model used, ontology representation language, purpose of ontology use, qualitative and quantitative benefits reported, and challenges faced. As a result, we synthesized a conceptual model of ontology-based support in SPA. We hope the results of our work will be useful for researchers and practitioners to direct their future studies on the use of ontologies for SPA.","PeriodicalId":192290,"journal":{"name":"Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering","volume":"2013 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127387470","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 22

Change Prediction through Coding Rules Violations 通过违反编码规则进行变更预测

Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering Pub Date : 2017-06-15 DOI: 10.1145/3084226.3084282

Irene Tollin, F. Fontana, M. Zanoni, Riccardo Roveda

引用次数: 20