{"title":"Requirement error abstraction and classification: an empirical study","authors":"G. Walia, Jeffrey C. Carver, T. Philip","doi":"10.1145/1159733.1159784","DOIUrl":"https://doi.org/10.1145/1159733.1159784","url":null,"abstract":"Software quality and reliability is a primary concern for successful development organizations. Monitoring and controlling quality by helping developers detect as many faults as possible is a subjective and intricate approach. Due to the inherent difficulties and limitations, additional methods are required to obtain a more complete solution to the software quality problem. This paper analyzes the software quality problem from a different perspective involving a step back from faults to focus on the fundamental causes of faults. The first step in this direction is the application of the Error Abstraction Process (EAP) to the requirements phase of the software lifecycle to develop a Requirement Error Taxonomy (RET). This paper presents an empirical study on the application of the EAP and RET to requirement documents in a controlled classroom setting. The results show that the EAP significantly improves the productivity of subjects, that the RET is useful for improving software quality, that it provides useful insights into the requirements document, and that various context variables also impact the results. These results are promising and are important to motivate further investigation, to refine the RET, and to derive more formalized tools and methods for assisting developers. The result of this investigation will be a sound verification process for requirements phase.","PeriodicalId":201305,"journal":{"name":"International Symposium on Empirical Software Engineering","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126818574","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A case study on the application of UML in legacy development","authors":"Bente Anda, K. Hansen","doi":"10.1145/1159733.1159754","DOIUrl":"https://doi.org/10.1145/1159733.1159754","url":null,"abstract":"Model-driven development with UML is becoming a de facto standard in industry, but although much of today's software development is about enhancing existing systems, there is no welldefined process for model-driven development in the context of legacy systems. To ensure the relevance of research on modeldriven development with UML, there is a need for studies of actual use of UML in software development companies. As part of a software process initiative, we conducted a case study in a large development project where some of the development teams enhanced existing components, while other teams developed software from scratch. The results from this case study showed that those who applied UML in modelling and enhancing legacy software experienced more challenges and fewer benefits from the use of UML than did the developers who modelled and developed new software. Overall our results show a need for better methodological support on applying UML in legacy development.","PeriodicalId":201305,"journal":{"name":"International Symposium on Empirical Software Engineering","volume":"82 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132724102","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Testing and inspecting reusable product line components: first empirical results","authors":"C. Denger, R. Kolb","doi":"10.1145/1159733.1159762","DOIUrl":"https://doi.org/10.1145/1159733.1159762","url":null,"abstract":"In recent years, product line development has increasingly received attention in industry as it enables software-developing organizations to reduce both cost and time of developing and maintaining increasingly complex systems as well as to address the demands for individually customized products. Successful product line development requires high quality of reusable artifacts in order to achieve the promised benefits. The unique issues of quality assurance in the context of systematic reuse, however, have not been quantitatively investigated so far. This paper describes a first empirical study comparing the two defect detection techniques, code inspections and functional testing, in the context of product line development. The primary goal of the study was to initially investigate the defect finding potential of the techniques on reusable software components with common and variant features. The major findings of the study are that the two techniques identified different types of defects on variants of a reusable component. Inspections are on average 66.39% more effective and need on average 36.84% less effort to detect a defect We found that both the testing and inspection techniques applied in the experiment were ineffective in identifying variant-specific defects. Overall, the results indicate that the standard quality assurance techniques seem to be insufficient to address special characteristics of reusable components.","PeriodicalId":201305,"journal":{"name":"International Symposium on Empirical Software Engineering","volume":"205 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131856272","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An empirical evaluation of a testing and debugging methodology for Excel","authors":"Jeffrey C. Carver, M. Fisher, G. Rothermel","doi":"10.1145/1159733.1159775","DOIUrl":"https://doi.org/10.1145/1159733.1159775","url":null,"abstract":"Spreadsheets are one of the most commonly used types of programs in the world, and it is important that they be sufficiently dependable. To help end users who create spreadsheets do so more reliably, we have created a testing and debugging methodology and environment for use in spreadsheets, known as the WYSIWYT methodology. Our prior experiments with WYSIWYT show that users can utilize it to ensure that their spreadsheets are more dependable, but these experiments to date have considered only an unfamiliar prototype spreadsheet environment, and have not involved spreadsheet creation tasks. In this work we conducted a controlled experiment that addresses these limitations. The results of this study indicate that the use of WYSIWYT did not affect the correctness of spreadsheets created by users, but it did significantly reduce the amount of effort required to create them. Further, the subjects' evaluation of the help provided by WYSIWYT was very positive. Our results provide several insights into the use of the WYSIWYT methodology by end users.","PeriodicalId":201305,"journal":{"name":"International Symposium on Empirical Software Engineering","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134424577","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Drivers for software refactoring decisions","authors":"M. Mäntylä, C. Lassenius","doi":"10.1145/1159733.1159778","DOIUrl":"https://doi.org/10.1145/1159733.1159778","url":null,"abstract":"This paper presents an empirical study of drivers for software refactoring decisions. We studied the refactoring decisions made by 37 students evaluating ten methods of a purposefully constructed Java program. The decision rationales reported by the evaluators were coded to identify the drivers behind the decisions. The identified drivers were categorized into Structure, Documentation, Visual Representation, and General drivers. The evaluators had conflicting opinions both regarding the internal quality of the methods and refactoring decisions. Complex code problems were detected only by experienced evaluators. Using regression analysis, we looked at the predictive value of drivers explaining the refactoring decisions. The most salient driver leading to a favourable refactoring decision was method size. This study provides information of the refactoring decisions and helps form a basis for creating code problem detectors. By comparing automatic detection and the identified drivers we gained understanding of code problems that are difficult or impossible to detect automatically, for example Poor Algorithm. Issues detected only by experienced developers, and code problems for which the human eye surpasses automatic detection indicate good areas for developer education.","PeriodicalId":201305,"journal":{"name":"International Symposium on Empirical Software Engineering","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130274380","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evaluating the practical use of different measurement scales in requirements prioritisation","authors":"L. Karlsson, Martin Höst, B. Regnell","doi":"10.1145/1159733.1159782","DOIUrl":"https://doi.org/10.1145/1159733.1159782","url":null,"abstract":"The importance of prioritising requirements is widely recognised. A number of different techniques for prioritising requirements have been proposed, some based on an ordinal scale, others on a ratio scale. Some measurement scales provide more information than others, i.e. the ratio scale is richer than the ordinal scale. This paper aims to investigate the differences between the scales used in prioritisation. This is important since techniques using a richer scale tend to be more time-consuming and complex to use. Thus, there is a trade-off between simple techniques only providing ranks and complex techniques providing information about the relative distance between requirements priorities. The paper suggests an approach to measure the skewness of the ratio distribution and a way to use the cost-value approach on ordinal scale data. Four different empirical data sets were used to verify the suggested approaches. The skewness measure seems feasible to determine in which cases the ratio scale is valuable. It indicates that some of our subjects tend to use the extreme values of the scale while others are more modest. The cost-value approach based on ordinal scale data also seems feasible. The requirements selection decisions based on ordinal scale data agree substantially with the decisions based on ratio scale data.","PeriodicalId":201305,"journal":{"name":"International Symposium on Empirical Software Engineering","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121060058","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Cross-company and single-company effort models using the ISBSG database: a further replicated study","authors":"C. Lokan, E. Mendes","doi":"10.1145/1159733.1159747","DOIUrl":"https://doi.org/10.1145/1159733.1159747","url":null,"abstract":"Five years ago the ISBSG database was used by Jeffery et al. [6] (S1) to compare the effort prediction accuracy between cross- and single-company effort models. Given that more than 2,000 projects were later volunteered to this database, in 2005 Mendes et al. [17] (S2) replicated S1 but obtained different results. The difference in results between both studies could have resulted from legitimate differences in data set patterns but also could have been influenced by differences in experimental procedure. S2 was unable to employ exactly the same experimental procedure used in S1, as S1's procedure was not fully documented. Therefore this paper aimed to apply S2's experimental procedure to the ISBSG database version used in S1 (release 6) to assess if differences in experimental procedure would have contributed towards different results. Our results corroborated those from S1: we found that predictions based on a single-company model were significantly more accurate than those based on a cross-company model.","PeriodicalId":201305,"journal":{"name":"International Symposium on Empirical Software Engineering","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127235253","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Fredrik Törner, Martin Ivarsson, Fredrik Pettersson, Peter Öhman
{"title":"Defects in automotive use cases","authors":"Fredrik Törner, Martin Ivarsson, Fredrik Pettersson, Peter Öhman","doi":"10.1145/1159733.1159753","DOIUrl":"https://doi.org/10.1145/1159733.1159753","url":null,"abstract":"This paper presents an empirical quality assessment of use cases with the purpose to provide defect data from industry. In the assessment, twelve criteria, based on earlier research, were applied to 43 use cases from Volvo Car Corporation developed according to current practice. The collected defect data were statistically analyzed to determine significant intensity differences and to establish a partial order between the defects types based on their intensity. In addition, a qualitative assessment was made to assess the effect of remaining defects on the established order. The study shows that the defect types with the highest defect intensity are Missing element and Incorrect linguistics. Further, the established defect order is the basis for a comparison between the criteria used and earlier research on guidelines and checklists for use case authoring, resulting in improvement propositions.","PeriodicalId":201305,"journal":{"name":"International Symposium on Empirical Software Engineering","volume":"172 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121778646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Common refactorings, a dependency graph and some code smells: an empirical study of Java OSS","authors":"S. Counsell, Y. Hassoun, G. Loizou, R. Najjar","doi":"10.1145/1159733.1159777","DOIUrl":"https://doi.org/10.1145/1159733.1159777","url":null,"abstract":"Refactoring, as a software engineering discipline has emerged over recent years to become an important aspect of maintaining software. Refactoring refers to the restructuring of software according to specific mechanics and principles. In this paper, we describe an analysis of the results from a tool whose purpose was to identify and extract refactorings from seven open-source Java systems. In particular, we analyzed the mechanics of the most commonly and least commonly applied refactorings to try and account for their frequency. Results showed the most common refactorings of the fifteen coined a 'Gang of Six', to be generally those with a high in-degree and low out-degree when mapped on a dependency graph; the same refactorings also featured strongly in the remedying of bad code smells. Remarkably and surprisingly, inheritance and encapsulationbased refactorings were found to have been applied relatively infrequently - we offer explanations for why this may be the case. The paper thus identifies 'core' refactorings central to many of the changes made by developers on open-source systems. While we can not guarantee that developers consciously undertake refactoring in any sense, the empirical results demonstrate that simple renaming and moving fields/methods between classes are common components of open-source system re-engineering. From a wider software engineering perspective, knowledge of what a modification will incur in likely sub-tasks is of value to developers whether working on open-source or other forms of software.","PeriodicalId":201305,"journal":{"name":"International Symposium on Empirical Software Engineering","volume":"184 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131717379","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An industrial case study of structural testing applied to safety-critical embedded software","authors":"J. Guan, A. Offutt, P. Ammann","doi":"10.1145/1159733.1159774","DOIUrl":"https://doi.org/10.1145/1159733.1159774","url":null,"abstract":"Effective testing of safety-critical real-time embedded software is difficult and expensive. Many companies are hesitant about the cost of formalized criteria-based testing and are not convinced of the benefits. This paper presents the results of an industrial case study that compared the normal testing at a company (manual functional testing) with testing based on the logic-based criterion of correlated active clause coverage (CACC). The evaluation was performed during the testing of embedded, real-time control software that has been deployed in a safety-critical application in the transportation industry. We found in our study that the test cases generated to satisfy the CACC criterion detected major safety-critical faults that were not detected by functional testing. We also found that the cost required for CACC testing was not necessarily higher than the cost of functional testing. There were also several faults that were found by the functional tests that were not found by CACC tests.","PeriodicalId":201305,"journal":{"name":"International Symposium on Empirical Software Engineering","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125737137","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}