{"title":"Using public domain metrics to estimate software development effort","authors":"R. Jeffery, M. Ruhe, I. Wieczorek","doi":"10.1109/METRIC.2001.915512","DOIUrl":"https://doi.org/10.1109/METRIC.2001.915512","url":null,"abstract":"The authors investigate the accuracy of cost estimates when applying most commonly used modeling techniques to a large-scale industrial data set which is professionally maintained by the International Software Standards Benchmarking Group (ISBSG). The modeling techniques applied are ordinary least squares regression (OLS), analogy based estimation, stepwise ANOVA, CART, and robust regression. The questions addresses in the study are related to important issues. The first is the appropriate selection of a technique in a given context. The second is the assessment of the feasibility of using multi-organizational data compared to the benefits from company-specific data collection. We compare company-specific models with models based on multi-company data. This is done by using the estimates derived for one company that contributed to the ISBSG data set and estimates from using carefully matched data from the rest of the ISBSG data. When using the ISBSG data set to derive estimates for the company, generally poor results were obtained. Robust regression and OLS performed most accurately. When using the company's own data as the basis for estimation, OLS, a CART-variant, and analogy performed best. In contrast to previous studies, the estimation accuracy when using the company's data is significantly higher than when using the rest of the ISBSG data set. Thus, from these results, the company that contributed to the ISBSG data set, would be better off when using its own data for cost estimation.","PeriodicalId":180205,"journal":{"name":"Proceedings Seventh International Software Metrics Symposium","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2001-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125581485","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Investigation of logistic regression as a discriminant of software quality","authors":"N. Schneidewind","doi":"10.1109/METRIC.2001.915540","DOIUrl":"https://doi.org/10.1109/METRIC.2001.915540","url":null,"abstract":"Investigates the possibility that logistic regression functions (LRFs), when used in combination with Boolean discriminant functions (BDFs), which we had previously developed, would improve the quality classification ability of BDFs when used alone; this was found to be the case. When the union of a BDF and LRF was used to classify quality, the predictive accuracy of quality and inspection cost was improved over that of using either function alone for the Space Shuttle. Also, the LRFs proved useful for ranking the quality of modules in a build. The significance of these results is that very high-quality classification accuracy (1.25% error) can be obtained while reducing the inspection cost incurred in achieving high quality. This is particularly important for safety-critical systems. Because the methods are general and not particular to the Shuttle, they could be applied to other domains. A key part of the LRF development was a method for identifying the critical value (i.e. threshold) that could discriminate between high and low quality, and at the same time constrain the cost of inspection to a reasonable value.","PeriodicalId":180205,"journal":{"name":"Proceedings Seventh International Software Metrics Symposium","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2001-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129153051","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Challenges of global software development","authors":"A. Mockus, J. Herbsleb","doi":"10.1109/METRIC.2001.915526","DOIUrl":"https://doi.org/10.1109/METRIC.2001.915526","url":null,"abstract":"Reviews the problems, solutions and open issues in global software development (GSD), in which the software development activities are distributed across multiple sites. First, we introduce the reasons why GSD is becoming more and more prevalent. Then we outline the issues created or amplified by GSD and illustrate various effects it has on a software project. Finally, we summarize the literature on existing approaches to reduce the negative effects of GSD and list key open research questions.","PeriodicalId":180205,"journal":{"name":"Proceedings Seventh International Software Metrics Symposium","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2001-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132194219","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Defining and applying metrics in the context of continuing software evolution","authors":"J. Fernández-Ramil, M. Lehman","doi":"10.1109/METRIC.2001.915529","DOIUrl":"https://doi.org/10.1109/METRIC.2001.915529","url":null,"abstract":"Addresses a set of relevant issues in the context of the definition and application of measurement to long-term software evolution processes and their products. It presents a practical example using empirical data from one of the systems studied as part of the on-going FEAST (Feedback, Evolution And Software Technology) investigation. Such example demonstrates the use of a sequential statistical test (CUSUM) on a suite of eight evolution activity metrics. The metrics are based on module and subsystem counts. The test permitted the examination of whether productivity had changed over an 11-year period. The results were consistent in six of the eight metrics studied, suggesting that sequential tests have potential in, for example, measuring the impact of process improvement (intervention analysis), in the identification of the stages of software evolution and, in general, in supporting evolution management.","PeriodicalId":180205,"journal":{"name":"Proceedings Seventh International Software Metrics Symposium","volume":"50 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2001-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131627327","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The impact of design properties on development cost in object-oriented systems","authors":"L. Briand, J. Wüst","doi":"10.1109/METRIC.2001.915534","DOIUrl":"https://doi.org/10.1109/METRIC.2001.915534","url":null,"abstract":"In the context of software cost estimation, system size is widely taken as a main driver of system development effort, but other structural design properties, such as coupling, cohesion and complexity, have been suggested as additional cost factors. In this paper, using effort data from an object-oriented development project, we empirically investigate the relationship between class size and the development effort for a class, and what additional impact structural properties such as class coupling have on effort. We use Poisson regression and regression trees to build cost prediction models from size and design measures, and use these models to predict the system development effort. We also investigate a technique to combine regression trees with regression analysis, which aims at building more accurate models. The results indicate that fairly accurate predictions of class effort can be made based on simple measures of the class interface size alone; mean MREs (magnitudes of relative error) are below 30%. Effort predictions at the system level are even more accurate as, using bootstrapping, the estimated 95% confidence interval for MREs is 3%-23%, but more sophisticated coupling and cohesion measures do not help to improve these predictions to a degree that would be practically significant. However, the use of hybrid models, combining Poisson regression and CART (classification and regression trees) clearly improves the accuracy of the models as compared to using Poisson regression alone.","PeriodicalId":180205,"journal":{"name":"Proceedings Seventh International Software Metrics Symposium","volume":"50 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2001-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116529832","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Learning to measure or measuring to learn?","authors":"A. Andrews","doi":"10.1109/METRIC.2001.915510","DOIUrl":"https://doi.org/10.1109/METRIC.2001.915510","url":null,"abstract":"Increasingly, however, the focus has shifted away from defining ever more metrics and subjecting measurement results to increasingly sophisticated statistical analyses to an emphasis of understanding software phenomena. This has led to more case studies, empirical work like field studies, as well as experiments. This emphasis can be clearly seen in this year's METRICS program. With this, the methodological concern shifts from measurement theory to proper design of studies. It also has caused a new look at use (and abuse) of various statistical analysis methods.","PeriodicalId":180205,"journal":{"name":"Proceedings Seventh International Software Metrics Symposium","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2001-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115455458","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An experiment for evaluating the effectiveness of using a system dynamics simulation model in software project management education","authors":"Dietmar Pfahl, G. Ruhe, N. Koval","doi":"10.1109/METRIC.2001.915519","DOIUrl":"https://doi.org/10.1109/METRIC.2001.915519","url":null,"abstract":"Due to increasing demand for software project managers in industry, efforts are needed to develop the management-related knowledge and skills of the current and future software workforce. In particular, university education needs to provide to their computer science students not only with technology-related skills but, in addition, a basic understanding of typical phenomena occurring in industrial (and academic) software projects. The paper presents a controlled experiment that evaluates the effectiveness of using a process simulation model for university education in software project management. The experiment uses a pre-test-post-test control group design with random assignment of computer science students. The treatment of the experimental group involves a system dynamics simulation model. The treatment of the control group involves a conventional predictive model for project planning, i.e. the well-known COCOMO model. In addition to the presentation of the results of the empirical study, the paper discusses limitations and threats to validity. Proposals for modifications of the experimental design and the treatments are made for future replications.","PeriodicalId":180205,"journal":{"name":"Proceedings Seventh International Software Metrics Symposium","volume":"89 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2001-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121212116","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Using simulation to evaluate prediction techniques [for software]","authors":"M. Shepperd, G. Kadoda","doi":"10.1109/METRIC.2001.915542","DOIUrl":"https://doi.org/10.1109/METRIC.2001.915542","url":null,"abstract":"The need for accurate software prediction systems increases as software becomes larger and more complex. A variety of techniques have been proposed, but none has proved consistently accurate. The underlying characteristics of the data set influence the choice of the prediction system to be used. It has proved difficult to obtain significant results over small data sets; consequently, we required large validation data sets. Moreover, we wished to control the characteristics of such data sets in order to systematically explore the relationship between accuracy, choice of prediction system and data set characteristics. Our solution has been to simulate data, allowing both control and the possibility of large validation cases. We compared regression, rule induction and nearest neighbours (a form of case-based reasoning). The results suggest that there are significant differences depending upon the characteristics of the data set. Consequently, researchers should consider the prediction context when evaluating competing prediction systems. We also observed that the more \"messy\" the data and the more complex the relationship with the dependent variable, the more variability in the results. This became apparent since we sampled two different training sets from each simulated population of data. In the more complex cases, we observed significantly different results depending upon the training set. This suggests that researchers will need to exercise caution when comparing different approaches and utilise procedures such as bootstrapping in order to generate multiple samples for training purposes.","PeriodicalId":180205,"journal":{"name":"Proceedings Seventh International Software Metrics Symposium","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2001-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128519944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Predicting with sparse data","authors":"M. Shepperd, M. Cartwright","doi":"10.1109/METRIC.2001.915513","DOIUrl":"https://doi.org/10.1109/METRIC.2001.915513","url":null,"abstract":"It is well known that effective prediction of project cost related factors is an important aspect of software engineering. Unfortunately, despite extensive research over more than 30 years, this remains a significant problem for many practitioners. A major obstacle is the absence of reliable and systematic historic data, yet this is a sine qua non for almost all proposed methods: statistical, machine learning or calibration of existing models. The authors describe their sparse data method (SDM) based upon a pairwise comparison technique and T.L. Saaty's (1980) Analytic Hierarchy Process. Our minimum data requirement is a single known point. The technique is supported by a software tool known as DataSalvage. We show, for data from two companies, how our approach, based upon expert judgement, adds value to expert judgement by producing significantly more accurate and less biased results. A sensitivity analysis shows that our approach is robust to pairwise comparison errors. We then describe the results of a small usability trial with a practising project manager. From this empirical work we conclude that the technique is promising and may help overcome some of the present barriers to effective project prediction.","PeriodicalId":180205,"journal":{"name":"Proceedings Seventh International Software Metrics Symposium","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2001-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129386878","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Investigating the impact of reading techniques on the accuracy of different defect content estimation techniques","authors":"B. Freimut, O. Laitenberger, S. Biffl","doi":"10.1109/METRIC.2001.915515","DOIUrl":"https://doi.org/10.1109/METRIC.2001.915515","url":null,"abstract":"Software inspections have established an impressive track record for early defect detection and correction. To increase their benefits, recent research efforts have focused on two different areas: systematic reading techniques and defect content estimation techniques. While reading techniques are to provide guidance for inspection participants on how to scrutinize a software artifact in a systematic manner, defect content estimation techniques aim at controlling and evaluating the inspection process by providing an estimate of the total number of defects in an inspected document. Although several empirical studies have been conducted to evaluate the accuracy of defect content estimation techniques, only few consider the reading approach as an influential factor. The authors examine the impact of two specific reading techniques: a scenario based reading technique and checklist based reading, on the accuracy of different defect content estimation techniques. The examination is based on data that were collected in a large experiment with students of the Vienna University of Technology. The results suggest that the choice of the reading technique has little impact on the accuracy of defect content estimation techniques. Although more empirical work is necessary to corroborate this finding, it implies that practitioners can use defect content estimation techniques without any consideration of their current reading technique.","PeriodicalId":180205,"journal":{"name":"Proceedings Seventh International Software Metrics Symposium","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2001-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129011071","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}