M. S. Alam, W. H. Chen, W. K. Ehrlich, M. Engel, D. Kropfl, P. Verma
{"title":"Assessing software reliability performance under highly critical but infrequent event occurrences","authors":"M. S. Alam, W. H. Chen, W. K. Ehrlich, M. Engel, D. Kropfl, P. Verma","doi":"10.1109/ISSRE.1997.630878","DOIUrl":"https://doi.org/10.1109/ISSRE.1997.630878","url":null,"abstract":"Operational testing is expensive and time-consuming in applications with highly critical but infrequent event occurrences, and procedures are needed for obtaining valid software reliability estimates. Two such procedures are accelerated stress testing and experimental design techniques. Examples of accelerated stress in software include: a higher request arrival rate, increasing data traffic \"burstiness\", and so on. Furthermore, when the software is tested by executing the system under emergency conditions or exception events, it is critical that experimental design techniques are used to account for \"carry-over\" effects in failure performance. An application of software reliability testing of a telecommunication system node controller is used to describe the issues and our overall approach to software reliability assessment. The reliability projections based on our approach are compared to reliability estimates derived analytically and through simulation. Our results suggest that these statistical techniques are useful techniques for measuring reliability performance under critical events that have a low probability of occurrence.","PeriodicalId":170184,"journal":{"name":"Proceedings The Eighth International Symposium on Software Reliability Engineering","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1997-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123822854","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Input requirements needed to produce an operational profile for a new telecommunications system","authors":"Raymond V. Sandfoss, S. A. Meyer","doi":"10.1109/ISSRE.1997.630852","DOIUrl":"https://doi.org/10.1109/ISSRE.1997.630852","url":null,"abstract":"Summary form only given, as follows. The paper describes our practical experiences in developing an operational profile for a telecommunications system given little existing knowledge about the maintenance/provisioning process and the system's environment. The specific system under test is a high availability, real time system used to provision, maintain, and forecast network hardware for AT&T's emerging technologies. The system requirements needed to derive the various system operating modes, together with the modes' operational profiles are discussed. We also demonstrate how reliability objectives and failure intensity objectives can be defined and subsequently tested under different automated test scenarios for these different system modes.","PeriodicalId":170184,"journal":{"name":"Proceedings The Eighth International Symposium on Software Reliability Engineering","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1997-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124479610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A study of effective regression testing in practice","authors":"W. E. Wong, J. R. Horgan, S. London, H. Agrawal","doi":"10.1109/ISSRE.1997.630875","DOIUrl":"https://doi.org/10.1109/ISSRE.1997.630875","url":null,"abstract":"The purpose of regression testing is to ensure that changes made to software, such as adding new features or modifying existing features, have not adversely affected features of the software that should not change. Regression testing is usually performed by running some, or all, of the test cases created to test modifications in previous versions of the software. Many techniques have been reported on how to select regression tests so that the number of test cases does not grow too large as the software evolves. Our proposed hybrid technique combines modification, minimization and prioritization-based selection using a list of source code changes and the execution traces from test cases run on previous versions. This technique seeks to identify a representative subset of all test cases that may result in different output behavior on the new software version. We report our experience with a tool called ATAC (Automatic Testing Analysis tool in C) which implements this technique.","PeriodicalId":170184,"journal":{"name":"Proceedings The Eighth International Symposium on Software Reliability Engineering","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1997-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126253614","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Guiding reengineering with the operational profile","authors":"James J. Cusick, Max Fine","doi":"10.1109/ISSRE.1997.630853","DOIUrl":"https://doi.org/10.1109/ISSRE.1997.630853","url":null,"abstract":"Summary form only given, as follows. Results of applied software reliability engineering (SRE) processes and tools on two teleconference support systems are documented. Beginning with the initial evaluation of SRE technology for AT&T's TeleConference Service Development in 1992, the article covers the eventual use of SRE in many phases of software development for two separate systems. Introduction of SRE into the system test and production support processes of a legacy system, in conjunction with other development process improvements, brought early rewards in quality and timely delivery. SRE provided a gauge by which quality efforts could be measured. Ramp up time and costs, as well as both successes and failures in the implementation of SRE are discussed. Next, presentation of preliminary findings regarding the use of SRE during the analysis, design, and integration testing phases of a next generation client/server conferencing system are discussed. Reuse of an operational profile and SRE tools originally developed and deployed for the legacy system to guide design on a new mixed platform client/server architecture incorporating a Windows interface and Unix application server is explained. Lastly, a discussion of the appropriateness of SRE methods for both traditional functional decomposition techniques and object oriented analysis and design is offered.","PeriodicalId":170184,"journal":{"name":"Proceedings The Eighth International Symposium on Software Reliability Engineering","volume":"68 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1997-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126328846","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Reducing uncertainty about common-mode failures","authors":"J. Voas, Anup K. Ghosh, F. Charron, L. Kassab","doi":"10.1109/ISSRE.1997.630879","DOIUrl":"https://doi.org/10.1109/ISSRE.1997.630879","url":null,"abstract":"Multi-version programming is employed in fault-tolerant computer systems in order to provide protection against common-mode failure in software. Multi-version programming involves building diverse software implementations of critical functions. The premise of building diverse versions is that the likelihood of a programming error in one version causing a failure in an identical manner as an error in another version is reduced. Skeptics of multi-version programming have correctly pointed out that common-mode failures between redundant diverse versions can reduce the return on investment in creating diverse versions. To date, other than using historical data from other projects, there has been no way to estimate the potential for a given multi-version programming system to suffer a common-mode failure. This paper presents an algorithm and software analysis prototype to reduce the uncertainty of whether software flaws in diverse versions can result in common-mode failure. The analysis uses software fault-injection techniques to subject one or more versions to anomalous behavior. From this, we can predict how the software will behave if real faults exist in the multiple versions.","PeriodicalId":170184,"journal":{"name":"Proceedings The Eighth International Symposium on Software Reliability Engineering","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1997-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129387455","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A practical method for the estimation of software reliability growth in the early stage of testing","authors":"M. Xie, G. Y. Hong, C. Wohlin","doi":"10.1109/ISSRE.1997.630856","DOIUrl":"https://doi.org/10.1109/ISSRE.1997.630856","url":null,"abstract":"The traditional approach of reliability prediction using software reliability growth models requires a large number of failures which might not be available at the beginning of the testing. The commonly used maximum likelihood estimates may not even exist or converge to a reasonable value. In this paper, an approach of making use of information from similar projects in order to obtain an early estimation of one model parameter for a current project is studied. As most of the two-parameter reliability growth models contains one parameter related to the number of faults in the software and a reliability growth rate parameter related to the testing efficiency, information from a similar project can used to estimate the reliability growth rate parameter and the limited failure data from initial testing is used to estimate the other parameter. Our case study shows that this approach is very easy to use as the estimation does not require a numerical algorithm and it always exists. It is also very stable and when the maximum likelihood estimates exist and are reasonable, our approach gives values very close to that, and the approximate confidence interval is overlapping for most cases.","PeriodicalId":170184,"journal":{"name":"Proceedings The Eighth International Symposium on Software Reliability Engineering","volume":"91 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1997-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131434634","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Keynote: Software Reliability In Theory And Practice","authors":"L. Dalton","doi":"10.1109/ISSRE.1997.630867","DOIUrl":"https://doi.org/10.1109/ISSRE.1997.630867","url":null,"abstract":"Quantitative measurement of software reliability remains an elusive goal. Simultaneously, there is an ever increasing critical dependency on software-based systems in the human infrastructure. There is a great deal of research in addition to well-intentioned methods and practices focused on improving the reliability of software-based systems. To date, most of which still only yields qualitative measurements except in very simple cases, e.g. a very few hundred lines of code. Probabilistic methods, such as reliability growth models, do offer a statistical inference for applications amenable to the methods. This talk will explore the meaning of software rcliahility in the context of safety, security, control, and quality attributes from an ultra-high assurance systems perspective, e.g. nuclear weapons. A systems level perspective will he taken related to the sources of defects (failure potential), including the “cross product” of software and hardware failure state spaces. The authors view of opportunities to improve the reliability of software-based will he presented.","PeriodicalId":170184,"journal":{"name":"Proceedings The Eighth International Symposium on Software Reliability Engineering","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1997-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131318527","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An architectural model for software reliability quantification","authors":"C. Smidts, D. Sova, G. K. Mandela","doi":"10.1109/ISSRE.1997.630880","DOIUrl":"https://doi.org/10.1109/ISSRE.1997.630880","url":null,"abstract":"Software reliability assessment models in use today treat software as a monolithic block. An aversion towards \"atomic\" models seems to exist. These models appear to add complexity to the modeling, to the data collection and seem intrinsically difficult to generalize. This paper presents an architecturally based software reliability model and underlines its benefits. The model is based on an architecture derived from the requirements which captures both functional and non-functional requirements and on a generic classification of functions, attributes and failure modes. The model focuses on evaluation of failure mode probabilities and uses a Bayesian quantification framework. Failure mode probabilities of functions and attributes are propagated to the system level using fault trees. The model accounts explicitly for the type of software development life-cycle. It can incorporate any type of prior information, such as results of developers' testing or historical information on a specific functionality and its attributes, and is ideally suited for reusable software. By building an architecture and deriving its potential failure modes, the model forces early appraisal and understanding of the weaknesses of the software, allows reliability analysis of the structure of the system, and provides assessments at a functional level as well as at the system level. The model is applied to a simple example. The model's assessment of reliability tends to be conservative and the model appears to be scalable to larger, more complex problems.","PeriodicalId":170184,"journal":{"name":"Proceedings The Eighth International Symposium on Software Reliability Engineering","volume":"105 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1997-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115550822","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. Hochman, T. Khoshgoftaar, E. B. Allen, J. Hudepohl
{"title":"Evolutionary neural networks: a robust approach to software reliability problems","authors":"R. Hochman, T. Khoshgoftaar, E. B. Allen, J. Hudepohl","doi":"10.1109/ISSRE.1997.630844","DOIUrl":"https://doi.org/10.1109/ISSRE.1997.630844","url":null,"abstract":"In this empirical study, from a large data set of software metrics for program modules, thirty distinct partitions into training and validation sets are automatically generated with approximately equal distributions of fault prone and not fault prone modules. Thirty classification models are built for each of the two approaches considered-discriminant analysis and the evolutionary neural network (ENN) approach-and their performances on corresponding data sets are compared. The lower error proportions for ENNs on fault prone, not fault prone, and overall classification were found to be statistically significant. The robustness of ENNs follows from their superior performance on the range of data configurations used. It is suggested that ENNs can be effective in other software reliability problem domains, where they have been largely ignored.","PeriodicalId":170184,"journal":{"name":"Proceedings The Eighth International Symposium on Software Reliability Engineering","volume":"59 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1997-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114902283","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Automatic simulation to predict software architecture reliability","authors":"J. J. Li, Josephine Micallef, J. R. Horgan","doi":"10.1109/ISSRE.1997.630862","DOIUrl":"https://doi.org/10.1109/ISSRE.1997.630862","url":null,"abstract":"One of the goals of software architecture research is to analyze the reliability of a software system early in the development lifecycle. This allows the system designers to predict reliability and evaluate alternative designs. This paper presents a methodology and accompanying toolset, Workflow-to-SDL-Direct-Simulation (W2S), for generating a simulator from a semi-formal architecture description. This simulator allows the analysis of the system's reliability based on the simulated behavior and performance. There are three steps in the W2S methodology: translation of the description to a communicating extended finite state machine (CEFSM) model; generation of a simulator from the CEFSM model and a model of the system environment; and execution of the simulator on top of an underlying CEFSM engine. The W2S methodology was applied to a Bellcore telecommunication control system, using a set of tools that we developed. Several sets of experiments were carried out. The result shows that the information generated by the simulator can be used to predict the reliability of a system solely based on its architectural description. This prediction allows designers to select more reliable architectures.","PeriodicalId":170184,"journal":{"name":"Proceedings The Eighth International Symposium on Software Reliability Engineering","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1997-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126827739","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}