{"title":"An Effective Software Reliability Analysis Framework for Weapon System Development in Defense Domain","authors":"Dalju Lee, Jongmoon Baik, Hoyeon Ryu, Ju-Hwan Shin","doi":"10.1109/ISSRE.2008.38","DOIUrl":"https://doi.org/10.1109/ISSRE.2008.38","url":null,"abstract":"Software reliability has been regarded as one of the most important quality attributes for software intensive systems, especially in defense domain. As most of weapon systems complicated functionalities and controls are implemented by software which is embedded in hardware systems, it became more critical to assure high reliability for software itself. However, many software development organizations in Korea defense domain have had problems in performing reliability engineered processes for developing mission-critical and/or safety-critical weapon systems. In this paper, we propose an effective framework with which software organizations can identify and select metrics associated software reliability, analyze the collected data, appraise software reliability, and develop software reliability prediction/estimation model based on the result of data analyses.","PeriodicalId":448275,"journal":{"name":"2008 19th International Symposium on Software Reliability Engineering (ISSRE)","volume":"61 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127695705","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
P. Li, Mingtian Ni, Song Xue, Joseph P. Mullally, M. Garzia, Mujtaba Khambatti
{"title":"Reliability Assessment of Mass-Market Software: Insights from Windows Vista®","authors":"P. Li, Mingtian Ni, Song Xue, Joseph P. Mullally, M. Garzia, Mujtaba Khambatti","doi":"10.1109/ISSRE.2008.60","DOIUrl":"https://doi.org/10.1109/ISSRE.2008.60","url":null,"abstract":"Assessing the reliability of mass-market software (MMS), such as the Windowsreg operating system, presents many challenges. In this paper, we share insights gained from the Windows Vistareg and Windows Vistareg SP1 operating systems. First, we find that the automated reliability monitoring approach, which periodically reports reliability status, provides higher quality data and requires less effort compared to other approaches available today. We describe one instance in detail: the Windows reliability analysis component, and illustrate its advantages using data from Windows Vista. Second, we show the need to account for usage scenarios during reliability assessments. For pre-release versions of Windows Vista and Vista SP1, usage scenarios differ by 2-4X for Microsoft internal and external samples; corresponding reliability assessments differ by 2-3X. Our results help motivate and guide further research in reliability assessment.","PeriodicalId":448275,"journal":{"name":"2008 19th International Symposium on Software Reliability Engineering (ISSRE)","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123535906","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"GB Coverage Criteria: The Measurement for Testing a \"Go Back\" Function Based on a Pushdown Automaton","authors":"Tomohiko Takagi, Z. Furukawa","doi":"10.1109/ISSRE.2008.41","DOIUrl":"https://doi.org/10.1109/ISSRE.2008.41","url":null,"abstract":"A GB (go back) function of software for canceling recent user operations and going back to previous states is indispensable to realize usability, and is implemented in much software regardless of its application domain. This paper shows GB coverage criteria that measure the progress of GB function testing based on a PDA (pushdown automaton). They are the extensions of existing coverage criteria based on a state machine, such as a state and N-switch.","PeriodicalId":448275,"journal":{"name":"2008 19th International Symposium on Software Reliability Engineering (ISSRE)","volume":"161 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121825000","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Software Reliability Modeling with Logistic Test Coverage Function","authors":"Haifeng Li, Qiuying Li, Minyan Lu","doi":"10.1109/ISSRE.2008.51","DOIUrl":"https://doi.org/10.1109/ISSRE.2008.51","url":null,"abstract":"Test coverage is a good indicator for testing completeness and effectiveness. This paper utilizes the logistic function to describe the test coverage growth behavior. Based on the logistic test coverage function, a model that relates test coverage to fault detection is presented and fitted to one actual data set. The experimental results show that, compared with three existing models, the evaluation performance of this new model is the best at least with the experimental data. Finally, the logistic test coverage function is applied to the NHPP software reliability modeling for further research.","PeriodicalId":448275,"journal":{"name":"2008 19th International Symposium on Software Reliability Engineering (ISSRE)","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130003487","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Hardware-Assisted Tool for Fast, Full Code Coverage Analysis","authors":"Albert Tran, Michael R. Smith, James Miller","doi":"10.1109/ISSRE.2008.22","DOIUrl":"https://doi.org/10.1109/ISSRE.2008.22","url":null,"abstract":"Software reliability can be improved by using code coverage analysis to ensure that all statements are executed at least once during the testing process. When full code coverage information is obtained through software code instrumentation, high runtime performance overheads are incurred. Techniques that perform deferred or selective code instrumentation have shown success in reducing run-time overheads; however, the execution profile remains distorted. Techniques have been proposed that use internal processor hardware during the data gathering process, e.g. program counter logging. These approaches have been shown to reduce overheads; but currently trade swift execution for sparse code coverage. By combining the branch-vector hardware designed for debugging modern embedded processors with on-demand code coverage analysis, we have developed a new tool which provides full code coverage, while minimizing performance distortions. Experimental results show a performance impact of only 8 - 12%, while still providing 100% code coverage information.","PeriodicalId":448275,"journal":{"name":"2008 19th International Symposium on Software Reliability Engineering (ISSRE)","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127782474","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"RUGRAT: Runtime Test Case Generation Using Dynamic Compilers","authors":"B. Breech, L. Pollock, John Cavazos","doi":"10.1109/ISSRE.2008.28","DOIUrl":"https://doi.org/10.1109/ISSRE.2008.28","url":null,"abstract":"The testing of error handling and dynamic security mechanisms often depends on reproducing specific conditions outside the realm of an application's normal program state. We present RUGRAT, a novel technique to automatically generate tests for these challenging test situations. RUGRAT uses a dynamic compiler to add instructions to the program during execution, and thus dynamically generates tests to exercise code designed to handle uncommon situations during program execution. The RUGRAT testing approach is independent of the source language, requires no modification to the source orbinary program under test and generates runtime tests automatically based on a simple test specification. We demonstrate RUGRAT's capabilities by targeting two particular uncommon situations: handling errors from system and application calls, and testing security mechanisms that protect a program against attacks on function pointers. Both code coverage and failure detection results indicate that RUGRAT is a cost effective approach that reduces the number of required test inputs and need for vulnerable programs.","PeriodicalId":448275,"journal":{"name":"2008 19th International Symposium on Software Reliability Engineering (ISSRE)","volume":"51 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124937391","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lily Huang, Michael R. Smith, Albert Tran, James Miller
{"title":"E-RACE, A Hardware-Assisted Approach to Lockset-Based Data Race Detection for Embedded Products","authors":"Lily Huang, Michael R. Smith, Albert Tran, James Miller","doi":"10.1109/ISSRE.2008.23","DOIUrl":"https://doi.org/10.1109/ISSRE.2008.23","url":null,"abstract":"Limited research exists for identifying data races under the specific characteristics found in embedded systems. E-RACE is a new style of data-race identification tool which directly utilizes specialized hardware capabilities to monitor the flow of data and instructions. Compared to existing data race analysis approaches, the hardware-assisted E-RACE tool has advantages of recognizing data-race issues without requiring extensive software code instrumentation. The tool is integrated into an Embedded Unit Testing Driven Development Framework to encourage the construction of testable code and early identification of data-races.","PeriodicalId":448275,"journal":{"name":"2008 19th International Symposium on Software Reliability Engineering (ISSRE)","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125508335","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"On Reliability Analysis of Open Source Software - FEDORA","authors":"P. Anbalagan, M. Vouk","doi":"10.1109/ISSRE.2008.53","DOIUrl":"https://doi.org/10.1109/ISSRE.2008.53","url":null,"abstract":"Reliability analyses of software systems often focus only on the number of faults reported against the software. Using a broader set of metrics, such as problem resolution times and field software usage levels, can provide a more comprehensive view of the product. Some of these metrics are more readily available for open source products. We analyzed a suite of FEDORA releases and obtained some interesting findings. For example, we show that traditional reliability models may be used to predict problem rates across releases. We also show that security related reports tend to have a different profile than non-security related problem reporting and repair.","PeriodicalId":448275,"journal":{"name":"2008 19th International Symposium on Software Reliability Engineering (ISSRE)","volume":"41 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127986940","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Static Detection of Redundant Test Cases: An Initial Study","authors":"Nuo Li, Patrick Francis, Brian P. Robinson","doi":"10.1109/ISSRE.2008.50","DOIUrl":"https://doi.org/10.1109/ISSRE.2008.50","url":null,"abstract":"As software systems evolve, the size of their test suites grow due to added functionality and customer-detected defects. Many of these tests may contain redundant elements with previous tests. Existing techniques to minimize test suite size generally require dynamic execution data, but this is sometimes unavailable. We present a static technique that identifies test cases with redundant instruction sequences, allowing them to be merged or eliminated. Initial results at ABB show that 7%-23% of one test suite may be redundant.","PeriodicalId":448275,"journal":{"name":"2008 19th International Symposium on Software Reliability Engineering (ISSRE)","volume":"50 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124457726","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Operating System Robustness Forecast and Selection","authors":"Xiaoen Ju, Hengming Zou","doi":"10.1109/ISSRE.2008.10","DOIUrl":"https://doi.org/10.1109/ISSRE.2008.10","url":null,"abstract":"While commercial off-the-shelf (COTS) operating systems (OS) have long been widely used, the issue regarding their robustness is far from being solved. Although many efforts have been made in this research domain, people still find it difficult to make choices among various OSs for robustness concerns. This paper proposes a reference model for OS robustness forecast and selection that aims to forecast the robustness of specific OSs under given operational profiles. At the same time, the model can select appropriate OSs as development/operating platforms that meet the particular robustness requirements of the target workloads. Our model combines OSspsila overall robustness with operational profiles and uses extensive tests on OS APIs to make our calculation. We have measured 255 APIs and C-library functions on windows XP and Vista, and 197 C-library functions on Linux 2.6.22 (Ubuntu 7.10). Our results show that on average Windows XP and Vista are more robust than Linux, but their performances are comparable under compute-intensive workloads. A demonstration of how these results are used in the proposed reference model for OSs robustness forecast and selection is given at the end.","PeriodicalId":448275,"journal":{"name":"2008 19th International Symposium on Software Reliability Engineering (ISSRE)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124981471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}