A. Amiar, Mickaël Delahaye, Yliès Falcone, L. D. Bousquet
{"title":"Fault localization in embedded software based on a single cyclic trace","authors":"A. Amiar, Mickaël Delahaye, Yliès Falcone, L. D. Bousquet","doi":"10.1109/ISSRE.2013.6698914","DOIUrl":"https://doi.org/10.1109/ISSRE.2013.6698914","url":null,"abstract":"Locating faults in embedded software, especially in microcontrollers, is still difficult. Quite recently, it became possible to recover execution traces from microcontrollers using specific hardware probes. However, the collected traces contain a huge volume of low-level data. Consequently, manual analysis is difficult and our industrial partners call for automatic and more effective fault-localization methods for embedded software. This paper presents a new approach to automatically locate faults in embedded programs given a single faulty execution trace. Our approach exploits the cyclic nature of embedded programs and uses several adapted spectrum-based methods in order to find faults on a single execution, rather than a set of multiple failing and passing executions. Our approach is implemented in the tool CoMET and evaluated on several faulty programs. The evaluation shows that our single-trace fault-localization method using Ochiai [1] allows engineers to find a fault by inspecting less than 5% of the program in most cases, and it confirms the interest of automatic fault localization for microcontrollers.","PeriodicalId":288473,"journal":{"name":"2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123756519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Rakesh Rana, M. Staron, C. Berger, J. Hansson, Martin Nilsson, Fredrik Törner
{"title":"Evaluating long-term predictive power of standard reliability growth models on automotive systems","authors":"Rakesh Rana, M. Staron, C. Berger, J. Hansson, Martin Nilsson, Fredrik Törner","doi":"10.1109/ISSRE.2013.6698922","DOIUrl":"https://doi.org/10.1109/ISSRE.2013.6698922","url":null,"abstract":"Software is today an integral part of providing improved functionality and innovative features in the automotive industry. Safety and reliability are important requirements for automotive software and software testing is still the main source of ensuring dependability of the software artifacts. Software Reliability Growth Models (SRGMs) have been long used to assess the reliability of software systems; they are also used for predicting the defect inflow in order to allocate maintenance resources. Although a number of models have been proposed and evaluated, much of the assessment of their predictive ability is studied for short term (e.g. last 10% of data). But in practice (in industry) the usefulness of SRGMs with respect to optimal resource allocation depends heavily on the long term predictive power of SRGMs i.e. much before the project is close to completion. The ability to reasonably predict the expected defect inflow provides important insight that can help project and quality managers to take necessary actions related to testing resource allocation on time to ensure high quality software at the release. In this paper we evaluate the long-term predictive power of commonly used SRGMs on four software projects from the automotive sector. The results indicate that Gompertz and Logistic model performs best among the tested models on all fit criterias as well as on predictive power, although these models are not reliable for long-term prediction with partial data.","PeriodicalId":288473,"journal":{"name":"2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE)","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115036664","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Request success rate of multipathing I/O with a paired storage controller","authors":"Gangadhar Enagandula, V. Apte, Bipul Raj","doi":"10.1109/ISSRE.2013.6698906","DOIUrl":"https://doi.org/10.1109/ISSRE.2013.6698906","url":null,"abstract":"The success probability of I/O requests in presence of failures is increased by a combination of failover mechanisms built into the storage server, multiple access paths from I/O clients to the server, and timeout-retry mechanisms at the client itself. We define and evaluate a unified availability metric, request failures per million (RFPM), which quantifies request failure probability while taking into account client-side as well as server-side mechanisms. We calculate this metric using a two-level model of I/O service - a probability tree that captures the I/O driver behaviour, and a set of CTMC (Continuous Time Markov Chain) models that capture failover mechanisms at the server. The I/O driver model captures detailed timeout-retry mechanisms including retries at multiple ports (“multipathing”). The server model captures transient phenomena such as failure detection, takeover and emulation behaviour of a paired storage controller. The model shows that client retry mechanisms provide significant improvement in request success probability. The model is then used to study the sensitivity of RFPMs to parameters such as timeouts, reboot time and failure detection delay. The results show that the model can help in answering several what-if questions related to how system parameters impact request success rate.","PeriodicalId":288473,"journal":{"name":"2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129872812","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Feedback-directed exploration of web applications to derive test models","authors":"A. M. Fard, A. Mesbah","doi":"10.1109/ISSRE.2013.6698880","DOIUrl":"https://doi.org/10.1109/ISSRE.2013.6698880","url":null,"abstract":"Dynamic exploration techniques play a significant role in automated web application testing and analysis. However, a general web application crawler that exhaustively explores the states can become mired in limited specific regions of the web application, yielding poor functionality coverage. In this paper, we propose a feedback-directed web application exploration technique to derive test models. While exploring, our approach dynamically measures and applies a combination of code coverage impact, navigational diversity, and structural diversity, to decide a-priori (1) which state should be expanded, and (2) which event should be exercised next to maximize the overall coverage, while minimizing the size of the test model. Our approach is implemented in a tool called FEEDEx. We have empirically evaluated the efficacy of FEEDEx using six web applications. The results show that our technique is successful in yielding higher coverage while reducing the size of the test model, compared to classical exhaustive techniques such as depth-first, breadth-first, and random exploration.","PeriodicalId":288473,"journal":{"name":"2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE)","volume":"94 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121702247","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Alex Groce, Chaoqiang Zhang, Mohammad Amin Alipour, E. Eide, Yang Chen, J. Regehr
{"title":"Help, help, i'm being suppressed! The significance of suppressors in software testing","authors":"Alex Groce, Chaoqiang Zhang, Mohammad Amin Alipour, E. Eide, Yang Chen, J. Regehr","doi":"10.1109/ISSRE.2013.6698892","DOIUrl":"https://doi.org/10.1109/ISSRE.2013.6698892","url":null,"abstract":"Test features are basic compositional units used to describe what a test does (and does not) involve. For example, in API-based testing, the most obvious features are function calls; in grammar-based testing, the obvious features are the elements of the grammar. The relationship between features as abstractions of tests and produced behaviors of the tested program is surprisingly poorly understood. This paper shows how large-scale random testing modified to use diverse feature sets can uncover causal relationships between what a test contains and what the program being tested does. We introduce a general notion of observable behaviors as targets, where a target can be a detected fault, an executed branch or statement, or a complex coverage entity such as a state, predicate-valuation, or program path. While it is obvious that targets have triggers - features without which they cannot be hit by a test - the notion of suppressors - features which make a test less likely to hit a target - has received little attention despite having important implications for automated test generation and program understanding. For a set of subjects including C compilers, a flash file system, and JavaScript engines, we show that suppression is both common and important.","PeriodicalId":288473,"journal":{"name":"2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131867261","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Stefano Di Alesio, S. Nejati, L. Briand, A. Gotlieb
{"title":"Stress testing of task deadlines: A constraint programming approach","authors":"Stefano Di Alesio, S. Nejati, L. Briand, A. Gotlieb","doi":"10.1109/ISSRE.2013.6698915","DOIUrl":"https://doi.org/10.1109/ISSRE.2013.6698915","url":null,"abstract":"Safety-critical Real Time Embedded Systems (RT-ESs) are usually subject to strict timing and performance requirements that must be satisfied for the system to be deemed safe. In this paper, we use effective search strategies whose goal is finding worst case scenarios with respect to deadline misses. Such scenarios can in turn be used to test the target RTES and ensure that it satisfies its timing requirements even under worst case conditions. Specifically, we develop an approach based on Constraint Programming (CP) to automate the generation of test cases that reveal, or are likely to, task deadline misses. We evaluate it through a comparison with a state-of-the-art approach based on Genetic Algorithms (GA). In particular, we compare CP and GA in five case studies for efficiency, effectiveness, and scalability. Our experimental results show that, on the largest and more complex case studies, CP performs significantly better than GA. Furthermore, CP offers some advantages over GA, such as it guarantees a complete search when there is sufficient time, and, being deterministic, it doesn't rely on parameters that potentially have a significant effect on the search and therefore need to be tuned. Hence, we conclude that our results are encouraging and suggest this is an advantageous approach for stress testing of RTESs with respect to timing constraints.","PeriodicalId":288473,"journal":{"name":"2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE)","volume":"11 4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128582353","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Finding test data with specific properties via metaheuristic search","authors":"R. Feldt, Simon M. Poulding","doi":"10.1109/ISSRE.2013.6698888","DOIUrl":"https://doi.org/10.1109/ISSRE.2013.6698888","url":null,"abstract":"For software testing to be effective the test data should cover a large and diverse range of the possible input domain. Boltzmann samplers were recently introduced as a systematic method to randomly generate data with a range of sizes from combinatorial classes, and there are a number of automated testing frameworks that serve a similar purpose. However, size is only one of many possible properties that data generated for software testing should exhibit. For the testing of realistic software systems we also need to trade off between multiple different properties or search for specific instances of data that combine several properties. In this paper we propose a general search-based framework for finding test data with specific properties. In particular, we use a metaheuristic, differential evolution, to search for stochastic models for the data generator. Evaluation of the framework demonstrates that it is more general and flexible than existing solutions based on random sampling.","PeriodicalId":288473,"journal":{"name":"2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127486847","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Qiushi Wang, Marti Griera Jorba, J. Ripoll, K. Wolter
{"title":"Analysis of local re-execution in mobile offloading system","authors":"Qiushi Wang, Marti Griera Jorba, J. Ripoll, K. Wolter","doi":"10.1109/ISSRE.2013.6698902","DOIUrl":"https://doi.org/10.1109/ISSRE.2013.6698902","url":null,"abstract":"Richer functional mobile applications are developed through offloading the heavy computation from resource constrained mobile devices to powerful cloud servers. For guaranteeing the completion of such tasks, strong connectivity with the servers is essential. However, the wireless network is not sufficiently reliable. It is often unable to satisfy the offloading condition, and thus interrupts the program execution. To deal with this problem, locally re-executing the offloaded computation in the mobile client is an efficient way to maintain the continuity of the application. In this paper, we present a synthetical method to find the optimal moment for launching the local re-execution. First, we develop a general program engine to implement offloading applications. Then we design a Stochastic Activity Network (SAN) model based on the engine structure and three metrics to evaluate the engine's performance. The model parameters are set to values obtained from experimental data of our engine. We compare the performance of different local re-execution launching intervals with the model simulation and find the optimum. Last, the optimal launching interval is returned to configure the engine and its efficiency is verified with experiments. The result shows that starting the local re-execution after an appropriate interval provides a better performance than always offloading, and our model can effectively find the optimal launching interval.","PeriodicalId":288473,"journal":{"name":"2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE)","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130148561","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Sinfer: Inferring information flow lattices for checking self-stabilization","authors":"Y. Eom, Brian Demsky","doi":"10.1109/ISSRE.2013.6698925","DOIUrl":"https://doi.org/10.1109/ISSRE.2013.6698925","url":null,"abstract":"Self-stabilizing programs are guaranteed to recover from state corruption caused by software bugs or other events and eventually reach the correct state. Many real-world applications including embedded controllers and multimedia applications can be designed to make key components self-stabilizing. Type systems and static analyses can automatically check whether a program is self stabilizing. The existing approach for checking self-stabilization requires developers to manually annotate code. We present an annotation inference algorithm that automatically derives an initial set of annotations and therefore lowers the effort to build self-stabilizing systems. Our experience with the algorithm indicates that it effectively inferred annotations for our benchmarks.","PeriodicalId":288473,"journal":{"name":"2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE)","volume":"225 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133754155","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Constructing subtle higher order mutants for Java and AspectJ programs","authors":"Elmahdi Omar, Sudipto Ghosh, L. D. Whitley","doi":"10.1109/ISSRE.2013.6698887","DOIUrl":"https://doi.org/10.1109/ISSRE.2013.6698887","url":null,"abstract":"One goal of higher order mutation testing is to produce higher order mutants (HOMs) that represent subtle faults. We define subtle HOMs as those that are not killed by an existing test set that kills all the first order mutants of a given program. The fault detection effectiveness of the test set can be improved by adding test cases that kill subtle HOMs. However, finding subtle HOMs can be costly even for small programs because of the large space of candidate HOMs. Moreover, a large majority of HOMs are killed by test sets that kill all first order mutants, making the subtle ones relatively rare. We introduce three search-based algorithms (Genetic Algo-rithm, Local Search, and Random Search) for finding subtle HOMs in Java and AspectJ programs. All three algorithms found subtle HOMs for all studied programs but Local Search was more successful in finding subtle HOMs than Genetic Algorithm and Random Search.","PeriodicalId":288473,"journal":{"name":"2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132294820","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}