Improving software robustness with dependability cases

Abstract

Programs fail mainly for two reasons: logic errors in the code, and exception failures. Exception failures can account for up to 2/3 of system crashes, hence are worthy of serious attention. Traditional approaches to reducing exception failures, such as code reviews, wallthroughs and formal testing, while very useful, are limited in their ability to address a core problem: the programmer's inadequate coverage of exceptional conditions. The problem of coverage might be rooted in cognitive factors that impede the mental generation (or recollection) of exception cases that would pertain in a particular situation, resulting in insufficient software robustness. This paper describes a study to test the hypothesis that robustness for exception failures can be improved through the use of dependability cases. Dependability cases, derived from safety cases, comprise a methodology based on structured taxonomies and memory aids for helping software designer think about and improve exception-handling coverage. A controlled experiment conducted with 59 subjects revealed a statistically significant 43% increase in exception-handling robustness. An ancillary experiment conducted with 38 subjects provides convergent evidence that the effect is authentic, and not due to programming expertise alone.