Evaluating Software Safety Standards: A Systematic Review and Comparison

Abstract

Software safety standards are commonly used to guide the development of safety-critical software systems. However, given the existence of multiple competing standards, it is critical to select the most appropriate one for a given project. We have developed a set of 15 criteria to evaluate each standard in terms of its usage, strengths, and limitations. Five standards are studied, including a NASA Software Safety Standard, an FAA System Safety Handbook, MIL-STD-882D (US Department of Defense), DEF-STAN 00-56 (UK Ministry of Defense), and DO-178B (Commercial avionics). Results of our evaluation suggest that different standards score differently with respect to each evaluation criterion. No standard performs better than others on all the criteria. The lessons learned from software-related accidents in which the standards were involved provide further insights on the pros and cons of using each standard.