Design of reliable software via general combination of N-version programming and acceptance testing

N-version programming (NVP) and acceptance testing (AT) are techniques for ensuring reliable computation results from imperfect software. Various symmetric combinations of NVP and AT have also been suggested. We take the view that one can insert an AT at virtually any point in a suitably constructed multi-channel computation graph and that judicious placement of ATs will lead to cost-effective reliability improvement. Hence, as a general framework for the creation, representation, and analysis of combined NVP-AT schemes, we introduce MTV graphs, and their simplified data-driven version called DD-MTV graphs, composed of computation module (M), acceptance test (T), and voter (V) building blocks. Previous NVP-AT schemes, such as consensus recovery blocks, recoverable N-version blocks, and N-self-checking programs can be viewed as special cases of our general combining scheme. Results on the design and analysis of new NVP-AT schemes are presented and the reliability improvements are quantified. We show, e.g., that certain, somewhat asymmetric, combinations of M, T, and V building blocks can lead to higher reliabilities than previously proposed symmetric arrangements having comparable or higher complexities.