Proof-guided testing: an experimental study

Abstract

Proof-guided testing is intended to enhance the test design with information extracted from the argument for correctness. The target application field is the verification of fault-tolerance algorithms where a paper proof is published Ideally, testing should be focused on the weak parts of the demonstration. The identification of weak parts proceeds by restructuring the informal discourse as a proof tree and analyzing it step by step. The approach is experimentally assessed using the example of a flawed group membership protocol (GMP). Results are quite promising: (1) compared to crude random testing, the proof-guided method allowed us to significantly improve the fault revealing power of test data; (2) the overall method also provided useful feedback on the proof and its potential flaw(s).