Comparing Risks for Binomial Reliability Assurance Test Planning

Balancing consumer's and producer's risk is an important consideration when planning tests. Instead of focusing on finding a single best test plan, we introduce a general framework to systematically identify a set of binomial test plans by leveraging the inverse relationship between the two risks. The framework is applied to compare a variety of assurance testing frameworks, including classical tests, and Bayesian reliability assurance tests such as the Bayesian assurance test, the assurance reliability demonstration test, and the coverage criterion test. Efficient algorithms are presented to compute the set of test plans, providing practitioners with a comprehensive range of options to choose from. In addition, we include a comparison to the sequential probability ratio test. We also provide formal proofs for the inverse relationship between consumer's and producer's risk in Bayesian reliability assurance tests that underlie our algorithms. A case study is presented to illustrate the framework's application and compare the risks associated with different test plans.