Statistics for Testing Under Adverse Conditions

Abstract

Here, we develop a statistical basis for limited adverse testing. This type of testing simultaneously evaluates system performance against minimum requirements and minimizes costs, particularly for large-scale engineering projects. Because testing is often expensive and narrow in scope, the data obtained are relatively limited—precisely the opposite of the recent big data movement but no less compelling. Although a remarkably common approach for industrial and large-scale government projects, a statistical basis for adverse testing remains poorly explored. Here, we prove mathematically, under specific conditions, that setting each independent variable to an adverse condition leads to a similar level of adversity in the dependent variable. For example, setting all normally distributed independent variables to at least their 95th percentile values leads to a result at the 95th percentile. The analysis considers sample size estimates to clarify the value of replicates in this type of testing, determines how many of the independent variables must be set to adverse condition values, and highlights the essential assumptions, so that engineers, statisticians, and subject matter experts know when this statistical framework may be applied successfully and design testing to satisfy statistical requisites.