Statistical derivation of cut-off values for in vitro assays.

Abstract

Chemical substances and mixtures are classified based on their toxicological hazard. Today, in vitro methods are more frequently applied for this purpose. The regulatory validation process assesses their relevance by comparing them to standard in vivo test data, which includes transforming continuous read-out data into ordinal data (hazard classes). Existing strategies for developing new methods overlook the constraints associated with small data sets omitting the use of contemporary statistical techniques, such as uncertainty quantification and bootstrapping. To overcome these limitations, we apply bootstrapping, estimates for the out-of-sample error, and uncertainty quantification to the validation dataset of Kaluzhny et al. (2011) and a dataset of plant protection products (PPPs) previously published by Kolle et al. (2015), which have been tested for eye irritation in vitro (OECD TG492) and in vivo (OECD TG 405). Assessment criteria for sensitivity, specificity, and accuracy are proposed, considering uncertainty quantification and estimation of the out-of-sample error. The cut-off value for plant protection products based on the available set of in vitro-in vivo data pairs can be improved by the application of modern cut-off approaches. For PPPs, the OECD recommended cut-off of 60% mean tissue viability based on single substances leads to lower sensitivity than the newly derived cut-off value of 67%. For liquid single substances, the OECD recommended cut-off is confirmed. This case study demonstrates that modern statistical methods for small datasets improve the reliability of in vitro cut-off values and should therefore be used to revise and derive cut-off values for hazard classification in future.