Comparison of the power and type 1 error of total score models for drug effect detection in clinical trials.

Abstract

Composite scale data consists of numerous categorical questions/items that are often summed as a total score and are commonly utilized as primary endpoints in clinical trials. These endpoints are conceptually discrete and constrained by nature. Item response theory (IRT) is a powerful approach for detecting drug effects in composite scale data from clinical trials, but estimating all parameters requires a large sample size and all item information, which may not be available. Therefore, total score models are often utilized. The most popular total score models are continuous variable (CV) models, but this strategy establishes assumptions that go against the integer nature, and typically also the bounded nature, of data. Bounded integer (BI) and Coarsened grid (CG) models respect the nature of the data. However, their power to detect drug effects has not been as thoroughly studied in clinical trials. When an IRT model is accessible, IRT-informed models (I-BI and I-CV) are promising methods in which the mean and variability of the total score at any position are extracted from the existing IRT model. In this study, total score data were simulated from the MDS-UPDRS motor subscale. Then, the power, type 1 error, and treatment effect bias of six total score models for detecting drug effects in clinical trials were explored. Further, it was investigated how the power, type 1 of error, and treatment effect bias for the I-BI and I-CV models were affected by mis-specified item information from the IRT model. The I-BI model demonstrated the highest statistical power, maintained an acceptable Type I error rate, and exhibited minimal bias, approaching zero. Following that, the I-CV, BI, and CG with Czado transformation (CG_Czado) models provided the maximum power. However, the CG_Czado model had inflated type 1 error under low sample size scenarios in each arm of clinical trials. The CG model among total score models displayed the lowest power and the most inflated type 1 error. Therefore, the results favor the I-BI model when an IRT model is available; otherwise, the BI model.