Evaluation of the survival-inferred fragility index to assess the robustness of the estimated treatment effect on survival endpoints

Background/aims

Phase III randomized clinical trials (RCTs) aim to evaluate the benefits of a new treatment. When conducted in patients with malignancies, most RCTs use a right censored endpoint, with conclusions regarding efficacy based on the $p$ value of the logrank test. Recently, the survival inferred fragility index (SIFI) has been proposed as a measure of the robustness of the treatment effect. Applied to real RCTs, the reported SIFI values were very low. We hypothesized that such values relied on the contamination of the trial. Methods: We performed a simulation study of individuals enrolled in an RCT, generating survival times under several realistic scenarios differing in treatment effects, sample sizes and amount of censoring. Contamination of the sample by individuals with a very specific prognosis was studied.

Results

Surprisingly, under the null of no treatment effect, the standard SIFI exhibited very low values, poorly sensitive to the sample size. However, under both the null and the alternative hypotheses, the $p$ value and the amount of censoring influenced the value. By contrast, randomly selected patients, rather than those selected at the survival tails, widely modified the results, notably with an impossibility of finding value under the null in a large proportion of cases. When contaminating the sample with individuals with very poor or good outcomes, results were close to those of the standard.

Conclusions

The SIFI should not be used as a measure of robustness of survival trials, as it relates to a very specific group of individuals. At least, a random selection of patients should be used in its calculation.