Nonattainability of the Fragility Index.

Abstract

Background: The fragility index (FI) is intended to quantify how many outcome changes would be required to convert a statistically significant two-arm trial result into a nonsignificant one. A reliable statistical metric should produce a result for every valid case it evaluates. This study examined whether a fragility value is always attainable for every statistically significant trial result.

Methods: FI was analyzed as follows: baseline significance was required (p < 0.05), one-way movement only, and outcome changes were restricted to converting a nonevent to an event in the arm with fewer events, while keeping the arm size fixed. Nonattainability was assessed by determining whether valid 2×2 tables exist for which no finite FI can be obtained under these rules. Evidence is provided through formal counterexamples, complete enumeration of all valid nondegenerate 2 × 2 tables up to total sample size N = 60, and empirical evaluation of published two-arm trials with binary outcomes.

Results: Valid baseline-significant 2 × 2 tables exist for which FI is not attainable. A simple counterexample is {3,0,4,11}: baseline two-sided Fisher's exact p = 0.0429, the arm with fewer events is uniquely identified, but that arm has no nonevents available for the required toggle; thus, no legal FI path exists. Enumeration revealed that unattainable cases first appeared at N = 18 and then recurred at every larger sample size through N = 60; by N = 60, a total of 2,390 of 20,774 evaluable baseline-significant tables were unattainable (11.5%). In an empirical dataset of published trials, 2 of 82 baseline-significant evaluable trials (2.4%) were not attainable.

Conclusions: The FI is not universally attainable. This is a structural property of the FI algorithm, confirmed by mathematical proof, a complete table enumeration, and published trial data.