STUDY POPULATION SELECTION USING MACHINE LEARNING FROM THE FNIH BIOMARKERS CONSORTIUM PROGRESS OA COHORT

INTRODUCTION

Stringent participant selection criteria for DMOAD trials are crucial, but there is no consensus on the criteria currently accepted by the regulatory authorities, particularly not criteria adapted to different treatment targets. Ensuring a population with a higher probability of treatment-specific OA progression may facilitate cost-effective trials with less risk of failure.

OBJECTIVE

To investigate whether simple Machine Learning (ML) methods provide more effective and/or transparent selection criteria than conventional statistical methods.

METHODS

We investigated the FNIH Biomarkers Consortium cohorts. Phase 1 included 600 subjects from the OAI, as a case/control cohort wrt. OA progression defined by JSW and/or pain progression (JSN decrease ³ 0.7 mm, WOMAC total pain increase ³ 9). Phase 2 included control groups from DMOAD trials (SEKOIA, VIDEO, ILLUSTRATE-K, ROCCELLA), in total 1233 subjects, using the same JSW/Pain endpoints. Consortium members provided biomarker scores for potentially prognostic biomarkers. We focused on the baseline imaging biomarkers submitted for both phases, including semi-quantitative MOAKS readings and quantitative cartilage morphology from MRI. We analyzed the sub-cohorts with complete imaging and clinical biomarkers, i.e. 600 and 366 subjects, respectively. We used a k-nearest neighbor classifier to predict progression as defined by the endpoints selecting a biomarker subset using sequential forward feature selection (SFFS). Model training and validation were performed using 10-fold cross-validation (CV). We measured model performance by the median AUC score across the CV test sets. The performance was compared to a classical logistic regression model with elastic-net regularization, which was trained and scored using the same SFFS and CV.

RESULTS

The AUC scores and selected imaging biomarkers for each progression endpoints are shown in Table 1 below. In general, compared to the Logistic Regression models, the kNN models performed on par or slightly better (Phase 1: 0.80 vs 0.79 for JSN and 0.66 vs 0.68 for Pain; Phase 2: 0.76 vs 0.77 for JSN and 0.82 vs 0.73 for Pain).

CONCLUSION

The ML model possibly performed slightly better than classical logistic regression. However, it should be analyzed whether the two models include the same biomarkers and what the implications are for the required study sample size. Further, the feature selection step is very relevant for clinical trial design, to ensure a limited yet predictive set of features.