Development and evaluation of methods of clinical utility-based cut-point selection of diagnostic biomarkers: an analysis based on population-level parametric distributions of test results with application of clinical diagnostic data.

Abstract

Introduction: The cut-point selection of biomarkers based on clinical benefit of test results rather than accuracy-based is of interest for decision makers. We adapted the four methods of cut-point selection based on clinical utility of test results including Youden, Product, Union and the absolute difference of total utility with 2 times of AUC.

Methods: The population-based parametric pairs of distributions of test results comprising homoscedastic binormal model, non-homoscedastic binormal, bigamma and biexponential included in the study. For each pair of distributions for diseased and non-diseased the utility-based metrics of cut-point were calculated under different degrees of AUC and prevalence. The prevalence was varied from 0.01 to 0.05, 0.10, 0.30, and 0.50.

Results: For a low prevalence as low as 0.01, the two methods of Product, and Union that maximize and minimize the related metrics respectively yield rather similar a true value of cut-point but the Youden-based utility metrics suggest rather similarly the true value of for an optimal cut-point. In opposition, the Youden-based utility metric and the absolute difference of total utility with 2 times of AUC produce extremely high value for optimal cut-point because of their s-shaped metrics over various cut-off values. As prevalence increases to 10% or more, the metric of Youden -based utility becomes concave and its cut-point becomes closer to other methods. The four proposed methods yield roughly identical cut-point at prevalence of 10% or more for high accuracy of 0.90. The greater discrepancy of optimal cut-point was shown in skew distributions of bigamma and biexponential with low prevalence and low AUC. For prevalence < 10%, the utility-based produces larger cut-point than accuracy-based methods in our clinical data for CRP. The methods of utility-based cut-point selection were explained by CRP in predicting preeclampsia, and other clinical data.

Conclusion: The inconsistency of optimal cut-points is possible by different methods of utility-based criteria depending on the prevalence and degree of AUC. For high AUC, and prevalence > 10%, the four proposed methods yield rather identical optimal cut-points. Further studies of simulation are needed to evaluate the bias and sampling variability of utility-based of cut-point selection.