Tree-based item-response theory model for evaluating differential item functioning in patient-reported outcome measures: a web-based R Shiny implementation.

Abstract

Purpose: The validity of inferences from patient-reported outcome measure (PROM) scores can be confounded by differential item functioning (DIF). DIF occurs when there is heterogeneity in how patients respond to and interpret questions about their health, despite having the same underlying health status. Ignoring the effects of DIF could lead to inaccurate interpretations and misinformed clinical decisions resulting in compromised healthcare delivery. Tree-based item response theory (IRT) models are recommended as an alternative class of methods for analyzing PROMs because they offer a robust approach for identifying DIF when covariates associated with DIF are unknown a priori.

Methods: This paper introduces a web application developed using R Shiny, which enables users to implement tree-based IRT models for DIF assessment in potentially heterogeneous populations. The app provides flexible model specifications, visualization tools, and customizable settings to accommodate various data types and research needs. A practical tutorial is included, guiding users through the application interface, data preparation, model selection, and interpretation of results.

Results: The web application (https://ucalgary-pcma-lab.shinyapps.io/tree_based_dif_analysis/) offers interactive data upload in .CSV and .XLSX data formats. Recommendations are provided for selecting model parameters within the app based on the results of previous simulation studies. The web app tests for DIF on dichotomous- and polytomous-scored items. The coefficients, item parameters, and plots provide insights into potential sources of DIF.

Conclusion: This web application provides a user-friendly, interactive, innovative, easily accessible, and valuable tool for clinicians, applied health researchers, and analysts seeking to understand sample heterogeneity due to DIF in PROM data.