Harmonizing bifactor models of psychopathology between distinct assessment instruments: Reliability, measurement invariance, and authenticity

Objectives

Model configuration is important for mental health data harmonization. We provide a method to investigate the performance of different bifactor model configurations to harmonize different instruments.

Methods

We used data from six samples from the Reproducible Brain Charts initiative (N = 8,606, ages 5–22 years, 41.0% females). We harmonized items from two psychopathology instruments, Child Behavior Checklist (CBCL) and GOASSESS, based on semantic content. We estimated bifactor models using confirmatory factor analysis, and calculated their model fit, factor reliability, between-instrument invariance, and authenticity (i.e., the correlation and factor score difference between the harmonized and original models).

Results

Five out of 12 model configurations presented acceptable fit and were instrument-invariant. Correlations between the harmonized factor scores and the original full-item models were high for the p-factor (>0.89) and small to moderate (0.12–0.81) for the specific factors. 6.3%–50.9% of participants presented factor score differences between harmonized and original models higher than 0.5 z-score.

Conclusions

The CBCL-GOASSESS harmonization indicates that few models provide reliable specific factors and are instrument-invariant. Moreover, authenticity was high for the p-factor and moderate for specific factors. Future studies can use this framework to examine the impact of harmonizing instruments in psychiatric research.