When a Truly Positive Correlation Turns Negative: How Different Approaches to Model Hierarchically Structured Constructs Affect Estimated Correlations to Covariates

Many constructs in personality psychology assume a hierarchical structure positing a general factor along with several narrower subdimensions or facets. Different approaches are commonly used to model such a structure, including higher-order factor models, bifactor models, single-factor models based on the responses on the observed items, and single-factor models based on parcels computed from the mean observed scores on the subdimensions. The present article investigates the consequences of adopting a certain approach for the validity of conclusions derived from the thereby obtained correlation of the most general factor to a covariate. Any of the considered approaches may closely approximate the true correlation when its underlying assumptions are met or when model misspecifications only pertain to the measurement model of the hierarchical construct. However, when misspecifications involve nonmodeled covariances between parts of the hierarchically structured construct and the covariate, higher-order models, single-factor representations, and facet-parcel approaches can yield severely biased estimates sometimes grossly misrepresenting the true correlation and even incurring sign changes. In contrast, a bifactor approach proved to be most robust and to provide rather unbiased results under all conditions. The implications are discussed and recommendations are provided.