Power analysis to detect misfit in SEMs with many items: Resolving unrecognized problems, relating old and new approaches, and "matching" power analysis approach to data analysis approach.

Abstract

It is unappreciated that there are four different approaches to power analysis for detecting misspecification by testing overall fit of structural equation models (SEMs) and, moreover, that common approaches can yield radically diverging results for SEMs with many items (high p). Here we newly relate these four approaches. Analytical power analysis methods using theoretical null and theoretical alternative distributions (Approach 1) have a long history, are widespread, and are often contrasted with "the" Monte Carlo method-which is an oversimplification. Actually, three Monte Carlo methods can be distinguished; all use an empirical alternative distribution but differ regarding whether the null distribution is theoretical (Approach 2), empirical (Approach 3), or-as we newly propose and demonstrate the need for-adjusted empirical (Approach 4). Because these four approaches can yield radically diverging power results under high p (as demonstrated here), researchers need to "match" their a priori SEM power analysis approach to their later SEM data analysis approach for testing overall fit, once data are collected. Disturbingly, the most common power analysis approach for a global test-of-fit is mismatched with the most common data analysis approach for a global test-of-fit in SEM. Because of this mismatch, researchers' anticipated versus actual/obtained power can differ substantially. We explain how/why to "match" across power-analysis and data-analysis phases of a study and provide software to facilitate doing so. As extensions, we explain how to relate and implement all four approaches to power analysis (a) for testing overall fit using χ² versus root-mean-square error of approximation and (b) for testing overall fit versus testing a target parameter/effect. (PsycInfo Database Record (c) 2024 APA, all rights reserved).