Factor Analysis: Dealing with Response Bias

Abstract

This paper proposes an innovative method for factor analyzing data that potentially contains individual response bias. Past methods include the use of “ipsative” data, or, related to that, “ipsatized” data. Unfortunately, factor analysis as the main method used for analyzing the dimensionality of data, cannot be applied to ipsative data. In contrast, normalization of data as an alternative method to filter out response bias, is not hampered by the technical statistical issues inherent to applying multivariate techniques to ipsative data. Using high-quality data from a survey in Nepal that makes use of – among others – the High-Performance Organizations (HPO) framework, this paper shows that the traditional approach of directly applying Confirmatory Factor Analysis (CFA) starting from an existing model or theory, is inferior to our approach. Even applying Exploratory Factor Analysis (EFA) to the raw (non-normalized) data before using CFA, is unable to detect the optimal dimensionality, or structure, in the data. A better structure can be obtained by performing EFA on normalized data that corrects for response bias in the raw data. This paper convincingly shows that the newly identified structure is superior to the original structure suggested by the HPO framework. Applying a CFA using the newly detected structure on the raw data, gives excellent goodness-of-fit statistics, with more items retained, and no need of forced methods to improve the model fit. The findings suggest that existing models and questionnaires based on these models, are not necessarily as valid and reliable as empirical studies that make use of traditional analyses seem to suggest. When adopting existing instruments, researchers are advised to critically check the validity and reliability of these instruments – especially those vulnerable to response bias - and to apply the procedures laid out in this paper, in order to enhance the quality of their research, and to inform future researchers who consider using the same instruments or to warn them about the potential shortcomings of these instruments.