How valid, really? A meta-analysis of the validity evidence of Technological Pedagogical and Content Knowledge (TPACK) self-report assessments

Abstract

Technological Pedagogical and Content Knowledge (TPACK) plays a central role for teacher training and measuring teachers’ preparedness for technology-enhanced teaching. Hence, crafting a validity argument of the popular TPACK self-assessments is key to facilitating evidence-based practices in these areas. However, the existing body of validity evidence abounds in divergent findings, for instance on the factor structure of TPACK self-reports across pre- and in-service teacher samples. To synthesize this evidence and capture its heterogeneity, we conducted a systematic review with a meta-analysis. First, we examined the internal validity of TPACK by analyzing its factor structure using meta-analytic structural equation and network modeling. Second, we compared two theoretical models describing the relations among the TPACK dimension—the transformative and integrative models—against the data. Third, we assessed the external validity of self-reported TPCK by examining its correlation with actual technology use in teaching. Our findings revealed two distinguishable dimensions of TPACK for both pre- and in-service teacher samples, which represented the technological and non-technological dimensions. The integrative model provided an overall better fit to the data. Moreover, TPACK self-reports were moderately correlated with technology use ($\bar{r}$ s = 0.23–0.49) and exhibited substantial heterogeneity. Our results underscore the importance of generating and considering both internal and external validity evidence when applying TPACK assessments to generate implications for educational research and practice.