Mediating effect of pubertal stages on the family environment and neurodevelopment: An open-data replication and multiverse analysis of an ABCD Study®

Abstract

Increasing evidence demonstrates that environmental factors meaningfully impact the development of the brain (Hyde et al., 2020; McEwen and Akil, 2020). Recent work from the Adolescent Brain Cognitive Development (ABCD) Study® suggests that puberty may indirectly account for some association between the family environment and brain structure and function (Thijssen et al., 2020). However, a limited number of large studies have evaluated what, how, and why environmental factors impact neurodevelopment. When these topics are investigated, there is typically inconsistent operationalization of variables between studies which may be measuring different aspects of the environment and thus different associations in the analytic models. Multiverse analyses (Steegen et al., 2016) are an efficacious technique for investigating the effect of different operationalizations of the same construct on underlying interpretations. While one of the assets of Thijssen et al. (2020) was its large sample from the ABCD data, the authors used an early release that contained 38% of the full ABCD sample. Then, the analyses used several ‘researcher degrees of freedom’ (Gelman and Loken, 2014) to operationalize key independent, mediating and dependent variables, including but not limited to, the use of a latent factor of preadolescents' environment comprised of different subfactors, such as parental monitoring and child-reported family conflict. While latent factors can improve reliability of constructs, the nuances of each subfactor and measure that comprise the environment may be lost, making the latent factors difficult to interpret in the context of individual differences. This study extends the work of Thijssen et al. (2020) by evaluating the extent to which the analytic choices in their study affected their conclusions. In Aim 1, using the same variables and models, we replicate findings from the original study using the full sample in Release 3.0. Then, in Aim 2, using a multiverse analysis we extend findings by considering nine alternative operationalizations of family environment, three of puberty, and five of brain measures (total of 135 models) to evaluate the impact on conclusions from Aim 1. In these results, 90% of the directions of effects and 60% of the p-values (e.g. p > .05 and p < .05) across effects were comparable between the two studies. However, raters agreed that only 60% of the effects had replicated. Across the multiverse analyses, there was a degree of variability in beta estimates across the environmental variables, and lack of consensus between parent reported and child reported pubertal development for the indirect effects. This study demonstrates the challenge in defining which effects replicate, the nuance across environmental variables in the ABCD data, and the lack of consensus across parent and child reported puberty scales in youth.