Best practices in multilevel modeling for within-cluster group comparisons: An evaluation of coding strategies reflecting group composition and heterogeneity.
{"title":"Best practices in multilevel modeling for within-cluster group comparisons: An evaluation of coding strategies reflecting group composition and heterogeneity.","authors":"Qian Zhang, Xiao Liu, Zijun Ke","doi":"10.1037/met0000836","DOIUrl":null,"url":null,"abstract":"<p><p>In multilevel analysis, Level-1 dichotomous predictors (e.g., treatment vs. control, minority vs. majority; <i>D</i>) are commonly used to examine the existence of average within-cluster mean differences. To address this research question, we examine the best practices for coding <i>D</i>, considering within-cluster group compositions and between-cluster heterogeneity in them, as well as possible between-group heterogeneity in residual variances. We compare traditional dummy coding and unweighted effect coding with two proposed methods: overall and within-cluster weighted effect coding. Each is combined with cluster-mean centering to clearly define the within-cluster mean difference. Analytical results show that the intraclass correlation of <i>D</i> (ICC-D) substantially influences differences across coding methods regarding test results for the average within-cluster effect (group mean difference). When ICC-D is zero, all strategies yield identical test results (i.e., the significance tests based on the point estimates and standard error estimates) for the average within-cluster effect. However, as the ICC-D increases, test statistics (e.g., the t statistic) decrease for all coding strategies. An empirical study and simulations further confirm that ICC-D strongly influences results. Based on the simulations, we recommend dummy coding, unweighted effect coding, and overall weighted effect coding, which generally provide comparable or higher power than within-cluster weighted effect coding. The power contrast is most pronounced when both ICC-D and Level-1 residual variance heterogeneity increase, with the difference in power reaching up to 40%. Furthermore, cluster-mean centering not only disaggregates within- and between-cluster effects but also achieves generally reasonable Type I error rates even when homogeneity assumptions are not met. (PsycInfo Database Record (c) 2026 APA, all rights reserved).</p>","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":" ","pages":"1218-1244"},"PeriodicalIF":7.5000,"publicationDate":"2026-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Psychological methods","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1037/met0000836","RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2026/6/15 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"PSYCHOLOGY, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0
Abstract
In multilevel analysis, Level-1 dichotomous predictors (e.g., treatment vs. control, minority vs. majority; D) are commonly used to examine the existence of average within-cluster mean differences. To address this research question, we examine the best practices for coding D, considering within-cluster group compositions and between-cluster heterogeneity in them, as well as possible between-group heterogeneity in residual variances. We compare traditional dummy coding and unweighted effect coding with two proposed methods: overall and within-cluster weighted effect coding. Each is combined with cluster-mean centering to clearly define the within-cluster mean difference. Analytical results show that the intraclass correlation of D (ICC-D) substantially influences differences across coding methods regarding test results for the average within-cluster effect (group mean difference). When ICC-D is zero, all strategies yield identical test results (i.e., the significance tests based on the point estimates and standard error estimates) for the average within-cluster effect. However, as the ICC-D increases, test statistics (e.g., the t statistic) decrease for all coding strategies. An empirical study and simulations further confirm that ICC-D strongly influences results. Based on the simulations, we recommend dummy coding, unweighted effect coding, and overall weighted effect coding, which generally provide comparable or higher power than within-cluster weighted effect coding. The power contrast is most pronounced when both ICC-D and Level-1 residual variance heterogeneity increase, with the difference in power reaching up to 40%. Furthermore, cluster-mean centering not only disaggregates within- and between-cluster effects but also achieves generally reasonable Type I error rates even when homogeneity assumptions are not met. (PsycInfo Database Record (c) 2026 APA, all rights reserved).
期刊介绍：
Psychological Methods is devoted to the development and dissemination of methods for collecting, analyzing, understanding, and interpreting psychological data. Its purpose is the dissemination of innovations in research design, measurement, methodology, and quantitative and qualitative analysis to the psychological community; its further purpose is to promote effective communication about related substantive and methodological issues. The audience is expected to be diverse and to include those who develop new procedures, those who are responsible for undergraduate and graduate training in design, measurement, and statistics, as well as those who employ those procedures in research.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
