{"title":"那么,它比其他东西更好吗?用随机效应荟萃分析的结果来描述效应大小的百分位数。","authors":"Peter Boedeker,Gena Nelson,Hannah Carter","doi":"10.1037/met0000704","DOIUrl":null,"url":null,"abstract":"The characterization of an effect size is best made in reference to effect sizes found in the literature. A random-effects meta-analysis is the systematic synthesis of related effects from across a literature, producing an estimate of the distribution of effects in the population. We propose using the estimated mean and variance from a random-effects meta-analysis to inform the characterization of an observed effect size. The percentile of an observed effect size within the estimated distribution of population effects can describe the magnitude of the observed effect. Because there is uncertainty in the population estimates, we propose using the prediction distribution (used frequently to estimate the prediction interval in a meta-analysis) to serve as the reference distribution when characterizing an effect size. Doing so, the percentile of an observed effect and the limits of the effect size's 95% confidence interval within the prediction distribution are calculated. With numerous meta-analyses available including various outcomes and contexts, the presented method can be useful to many researchers and practitioners. We demonstrate the application of an easy-to-use Excel worksheet to automate these percentile calculations. We follow this with a simulation study evaluating the method's performance over a range of conditions. Recommendations (and cautions) for meta-analysts and researchers conducting a single study are provided. (PsycInfo Database Record (c) 2024 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":null,"pages":null},"PeriodicalIF":7.6000,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"So is it better than something else? Using the results of a random-effects meta-analysis to characterize the magnitude of an effect size as a percentile.\",\"authors\":\"Peter Boedeker,Gena Nelson,Hannah Carter\",\"doi\":\"10.1037/met0000704\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The characterization of an effect size is best made in reference to effect sizes found in the literature. A random-effects meta-analysis is the systematic synthesis of related effects from across a literature, producing an estimate of the distribution of effects in the population. We propose using the estimated mean and variance from a random-effects meta-analysis to inform the characterization of an observed effect size. The percentile of an observed effect size within the estimated distribution of population effects can describe the magnitude of the observed effect. Because there is uncertainty in the population estimates, we propose using the prediction distribution (used frequently to estimate the prediction interval in a meta-analysis) to serve as the reference distribution when characterizing an effect size. Doing so, the percentile of an observed effect and the limits of the effect size's 95% confidence interval within the prediction distribution are calculated. With numerous meta-analyses available including various outcomes and contexts, the presented method can be useful to many researchers and practitioners. We demonstrate the application of an easy-to-use Excel worksheet to automate these percentile calculations. We follow this with a simulation study evaluating the method's performance over a range of conditions. Recommendations (and cautions) for meta-analysts and researchers conducting a single study are provided. (PsycInfo Database Record (c) 2024 APA, all rights reserved).\",\"PeriodicalId\":20782,\"journal\":{\"name\":\"Psychological methods\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":7.6000,\"publicationDate\":\"2024-09-09\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Psychological methods\",\"FirstCategoryId\":\"102\",\"ListUrlMain\":\"https://doi.org/10.1037/met0000704\",\"RegionNum\":1,\"RegionCategory\":\"心理学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"PSYCHOLOGY, MULTIDISCIPLINARY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Psychological methods","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1037/met0000704","RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHOLOGY, MULTIDISCIPLINARY","Score":null,"Total":0}
So is it better than something else? Using the results of a random-effects meta-analysis to characterize the magnitude of an effect size as a percentile.
The characterization of an effect size is best made in reference to effect sizes found in the literature. A random-effects meta-analysis is the systematic synthesis of related effects from across a literature, producing an estimate of the distribution of effects in the population. We propose using the estimated mean and variance from a random-effects meta-analysis to inform the characterization of an observed effect size. The percentile of an observed effect size within the estimated distribution of population effects can describe the magnitude of the observed effect. Because there is uncertainty in the population estimates, we propose using the prediction distribution (used frequently to estimate the prediction interval in a meta-analysis) to serve as the reference distribution when characterizing an effect size. Doing so, the percentile of an observed effect and the limits of the effect size's 95% confidence interval within the prediction distribution are calculated. With numerous meta-analyses available including various outcomes and contexts, the presented method can be useful to many researchers and practitioners. We demonstrate the application of an easy-to-use Excel worksheet to automate these percentile calculations. We follow this with a simulation study evaluating the method's performance over a range of conditions. Recommendations (and cautions) for meta-analysts and researchers conducting a single study are provided. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
期刊介绍:
Psychological Methods is devoted to the development and dissemination of methods for collecting, analyzing, understanding, and interpreting psychological data. Its purpose is the dissemination of innovations in research design, measurement, methodology, and quantitative and qualitative analysis to the psychological community; its further purpose is to promote effective communication about related substantive and methodological issues. The audience is expected to be diverse and to include those who develop new procedures, those who are responsible for undergraduate and graduate training in design, measurement, and statistics, as well as those who employ those procedures in research.