{"title":"序列方差分析:提高假设检验的效率","authors":"Meike Steinhilber,Martin Schnuerch,Anna-Lena Schubert","doi":"10.1037/met0000677","DOIUrl":null,"url":null,"abstract":"Researchers commonly use analysis of variance (ANOVA) to statistically test results of factorial designs. Performing an a priori power analysis is crucial to ensure that the ANOVA is sufficiently powered, however, it often poses a challenge and can result in large sample sizes, especially if the expected effect size is small. Due to the high prevalence of small effect sizes in psychology, studies are frequently underpowered as it is often economically unfeasible to gather the necessary sample size for adequate Type-II error control. Here, we present a more efficient alternative to the fixed ANOVA, the so-called sequential ANOVA that we implemented in the R package \"sprtt.\" The sequential ANOVA is based on the sequential probability ratio test (SPRT) that uses a likelihood ratio as a test statistic and controls for long-term error rates. SPRTs gather evidence for both the null and the alternative hypothesis and conclude this process when a sufficient amount of evidence has been gathered to accept one of the two hypotheses. Through simulations, we show that the sequential ANOVA is more efficient than the fixed ANOVA and reliably controls long-term error rates. Additionally, robustness analyses revealed that the sequential and fixed ANOVAs exhibit analogous properties when their underlying assumptions are violated. Taken together, our results demonstrate that the sequential ANOVA is an efficient alternative to fixed sample designs for hypothesis testing. (PsycInfo Database Record (c) 2024 APA, all rights reserved).","PeriodicalId":20782,"journal":{"name":"Psychological methods","volume":"49 1","pages":""},"PeriodicalIF":7.6000,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Sequential analysis of variance: Increasing efficiency of hypothesis testing.\",\"authors\":\"Meike Steinhilber,Martin Schnuerch,Anna-Lena Schubert\",\"doi\":\"10.1037/met0000677\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Researchers commonly use analysis of variance (ANOVA) to statistically test results of factorial designs. Performing an a priori power analysis is crucial to ensure that the ANOVA is sufficiently powered, however, it often poses a challenge and can result in large sample sizes, especially if the expected effect size is small. Due to the high prevalence of small effect sizes in psychology, studies are frequently underpowered as it is often economically unfeasible to gather the necessary sample size for adequate Type-II error control. Here, we present a more efficient alternative to the fixed ANOVA, the so-called sequential ANOVA that we implemented in the R package \\\"sprtt.\\\" The sequential ANOVA is based on the sequential probability ratio test (SPRT) that uses a likelihood ratio as a test statistic and controls for long-term error rates. SPRTs gather evidence for both the null and the alternative hypothesis and conclude this process when a sufficient amount of evidence has been gathered to accept one of the two hypotheses. Through simulations, we show that the sequential ANOVA is more efficient than the fixed ANOVA and reliably controls long-term error rates. Additionally, robustness analyses revealed that the sequential and fixed ANOVAs exhibit analogous properties when their underlying assumptions are violated. Taken together, our results demonstrate that the sequential ANOVA is an efficient alternative to fixed sample designs for hypothesis testing. (PsycInfo Database Record (c) 2024 APA, all rights reserved).\",\"PeriodicalId\":20782,\"journal\":{\"name\":\"Psychological methods\",\"volume\":\"49 1\",\"pages\":\"\"},\"PeriodicalIF\":7.6000,\"publicationDate\":\"2024-09-09\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Psychological methods\",\"FirstCategoryId\":\"102\",\"ListUrlMain\":\"https://doi.org/10.1037/met0000677\",\"RegionNum\":1,\"RegionCategory\":\"心理学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"PSYCHOLOGY, MULTIDISCIPLINARY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Psychological methods","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1037/met0000677","RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHOLOGY, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0
摘要
研究人员通常使用方差分析(ANOVA)来统计检验因子设计的结果。进行先验功率分析对于确保方差分析具有足够的功率至关重要,但这往往是一个挑战,可能会导致样本量过大,尤其是在预期效应大小较小的情况下。由于心理学中普遍存在小效应量的情况,因此研究往往动力不足,因为要收集足够的样本量来进行适当的 II 类误差控制,在经济上往往是不可行的。在这里,我们提出了一种比固定方差分析更有效的替代方法,即我们在 R 软件包 "sprtt "中实现的所谓序列方差分析。序列方差分析基于序列概率比检验(SPRT),它使用似然比作为检验统计量,并控制长期误差率。SPRT 为零假设和备择假设收集证据,当收集到足够的证据可以接受两个假设中的一个时,就结束这一过程。通过模拟,我们发现顺序方差分析比固定方差分析更有效,而且能可靠地控制长期错误率。此外,稳健性分析表明,当违反基本假设时,顺序方差分析和固定方差分析表现出类似的特性。综上所述,我们的研究结果表明,序列方差分析是固定样本设计假设检验的有效替代方案。(PsycInfo Database Record (c) 2024 APA, 版权所有)。
Sequential analysis of variance: Increasing efficiency of hypothesis testing.
Researchers commonly use analysis of variance (ANOVA) to statistically test results of factorial designs. Performing an a priori power analysis is crucial to ensure that the ANOVA is sufficiently powered, however, it often poses a challenge and can result in large sample sizes, especially if the expected effect size is small. Due to the high prevalence of small effect sizes in psychology, studies are frequently underpowered as it is often economically unfeasible to gather the necessary sample size for adequate Type-II error control. Here, we present a more efficient alternative to the fixed ANOVA, the so-called sequential ANOVA that we implemented in the R package "sprtt." The sequential ANOVA is based on the sequential probability ratio test (SPRT) that uses a likelihood ratio as a test statistic and controls for long-term error rates. SPRTs gather evidence for both the null and the alternative hypothesis and conclude this process when a sufficient amount of evidence has been gathered to accept one of the two hypotheses. Through simulations, we show that the sequential ANOVA is more efficient than the fixed ANOVA and reliably controls long-term error rates. Additionally, robustness analyses revealed that the sequential and fixed ANOVAs exhibit analogous properties when their underlying assumptions are violated. Taken together, our results demonstrate that the sequential ANOVA is an efficient alternative to fixed sample designs for hypothesis testing. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
期刊介绍:
Psychological Methods is devoted to the development and dissemination of methods for collecting, analyzing, understanding, and interpreting psychological data. Its purpose is the dissemination of innovations in research design, measurement, methodology, and quantitative and qualitative analysis to the psychological community; its further purpose is to promote effective communication about related substantive and methodological issues. The audience is expected to be diverse and to include those who develop new procedures, those who are responsible for undergraduate and graduate training in design, measurement, and statistics, as well as those who employ those procedures in research.