Statistical Science最新文献

{"title":"On the mixed-model analysis of covariance in cluster-randomized trials.","authors":"Bingkai Wang, Michael O Harhay, Jiaqi Tong, Dylan S Small, Tim P Morris, Fan Li","doi":"10.1214/24-sts944","DOIUrl":"10.1214/24-sts944","url":null,"abstract":"<p><p>In the analyses of cluster-randomized trials, mixed-model analysis of covariance (ANCOVA) is a standard approach for covariate adjustment and handling within-cluster correlations. However, when the normality, linearity, or the random-intercept assumption is violated, the validity and efficiency of the mixed-model ANCOVA estimators for estimating the average treatment effect remain unclear. Under the potential outcomes framework, we prove that the mixed-model ANCOVA estimators for the average treatment effect are consistent and asymptotically normal under arbitrary misspecification of its working model. If the probability of receiving treatment is 0.5 for each cluster, we further show that the model-based variance estimator under mixed-model ANCOVA1 (ANCOVA without treatment-covariate interactions) remains consistent, clarifying that the confidence interval given by standard software is asymptotically valid even under model misspecification. Beyond robustness, we discuss several insights on precision among classical methods for analyzing cluster-randomized trials, including the mixed-model ANCOVA, individual-level ANCOVA, and cluster-level ANCOVA estimators. These insights may inform the choice of methods in practice. Our analytical results and insights are illustrated via simulation studies and analyses of three cluster-randomized trials.</p>","PeriodicalId":51172,"journal":{"name":"Statistical Science","volume":"41 1","pages":"49-68"},"PeriodicalIF":6.6,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12803445/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145991759","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Replicable Bandits for Digital Health Interventions.","authors":"Kelly W Zhang, Nowell Closser, Anna L Trella, Susan A Murphy","doi":"10.1214/25-sts1017","DOIUrl":"10.1214/25-sts1017","url":null,"abstract":"<p><p>Adaptive treatment assignment algorithms, such as bandit algorithms, are increasingly used in digital health intervention clinical trials. Frequently the data collected from these trials is used to conduct causal inference and related data analyses to decide how to refine the intervention, and whether to roll-out the intervention more broadly. This work studies inference for estimands that depend on the adaptive algorithm itself; a simple example is the mean reward under the adaptive algorithm. Specifically, we investigate the replicability of statistical analyses concerning such estimands when using data from trials deploying adaptive treatment assignment algorithms. We demonstrate that many standard statistical estimators can be inconsistent and fail to be replicable across repetitions of the clinical trial, even as the sample size grows large. We show that this non-replicability is intimately related to properties of the adaptive algorithm itself. We introduce a formal definition of a \"replicable bandit algorithm\" and prove that under such algorithms, a wide variety of common statistical estimators are guaranteed to be consistent and asymptotically normal. We present both theoretical results and simulation studies based on a mobile health oral health self-care intervention. Our findings underscore the importance of designing adaptive algorithms with replicability in mind, especially for settings like digital health where deployment decisions rely heavily on replicated evidence. We conclude by discussing open questions on the connections between algorithm design, statistical inference, and experimental replicability.</p>","PeriodicalId":51172,"journal":{"name":"Statistical Science","volume":"40 4","pages":"671-691"},"PeriodicalIF":6.6,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12799202/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145971483","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Bayesian Transfer Learning.","authors":"Piotr M Suder, Jason Xu, David B Dunson","doi":"10.1214/25-STS987","DOIUrl":"10.1214/25-STS987","url":null,"abstract":"<p><p>Transfer learning is a burgeoning concept in statistical machine learning that seeks to improve inference and/or predictive accuracy on a domain of interest by leveraging data from related domains. While the term \"transfer learning\" has garnered much recent interest, its foundational principles have existed for years under various guises. Prior literature reviews in computer science and electrical engineering have sought to bring these ideas into focus, primarily by surveying general methodologies and works from these disciplines. This article highlights Bayesian approaches to transfer learning, which have received relatively limited attention despite their innate compatibility with the notion of drawing upon prior knowledge to guide new learning tasks. Our survey encompasses a wide range of Bayesian transfer learning frameworks applicable to a variety of practical settings. We discuss how these methods address the problem of finding the optimal information to transfer between domains, which is a central question in transfer learning. We illustrate the utility of Bayesian transfer learning methods via a simulation study where we compare performance against frequentist competitors.</p>","PeriodicalId":51172,"journal":{"name":"Statistical Science","volume":"40 3","pages":"477-494"},"PeriodicalIF":6.6,"publicationDate":"2025-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13029020/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147576490","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"On the Use of Auxiliary Variables in Multilevel Regression and Poststratification.","authors":"Yajuan Si","doi":"10.1214/24-sts932","DOIUrl":"10.1214/24-sts932","url":null,"abstract":"<p><p>Multilevel regression and poststratification (MRP) is a popular method for addressing selection bias in subgroup estimation, with broad applications across fields from social sciences to public health. In this paper, we examine the inferential validity of MRP in finite populations, exploring the impact of poststratification and model specification. The success of MRP relies heavily on the availability of auxiliary information that is strongly related to the outcome. To enhance the fitting performance of the outcome model, we recommend modeling the inclusion probabilities conditionally on auxiliary variables and incorporating flexible functions of estimated inclusion probabilities as predictors in the mean structure. We present a statistical data integration framework that offers robust inferences for probability and nonprobability surveys, addressing various challenges in practical applications. Our simulation studies indicate the statistical validity of MRP, which involves a tradeoff between bias and variance, with greater benefits for subgroup estimates with small sample sizes, compared to alternative methods. We have applied our methods to the Adolescent Brain Cognitive Development (ABCD) Study, which collected information on children across 21 geographic locations in the U.S. to provide national representation, but is subject to selection bias as a nonprobability sample. We focus on the cognition measure of diverse groups of children in the ABCD study and show that the use of auxiliary variables affects the findings on cognitive performance.</p>","PeriodicalId":51172,"journal":{"name":"Statistical Science","volume":"40 2","pages":"272-288"},"PeriodicalIF":3.9,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12140408/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144235908","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Scalable Empirical Bayes Inference and Bayesian Sensitivity Analysis.","authors":"Hani Doss, Antonio Linero","doi":"10.1214/24-sts936","DOIUrl":"10.1214/24-sts936","url":null,"abstract":"&lt;p&gt;&lt;p&gt;Consider a Bayesian setup in which we observe &lt;math&gt;&lt;mi&gt;Y&lt;/mi&gt;&lt;/math&gt; , whose distribution depends on a parameter &lt;math&gt;&lt;mi&gt;θ&lt;/mi&gt;&lt;/math&gt; , that is, &lt;math&gt;&lt;mi&gt;Y&lt;/mi&gt; &lt;mo&gt;∣&lt;/mo&gt; &lt;mi&gt;θ&lt;/mi&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;mo&gt;~&lt;/mo&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;msub&gt;&lt;mrow&gt;&lt;mi&gt;π&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mi&gt;Y&lt;/mi&gt; &lt;mo&gt;∣&lt;/mo&gt; &lt;mi&gt;θ&lt;/mi&gt;&lt;/mrow&gt; &lt;/msub&gt; &lt;/math&gt; . The parameter &lt;math&gt;&lt;mi&gt;θ&lt;/mi&gt;&lt;/math&gt; is unknown and treated as random, and a prior distribution chosen from some parametric family &lt;math&gt; &lt;mfenced&gt; &lt;mrow&gt; &lt;msub&gt;&lt;mrow&gt;&lt;mi&gt;π&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mi&gt;θ&lt;/mi&gt;&lt;/mrow&gt; &lt;/msub&gt; &lt;mo&gt;(&lt;/mo&gt; &lt;mo&gt;⋅&lt;/mo&gt; &lt;mo&gt;;&lt;/mo&gt; &lt;mi&gt;h&lt;/mi&gt; &lt;mo&gt;)&lt;/mo&gt; &lt;mo&gt;,&lt;/mo&gt; &lt;mi&gt;h&lt;/mi&gt; &lt;mo&gt;∈&lt;/mo&gt; &lt;mi&gt;ℋ&lt;/mi&gt;&lt;/mrow&gt; &lt;/mfenced&gt; &lt;/math&gt; , is to be placed on it. For the subjective Bayesian there is a single prior in the family which represents his or her beliefs about &lt;math&gt;&lt;mi&gt;θ&lt;/mi&gt;&lt;/math&gt; , but determination of this prior is very often extremely difficult. In the empirical Bayes approach, the latent distribution on &lt;math&gt;&lt;mi&gt;θ&lt;/mi&gt;&lt;/math&gt; is estimated from the data. This is usually done by choosing the value of the hyperparameter &lt;math&gt;&lt;mi&gt;h&lt;/mi&gt;&lt;/math&gt; that maximizes some criterion. Arguably the most common way of doing this is to let &lt;math&gt;&lt;mi&gt;m&lt;/mi&gt; &lt;mo&gt;(&lt;/mo&gt; &lt;mi&gt;h&lt;/mi&gt; &lt;mo&gt;)&lt;/mo&gt;&lt;/math&gt; be the marginal likelihood of &lt;math&gt;&lt;mi&gt;h&lt;/mi&gt;&lt;/math&gt; , that is, &lt;math&gt;&lt;mi&gt;m&lt;/mi&gt; &lt;mo&gt;(&lt;/mo&gt; &lt;mi&gt;h&lt;/mi&gt; &lt;mo&gt;)&lt;/mo&gt; &lt;mo&gt;=&lt;/mo&gt; &lt;mo&gt;∫&lt;/mo&gt; &lt;msub&gt;&lt;mrow&gt;&lt;mi&gt;π&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mi&gt;Y&lt;/mi&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;mo&gt;∣&lt;/mo&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;mi&gt;θ&lt;/mi&gt;&lt;/mrow&gt; &lt;/msub&gt; &lt;msub&gt;&lt;mrow&gt;&lt;mi&gt;v&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mi&gt;h&lt;/mi&gt;&lt;/mrow&gt; &lt;/msub&gt; &lt;mo&gt;(&lt;/mo&gt; &lt;mi&gt;θ&lt;/mi&gt; &lt;mo&gt;)&lt;/mo&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;mi&gt;d&lt;/mi&gt; &lt;mi&gt;θ&lt;/mi&gt;&lt;/math&gt; , and choose the value of &lt;math&gt;&lt;mi&gt;h&lt;/mi&gt;&lt;/math&gt; that maximizes &lt;math&gt;&lt;mi&gt;m&lt;/mi&gt; &lt;mo&gt;(&lt;/mo&gt; &lt;mo&gt;⋅&lt;/mo&gt; &lt;mo&gt;)&lt;/mo&gt;&lt;/math&gt; . Unfortunately, except for a handful of textbook examples, analytic evaluation of &lt;math&gt;&lt;mi&gt;a&lt;/mi&gt; &lt;mi&gt;r&lt;/mi&gt; &lt;mi&gt;g&lt;/mi&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;msub&gt;&lt;mrow&gt;&lt;mi&gt;m&lt;/mi&gt; &lt;mi&gt;a&lt;/mi&gt; &lt;mi&gt;x&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mi&gt;h&lt;/mi&gt;&lt;/mrow&gt; &lt;/msub&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;mi&gt;m&lt;/mi&gt; &lt;mo&gt;(&lt;/mo&gt; &lt;mi&gt;h&lt;/mi&gt; &lt;mo&gt;)&lt;/mo&gt;&lt;/math&gt; is not feasible. The purpose of this paper is two-fold. First, we review the literature on estimating it and find that the most commonly used procedures are either potentially highly inaccurate or don't scale well with the dimension of &lt;math&gt;&lt;mi&gt;h&lt;/mi&gt;&lt;/math&gt; , the dimension of &lt;math&gt;&lt;mi&gt;θ&lt;/mi&gt;&lt;/math&gt; , or both. Second, we present a method for estimating &lt;math&gt;&lt;mi&gt;a&lt;/mi&gt; &lt;mi&gt;r&lt;/mi&gt; &lt;mi&gt;g&lt;/mi&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;msub&gt;&lt;mrow&gt;&lt;mi&gt;m&lt;/mi&gt; &lt;mi&gt;a&lt;/mi&gt; &lt;mi&gt;x&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mi&gt;h&lt;/mi&gt;&lt;/mrow&gt; &lt;/msub&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;mi&gt;m&lt;/mi&gt; &lt;mo&gt;(&lt;/mo&gt; &lt;mi&gt;h&lt;/mi&gt; &lt;mo&gt;)&lt;/mo&gt;&lt;/math&gt; , based on Markov chain Monte Carlo, that applies very generally and scales well with dimension. Let &lt;math&gt;&lt;mi&gt;g&lt;/mi&gt;&lt;/math&gt; be a real-valued function of &lt;math&gt;&lt;mi&gt;θ&lt;/mi&gt;&lt;/math&gt; , and let &lt;math&gt;&lt;mi&gt;I&lt;/mi&gt; &lt;mo&gt;(&lt;/mo&gt; &lt;mi&gt;h&lt;/mi&gt; &lt;mo&gt;)&lt;/mo&gt;&lt;/math&gt; be the posterior expectation of &lt;math&gt;&lt;mi&gt;g&lt;/mi&gt; &lt;mo&gt;(&lt;/mo&gt; &lt;mi&gt;θ&lt;/mi&gt; &lt;mo&gt;)&lt;/mo&gt;&lt;/math&gt; when the prior is &lt;math&gt; &lt;msub&gt;&lt;mrow&gt;&lt;m","PeriodicalId":51172,"journal":{"name":"Statistical Science","volume":"39 4","pages":"601-622"},"PeriodicalIF":3.9,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11654829/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142856550","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Variable Selection Using Bayesian Additive Regression Trees.","authors":"Chuji Luo, Michael J Daniels","doi":"10.1214/23-sts900","DOIUrl":"10.1214/23-sts900","url":null,"abstract":"<p><p>Variable selection is an important statistical problem. This problem becomes more challenging when the candidate predictors are of mixed type (e.g. continuous and binary) and impact the response variable in nonlinear and/or non-additive ways. In this paper, we review existing variable selection approaches for the Bayesian additive regression trees (BART) model, a nonparametric regression model, which is flexible enough to capture the interactions between predictors and nonlinear relationships with the response. An emphasis of this review is on the ability to identify relevant predictors. We also propose two variable importance measures which can be used in a permutation-based variable selection approach, and a backward variable selection procedure for BART. We introduce these variations as a way of illustrating limitations and opportunities for improving current approaches and assess these via simulations.</p>","PeriodicalId":51172,"journal":{"name":"Statistical Science","volume":"39 2","pages":"286-304"},"PeriodicalIF":3.9,"publicationDate":"2024-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11395240/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142300349","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Causal Inference Methods for Combining Randomized Trials and Observational Studies: A Review.","authors":"Bénédicte Colnet, Imke Mayer, Guanhua Chen, Awa Dieng, Ruohong Li, Gaël Varoquaux, Jean-Philippe Vert, Julie Josse, Shu Yang","doi":"10.1214/23-sts889","DOIUrl":"10.1214/23-sts889","url":null,"abstract":"<p><p>With increasing data availability, causal effects can be evaluated across different data sets, both randomized controlled trials (RCTs) and observational studies. RCTs isolate the effect of the treatment from that of unwanted (confounding) co-occurring effects but they may suffer from unrepresentativeness, and thus lack external validity. On the other hand, large observational samples are often more representative of the target population but can conflate confounding effects with the treatment of interest. In this paper, we review the growing literature on methods for causal inference on combined RCTs and observational studies, striving for the best of both worlds. We first discuss identification and estimation methods that improve generalizability of RCTs using the representativeness of observational data. Classical estimators include weighting, difference between conditional outcome models and doubly robust estimators. We then discuss methods that combine RCTs and observational data to either ensure unconfoundedness of the observational analysis or to improve (conditional) average treatment effect estimation. We also connect and contrast works developed in both the potential outcomes literature and the structural causal model literature. Finally, we compare the main methods using a simulation study and real world data to analyze the effect of tranexamic acid on the mortality rate in major trauma patients. A review of available codes and new implementations is also provided.</p>","PeriodicalId":51172,"journal":{"name":"Statistical Science","volume":"39 1","pages":"165-191"},"PeriodicalIF":6.6,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12499922/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145245327","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Methods for Integrating Trials and Non-experimental Data to Examine Treatment Effect Heterogeneity","authors":"Carly Lupton Brantner, Ting-Hsuan Chang, Trang Quynh Nguyen, Hwanhee Hong, Leon Di Stefano, Elizabeth A. Stuart","doi":"10.1214/23-sts890","DOIUrl":"https://doi.org/10.1214/23-sts890","url":null,"abstract":"Estimating treatment effects conditional on observed covariates can improve the ability to tailor treatments to particular individuals. Doing so effectively requires dealing with potential confounding, and also enough data to adequately estimate effect moderation. A recent influx of work has looked into estimating treatment effect heterogeneity using data from multiple randomized controlled trials and/or observational datasets. With many new methods available for assessing treatment effect heterogeneity using multiple studies, it is important to understand which methods are best used in which setting, how the methods compare to one another, and what needs to be done to continue progress in this field. This paper reviews these methods broken down by data setting: aggregate-level data, federated learning, and individual participant-level data. We define the conditional average treatment effect and discuss differences between parametric and nonparametric estimators, and we list key assumptions, both those that are required within a single study and those that are necessary for data combination. After describing existing approaches, we compare and contrast them and reveal open areas for future research. This review demonstrates that there are many possible approaches for estimating treatment effect heterogeneity through the combination of datasets, but that there is substantial work to be done to compare these methods through case studies and simulations, extend them to different settings, and refine them to account for various challenges present in real data.","PeriodicalId":51172,"journal":{"name":"Statistical Science","volume":"19 2","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135515901","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Editorial: Special Issue on Reproducibility and Replicability","authors":"Alicia L Carriquiry, Michael J. Daniels, Nancy M Reid","doi":"10.1214/23-sts909","DOIUrl":"https://doi.org/10.1214/23-sts909","url":null,"abstract":"","PeriodicalId":51172,"journal":{"name":"Statistical Science","volume":"35 1","pages":""},"PeriodicalIF":5.7,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139295518","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Replication Success Under Questionable Research Practices—a Simulation Study","authors":"Francesca Freuli, Leonhard Held, Rachel Heyard","doi":"10.1214/23-sts904","DOIUrl":"https://doi.org/10.1214/23-sts904","url":null,"abstract":"Increasing evidence suggests that the reproducibility and replicability of scientific findings is threatened by researchers employing questionable research practices (QRPs) in order to achieve statistically significant results. Numerous metrics have been developed to determine replication success but it has not yet been investigated how well those metrics perform in the presence of QRPs. This paper aims to compare the performance of different metrics quantifying replication success in the presence of four types of QRPs: cherry picking of outcomes, questionable interim analyses, questionable inclusion of covariates, and questionable subgroup analyses. Our results show that the metric based on the version of the sceptical p-value that is recalibrated in terms of effect size performs better in maintaining low values of overall type-I error rate, but often requires larger replication sample sizes compared to metrics based on significance, the controlled version of the sceptical p-value, meta-analysis or Bayes factors, especially when severe QRPs are employed.","PeriodicalId":51172,"journal":{"name":"Statistical Science","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135515751","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}