Observational studiesPub Date : 2025-06-25eCollection Date: 2025-01-01DOI: 10.1353/obs.2025.a963648
Ian Shrier
{"title":"The interventionist approach can address questions related to causes of effects if causes are considered as states instead of interventions.","authors":"Ian Shrier","doi":"10.1353/obs.2025.a963648","DOIUrl":"https://doi.org/10.1353/obs.2025.a963648","url":null,"abstract":"<p><p>The interventionist approach to causal inference recommends that observational studies be framed as randomized trials with well-defined interventions to more precisely define the population of interest, exposure comparisons, assignment procedures, follow-up period, and outcomes. However, others suggest causes are not restricted to interventions, and the approach is too restrictive and will limit science. The described examples usually refer to questions about causes of effects/outcomes (rather than effects of interventions), where the 'cause' of interest often represents a mediator variable between an existing or hypothesized intervention. These questions are important because they represent the foundation for improving existing interventions and developing new interventions. In this article, I show how the interventionist approach can be used to try and answer these broader 'causes of effects/outcomes' questions. I use the sufficient casual set framework popularized by Rothman, which considers causes as states rather than interventions. The method is fully consistent with the potential outcomes approach and the need for well-defined counterfactuals. Whereas the effect of an intervention can theoretically be evaluated with an idealized single randomized trial, evaluating the causes of effects/outcomes requires evidence synthesis across multiple studies that each emulate a different target randomized trial. In other words, questions related to the effects of interventions are a special case where all the evidence that needs to be synthesized is available within a single idealized trial. Finally, considering states as causes also provides a transparent way to think about well-defined counterfactuals, and how factors that are commonly referred to as \"non-manipulable\" such as sex can also be studied as causes.</p>","PeriodicalId":74335,"journal":{"name":"Observational studies","volume":"11 2","pages":"189-208"},"PeriodicalIF":0.0,"publicationDate":"2025-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12959901/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147367526","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Observational studiesPub Date : 2025-06-25eCollection Date: 2025-01-01DOI: 10.1353/obs.2025.a963645
Nathaniel P Dowd, Bryan Blette, James D Chappell, Natasha B Halasa, Andrew J Spieker
{"title":"An overview of methods for receiver operating characteristic analysis, with an application to SARS-CoV-2 vaccine-induced humoral responses in solid organ transplant recipients.","authors":"Nathaniel P Dowd, Bryan Blette, James D Chappell, Natasha B Halasa, Andrew J Spieker","doi":"10.1353/obs.2025.a963645","DOIUrl":"https://doi.org/10.1353/obs.2025.a963645","url":null,"abstract":"<p><p>Receiver operating characteristic (ROC) analysis is a tool to evaluate the capacity of an ordinal or numeric measure to distinguish between states, often employed in the evaluation of diagnostic tests. Assumptions regarding the nature of the data or the structure of the ROC curve can improve efficiency of estimation for the ROC curve itself or its associated area under the curve (a measure of stochastic ordering closely related to the target parameter of a Mann-Whitney <math><mi>U</mi></math> test). Despite recent interest in methods to conduct comparisons by way of stochastic ordering, ROC modeling strategies are not widely known but may be appreciated by practitioners of these methods. The major goals of this manuscript are to (1) provide an overview of non-parametric, parametric, and semiparametric frameworks for ROC curve estimation, (2) guide the practice of ROC analysis by offering considerations regarding methodological trade-offs, and (3) supply sample R code to further guide implementation. We utilize simulations to demonstrate various features of the methods we discuss. As an illustrative example, we analyze data from a recent cohort study in order to compare responses to SARS-CoV-2 vaccination between solid organ transplant recipients and healthy individuals.</p>","PeriodicalId":74335,"journal":{"name":"Observational studies","volume":"11 2","pages":"91-212"},"PeriodicalIF":0.0,"publicationDate":"2025-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12959903/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147367555","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Observational studiesPub Date : 2025-06-25eCollection Date: 2025-01-01DOI: 10.1353/obs.2025.a963649
Nicole E Pashley
{"title":"Review of \"A First Course in Causal Inference\" by Peng Ding.","authors":"Nicole E Pashley","doi":"10.1353/obs.2025.a963649","DOIUrl":"https://doi.org/10.1353/obs.2025.a963649","url":null,"abstract":"<p><p>This is a review of Peng Ding's textbook \"A First Course in Causal Inference.\" The book builds causal inference topics up from basics in experiments to complex observational studies. This review discusses the book's style and content as well as who should use this book.</p>","PeriodicalId":74335,"journal":{"name":"Observational studies","volume":"11 2","pages":"209-212"},"PeriodicalIF":0.0,"publicationDate":"2025-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12959899/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147367550","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Observational studiesPub Date : 2025-06-25eCollection Date: 2025-01-01DOI: 10.1353/obs.2025.a963647
Bryan Keller, Vivian C Wong, Sangbaek Park, Jingru Zhang, Patrick Sheehan, Peter M Steiner
{"title":"A new four-arm within-study comparison: Design, implementation, and data.","authors":"Bryan Keller, Vivian C Wong, Sangbaek Park, Jingru Zhang, Patrick Sheehan, Peter M Steiner","doi":"10.1353/obs.2025.a963647","DOIUrl":"https://doi.org/10.1353/obs.2025.a963647","url":null,"abstract":"<p><p>Within-study comparisons (WSCs) use real, rather than simulated, data to compare estimates from observational studies against benchmarks from randomized controlled trials (RCTs). A primary goal of WSCs is to assess whether well-designed quasi-experimental designs (QEDs) can produce internally valid causal effect estimates comparable to those from RCTs. In this paper, we describe the design and implementation of a new type of WSC. Motivated by Shadish et al. (2008), we examine the impact of a mathematics training intervention and a vocabulary study session on posttest scores for mathematics and vocabulary, respectively. We extend the original design in three ways. First, before random assignment, we ask participants to express a preference for either the mathematics or vocabulary training session, after which they are randomly assigned regardless of preferences. This allows us to experimentally identify and estimate the overall average treatment effect (ATE) and two conditional ATEs: the average treatment effect on the treated (ATT) and the average treatment effect on the untreated (ATU). Second, participant recruitment and sample size (N = 2200) were determined through power analyses for comparing RCT and QED estimates, ensuring sufficient power for methodological comparisons. Finally, the study's eligibility criteria, recruitment, treatment allocation, and analysis plan were preregistered on the Open Science Foundation platform, and the data are publicly accessible. We believe that this WSC design and the resulting data set will be valuable for researchers seeking to evaluate causal inference methods and test identification assumptions using real-world data.</p>","PeriodicalId":74335,"journal":{"name":"Observational studies","volume":"11 2","pages":"153-188"},"PeriodicalIF":0.0,"publicationDate":"2025-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12959902/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147367499","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Observational studiesPub Date : 2025-06-25eCollection Date: 2025-01-01DOI: 10.1353/obs.2025.a963646
Han Ji, Arman Oganisian
{"title":"causalBETA: An R Package for Bayesian Semiparametric Causal Inference with Event-Time Outcomes.","authors":"Han Ji, Arman Oganisian","doi":"10.1353/obs.2025.a963646","DOIUrl":"https://doi.org/10.1353/obs.2025.a963646","url":null,"abstract":"<p><p>While randomized trials are the gold standard in causal inference, observational studies are often conducted when trials are infeasible either due to logistical or ethical constraints. Since treatments are not randomized in observational studies, techniques from causal inference are required to adjust for confounding. Bayesian approaches to causal estimation are desirable because they provide: 1) prior smoothing that offers useful regularization of causal effect estimates, 2) flexible models that are robust to misspecification, and 3) full inference (i.e., both point estimates and uncertainty quantification) for causal estimands. However, Bayesian causal inference is difficult to implement manually and there is a lack of user-friendly software, presenting a significant barrier to wide-spread use. Moreover, there is a lack of manuscripts aimed at explicitly connecting statistical/causal formula with implementation code. We address this gap by developing and describing causalBETA (<b>B</b>ayesian<b>E</b>vent<b>T</b>ime<b>A</b>nalysis) - an open-source R package for estimating causal effects on event-time outcomes using Bayesian semiparametric models. The package provides a familiar front-end to users, with syntax identical to existing survival analysis R packages, while leveraging Stan - a popular platform for high performance Bayesian computing - for efficient posterior computation. To improve user experience, the package provides custom S3 class objects and methods to facilitate visualizations and summaries of results using familiar generic functions like plot() and summary(). In this paper, we provide the methodological details of the package, a demonstration using publicly available data, and computational guidance.</p>","PeriodicalId":74335,"journal":{"name":"Observational studies","volume":"11 2","pages":"127-152"},"PeriodicalIF":0.0,"publicationDate":"2025-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12959900/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147367511","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Observational studiesPub Date : 2025-04-11eCollection Date: 2025-01-01DOI: 10.1353/obs.2025.a956842
Rachael Phillips, Mark van der Laan
{"title":"Commentary on ``Nonparametric identification is not enough, but randomized controlled trials are'': Statistical considerations for generating reliable evidence across a spectrum of studies that increasingly involve real-world elements.","authors":"Rachael Phillips, Mark van der Laan","doi":"10.1353/obs.2025.a956842","DOIUrl":"10.1353/obs.2025.a956842","url":null,"abstract":"<p><p>Judea Pearl, quoted in Pearl and Mackenzie (2008), stated that \"once we have understood why [randomized controlled trials] RCTs work, there is no need to put them on a pedestal and treat them as the gold standard of causal analysis, which all other methods should emulate.\" In Aronow et al. (2024), this claim is refuted, drawing on results of Robins and Ritov (1997). The argument is made that statistical estimation and inference tend to be fundamentally more difficult in observational studies than in randomized controlled trials, even when all confounders are observed and measured without error. We congratulate the authors for raising this highly timely, interesting discussion and welcome this opportunity to join this important debate. In this commentary, we focus on what it takes to generate reliable evidence across a spectrum of studies that increasingly involve real-world elements and less control over design. A related question is whether, along this spectrum of studies, the reliability of evidence generated by a statistical analysis decreases. We claim that this is not the case, but that the challenge for the appropriate statistical method increases, requiring sophisticated and careful execution.</p>","PeriodicalId":74335,"journal":{"name":"Observational studies","volume":"11 1","pages":"61-76"},"PeriodicalIF":0.0,"publicationDate":"2025-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12139718/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144251200","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Observational studiesPub Date : 2025-04-11eCollection Date: 2025-01-01DOI: 10.1353/obs.2025.a956837
P M Aronow, James M Robins, Theo Saarinen, Fredrik Sävje, Jasjeet S Sekhon
{"title":"Nonparametric identification is not enough, but randomized controlled trials are.","authors":"P M Aronow, James M Robins, Theo Saarinen, Fredrik Sävje, Jasjeet S Sekhon","doi":"10.1353/obs.2025.a956837","DOIUrl":"10.1353/obs.2025.a956837","url":null,"abstract":"<p><p>We argue that randomized controlled trials (RCTs) are special even among studies for which a nonparametric unconfoundedness assumption is credible. This claim follows from two results of Robins and Ritov (1997). First, in settings with at least one continuous confounder, there exists no estimator of the average treatment effect that is uniformly consistent unless the propensity score is known or additional assumptions are made on the complexity of the propensity score function. Second, with binary outcomes, knowledge of the propensity score yields a uniformly consistent estimator and finite-sample valid confidence intervals that shrink at a parametric rate, regardless of how complicated the propensity score function might be. We emphasize the latter point, and note that a successfully executed RCT provides knowledge of the propensity score to the researcher. We conclude that statistical estimation and inference tend to be fundamentally more difficult in observational settings than in RCTs, even when all confounders are observed and measured without error.</p>","PeriodicalId":74335,"journal":{"name":"Observational studies","volume":"11 1","pages":"3-16"},"PeriodicalIF":0.0,"publicationDate":"2025-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12139723/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144251202","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Observational studiesPub Date : 2025-04-11eCollection Date: 2025-01-01DOI: 10.1353/obs.2025.a956840
Christopher Harshaw
{"title":"Why are RCTs the Gold Standard? The Epistemological Difference Between Randomized Experiments and Observational Studies.","authors":"Christopher Harshaw","doi":"10.1353/obs.2025.a956840","DOIUrl":"10.1353/obs.2025.a956840","url":null,"abstract":"<p><p>In response to Pearl, Aronow et al. (2025) argue that randomized experiments are special among causal inference methods due to their statistical properties. I believe that the key distinction between randomized experiments and observational studies is not statistical, but rather epistemological in nature. In this comment, I aim to articulate this epistemological distinction and argue that it ought to take a more central role in these discussions.</p>","PeriodicalId":74335,"journal":{"name":"Observational studies","volume":"11 1","pages":"41-46"},"PeriodicalIF":0.0,"publicationDate":"2025-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12139715/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144251206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Observational studiesPub Date : 2025-04-11eCollection Date: 2025-01-01DOI: 10.1353/obs.2025.a956838
Drew Dimmery, Kevin Munger
{"title":"Enough?","authors":"Drew Dimmery, Kevin Munger","doi":"10.1353/obs.2025.a956838","DOIUrl":"10.1353/obs.2025.a956838","url":null,"abstract":"<p><p>We provide a critical response to Aronow et al. (2021) which argued that randomized controlled trials (RCTs) are \"enough,\" while nonparametric identification in observational studies is not. We first investigate what is meant by \"enough,\" arguing that this is a fundamentally a sociological claim about the relationship between statistical work and relevant institutional processes (here, academic peer review), rather than something that can be decided from within the logic of statistics. For a more complete conception of \"enough,\" we outline all that would need to be known - not just knowledge of propensity scores, but knowledge of many other spatial and temporal characteristics of the social world. Even granting the logic of the critique in Aronow et al. (2021), its practical importance is a question of the contexts under study. We argue that we should not be satisfied by appeals to intuition or experience about the complexity of \"naturally occurring\" propensity score functions. Instead, we call for more empirical metascience to begin to characterize this complexity. We apply this logic to the case of recommender systems as a demonstration of the weakness of allowing statisticians' intuitions to serve in place of metascientific data. This may be, as Aronow et al. (2021) claim, one of the \"few free lunches in statistics\"-but like many of the free lunches consumed by statisticians, it is only available to those working at a handful of large tech firms. Rather than implicitly deciding what is \"enough\" based on statistical applications the social world has determined to be most profitable, we are argue that practicing statisticians should explicitly engage with questions like \"for what?\" and \"for whom?\" in order to adequately answer the question of \"enough?\"</p>","PeriodicalId":74335,"journal":{"name":"Observational studies","volume":"11 1","pages":"17-26"},"PeriodicalIF":0.0,"publicationDate":"2025-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12139716/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144251201","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Observational studiesPub Date : 2025-04-11eCollection Date: 2025-01-01DOI: 10.1353/obs.2025.a956844
P M Aronow, James M Robins, Theo Saarinen, Fredrik Sävje, Jasjeet S Sekhon
{"title":"Rejoinder: Nonparametric identification is not enough, but randomized controlled trials are.","authors":"P M Aronow, James M Robins, Theo Saarinen, Fredrik Sävje, Jasjeet S Sekhon","doi":"10.1353/obs.2025.a956844","DOIUrl":"10.1353/obs.2025.a956844","url":null,"abstract":"<p><p>We thank the editor for organizing a diverse and wide-ranging discussion, and we thank the commentators for their detailed and thoughtful remarks. Most of the commentators provide broader perspectives on randomized experiments and their role in modern empirical practice. We believe this broader perspective is important, and the comments serve as complements to the somewhat narrow points we made in our paper. However, we believe these narrow points are of great consequence, and we find it useful to briefly recapitulate them here. When a practitioner aims to estimate averages of bounded potential outcomes (e.g., the average treatment effect on a binary outcome) in a setting where both ignorability and positivity are known to hold after adjusting for at least one continuous covariate, the following statements are true: • If the propensity score is known, such as in a randomized controlled trial (RCT), there exist simple estimators that are uniformly root-n consistent and asymptotically normal. Confidence intervals based on these estimators are finite-sample valid and their widths shrink at a root-n rate. • If the propensity score is not known, such as in an observational study, there exist neither uniformly consistent estimators nor uniform (i.e., honest) large-sample confidence intervals whose widths are shrinking with the sample size. To achieve these properties, the practitioner must impose untestable assumptions on either the propensity score function or the conditional expectation function of the outcomes.</p>","PeriodicalId":74335,"journal":{"name":"Observational studies","volume":"11 1","pages":"85-90"},"PeriodicalIF":0.0,"publicationDate":"2025-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12139717/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144251204","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}