Farasat A. S. Bokhari, Abel Brodeur, Michalis Drouvelis
{"title":"Introduction to the symposium on reproducibility and replicability in economics: Part I","authors":"Farasat A. S. Bokhari, Abel Brodeur, Michalis Drouvelis","doi":"10.1111/ecin.13285","DOIUrl":"https://doi.org/10.1111/ecin.13285","url":null,"abstract":"<p>Reproducibility and replicability are cornerstones of scientific progress, ensuring that findings can withstand scrutiny and that results hold under varied analyses. In economics, these principles support a self-correcting system, fostering more reliable empirical research and more robust foundations for policy design. However, despite the recognition of replication's importance, many empirical studies, particularly non-experimental ones, lack rigorous replication. This symposium of <i>Economic Inquiry</i> aims to address this gap, presenting new research that underscores the need for reproduction and replication, particularly in non-experimental studies, and proposes innovative approaches to overcoming the practical challenges of replicability in economics. Due to the volume of high-quality submissions we received, we have divided this symposium into two parts. This first part highlights key methodological advances and remaining challenges that contribute to the broader conversation on reproducibility and replicability in economics. A forthcoming second part will continue this exploration, further showcasing reproductions and replications of seminal and well-cited articles.</p><p>Our call for papers sought empirical replications, methodological advances, and theoretical contributions that enrich our understanding of replication's role in economic research. The selected papers in this first part introduce new methodological tools and provide frameworks for conducting and evaluating the effectiveness of reproductions and replications. These contributions advance our collective understanding of what it means for economics to be a replicable and self-correcting science. We provide a short summary of each article below.</p><p>The symposium's first article, “A Framework for Evaluating Reproducibility and Replicability in Economics”, proposes a structured approach to assess these core aspects of scientific reliability in economics. By distinguishing between various types of reproducibility (computational, recreate, robustness) and replicability (direct, conceptual), and introducing clear indicators to measure each, the article offers a practical and theoretically grounded framework that addresses long-standing ambiguities in the field. This contribution is particularly significant in the context of increasing efforts to improve transparency and credibility in economics research.</p><p>The article “Replication Code Availability Over Time and Across Fields: Evidence From the German Data Archive for Business and Economic Studies” provides trends in replication code availability over time and across disciplines by examining studies that used the German Socio-Economic Panel (SOEP) data, which, while restricted, is available to researchers and has been used in over 2500 articles in economics and social sciences. By concentrating on studies with large common data, the study highlights both progress and ongoing challenges in making replication code accessib","PeriodicalId":51380,"journal":{"name":"Economic Inquiry","volume":"63 2","pages":"335-337"},"PeriodicalIF":1.7,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/ecin.13285","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143629753","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"经济学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tongzhe Li, Collin Weigel, Paul Ferraro, Kent D. Messer
{"title":"Underpowered studies and exaggerated effects: A replication and re-evaluation of the magnitude of anchoring effects","authors":"Tongzhe Li, Collin Weigel, Paul Ferraro, Kent D. Messer","doi":"10.1111/ecin.13279","DOIUrl":"https://doi.org/10.1111/ecin.13279","url":null,"abstract":"<p>We reconsider one of the most widely studied behavioral biases: anchoring effects. We estimate that study designs in this literature, including replication studies, routinely fail to achieve statistical power of more than 30%. This study replicates an anchoring study that reported an effect size of a 31% increase in participants' bids. In the replication, we increased the design's statistical power from 46% to 96%, reducing the average exaggeration of a statistically significant result by a factor of seven. Our replication results reject the size of the original estimated effects. We find an estimated effect of 3.4% (95% CI [−3.4%, 10%]).</p>","PeriodicalId":51380,"journal":{"name":"Economic Inquiry","volume":"63 2","pages":"387-402"},"PeriodicalIF":1.7,"publicationDate":"2025-02-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/ecin.13279","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143629859","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"经济学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Idiosyncratic asset return and wage risk of US households","authors":"Stephen Snudden","doi":"10.1111/ecin.13275","DOIUrl":"https://doi.org/10.1111/ecin.13275","url":null,"abstract":"<p>This paper documents the degree of idiosyncratic asset return heterogeneity, serial correlation, and correlation with wage heterogeneity for US households. Novel panel-data measurements for returns on household assets are proposed. Sizable transitory idiosyncratic return heterogeneity is documented to exist concurrently with permanent heterogeneity in household-specific returns. On average, idiosyncratic permanent risk to wages and transitory risk to total asset returns are correlated. This arises primarily from correlated wage and return risk to primary housing assets, and is dependent on age and wealth. The estimates inform the covariance structure of idiosyncratic asset return and wage heterogeneity.</p>","PeriodicalId":51380,"journal":{"name":"Economic Inquiry","volume":"63 2","pages":"636-657"},"PeriodicalIF":1.7,"publicationDate":"2025-01-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/ecin.13275","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143629932","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"经济学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"How policing incentives affect crime, measurement, and justice","authors":"Jordan Adamson, Lucas Rentschler","doi":"10.1111/ecin.13270","DOIUrl":"https://doi.org/10.1111/ecin.13270","url":null,"abstract":"<p>In this paper we develop a model where the police choose between investigating and patrolling, while civilians choose between producing and stealing. We derive a truth table for the equilibrium numbers of criminals and producers, punished or not, that can holistically evaluate the effects of police performance incentives. To test the model, we conduct an experiment that varies how severely an officer is reprimanded for false punishments. We find that stronger reprimands do not change crime, increase civilian incomes, and decrease false positives. We also find that the clearance rate, a measure of performance used widely in econometric studies, suggests police performance is better when it is unambiguously worse.</p>","PeriodicalId":51380,"journal":{"name":"Economic Inquiry","volume":"63 2","pages":"545-567"},"PeriodicalIF":1.7,"publicationDate":"2025-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143629749","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"经济学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Does nature shape risk preferences? Evidence from Chile, Norway, and Tanzania","authors":"Florian Diekert, Robbert-Jan Schaap","doi":"10.1111/ecin.13272","DOIUrl":"https://doi.org/10.1111/ecin.13272","url":null,"abstract":"<p>Does exposure to a more risky environment affect risk preferences? Going beyond single-case study evidence, we report results from five surveys conducted in three countries and link this with administrative data to study whether a link between exposure and preferences is detectable and widespread. We find no evidence for endogenous preferences in Norway and Tanzania, but relatively strong evidence in Chile, where differences in risk exposure are most pronounced. Moreover, we make a first pass at disentangling selection from adaptation as potential mechanisms. For Tanzania and Norway, the data speaks for selection, while it speaks for adaptation in Chile.</p>","PeriodicalId":51380,"journal":{"name":"Economic Inquiry","volume":"63 2","pages":"568-590"},"PeriodicalIF":1.7,"publicationDate":"2024-12-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/ecin.13272","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143629857","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"经济学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Dynamic norms and organ donation","authors":"Ruqian Zang, Jianbiao Li, Xiaofei Niu","doi":"10.1111/ecin.13274","DOIUrl":"https://doi.org/10.1111/ecin.13274","url":null,"abstract":"<p>We conduct three experiments with over 1600 subjects to examine the impact of dynamic norms on organ donation. We find that dynamic norms (low-prevalence behaviors framed as increasing in popularity over time) significantly increase organ donor registration, and that this effect is mainly driven by dynamic trends (prevalence of behaviors with a growth trend). The positive effect of dynamic trends on organ donor registration is better than that of combined norms (low-descriptive but high-injunctive behaviors). The underlying mechanism may be future norm perceptions and moral emotions (elevation). Our paper presents a novel and effective strategy to increase organ donation.</p>","PeriodicalId":51380,"journal":{"name":"Economic Inquiry","volume":"63 2","pages":"591-607"},"PeriodicalIF":1.7,"publicationDate":"2024-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143629772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"经济学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Joakim A. Weill, Matthieu Stigler, Olivier Deschenes, Michael R. Springborn
{"title":"Researchers' degrees of flexibility: Revisiting COVID-19 policy evaluations","authors":"Joakim A. Weill, Matthieu Stigler, Olivier Deschenes, Michael R. Springborn","doi":"10.1111/ecin.13273","DOIUrl":"https://doi.org/10.1111/ecin.13273","url":null,"abstract":"<p>Empirical research involves multiple, seemingly-minor choices that can substantially impact a study's findings. While acknowledged, the importance of these “degrees of flexibility” on published estimates is not well understood. We examine the considerable literature focused on the impacts of early COVID-19 policies on social distancing to assess the role of researchers' degrees of flexibility on the estimated effects of mobility-reducing policies. We find that estimates reported in previous studies are not robust to minor changes in typically-unexplored dimensions of the degree of flexibility space, and usual robustness tests systematically fail to detect these issues.</p>","PeriodicalId":51380,"journal":{"name":"Economic Inquiry","volume":"63 2","pages":"441-462"},"PeriodicalIF":1.7,"publicationDate":"2024-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/ecin.13273","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143629771","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"经济学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Po-yang Yu, Hamid Beladi, Hsun Chu, Ching-chong Lai
{"title":"International licensing and quality-enhancing technology spillover in a product cycle model","authors":"Po-yang Yu, Hamid Beladi, Hsun Chu, Ching-chong Lai","doi":"10.1111/ecin.13271","DOIUrl":"https://doi.org/10.1111/ecin.13271","url":null,"abstract":"<p>This paper examines the effects of an international quality-enhancing technology spillover on the international industry distribution and innovation in a dynamic North-South model. When a Southern firm receives a technology transfer from the North through international licensing, the technology spills over to other Southern firms by enhancing the quality of their products. We find that a stronger spillover effect depresses both Northern innovative R&D and Southern adaptive R&D, and thus is welfare-impairing both for the advanced country and recipient country. The results cast doubt on the common view that regards the technology spillover as a positive externality.</p>","PeriodicalId":51380,"journal":{"name":"Economic Inquiry","volume":"63 2","pages":"608-635"},"PeriodicalIF":1.7,"publicationDate":"2024-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143629904","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"经济学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
William C. Boning, Joel Slemrod, Ellen Stuart, Alex Turk
{"title":"Does giving tax debtors a break improve compliance and income? Evidence from quasi-random assignment of IRS Revenue Officers","authors":"William C. Boning, Joel Slemrod, Ellen Stuart, Alex Turk","doi":"10.1111/ecin.13268","DOIUrl":"https://doi.org/10.1111/ecin.13268","url":null,"abstract":"<p>This paper uses the quasi-random assignment of IRS Revenue Officers to tax debtors' cases as an instrumental variable to identify the causal effects of suspending debt collection on tax compliance and future income. In contrast to uninstrumented estimates, we find no statistically significant evidence that putting off attempts to collect debt reduces compliance with future tax obligations or future reported income. Among marginal hardship cases, pausing collection instead increases future income, specifically wage earnings by the taxpayer's spouse. In addition, we address concerns about potential non-random assignment of Revenue Officers.</p>","PeriodicalId":51380,"journal":{"name":"Economic Inquiry","volume":"63 2","pages":"486-503"},"PeriodicalIF":1.7,"publicationDate":"2024-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143629805","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"经济学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Optimal lockdowns under constraints","authors":"Jihad C. Dagher, Christian Parkinson","doi":"10.1111/ecin.13265","DOIUrl":"https://doi.org/10.1111/ecin.13265","url":null,"abstract":"<p>We present a systematic examination of the impact of frictions on optimal pandemic response, bridging the significant gap between policy recommendations and implementation. We focus in particular on constraints in testing delivery and in lockdown efficacy in the context of a canonical pandemic model. The latter is modified for a more faithful representation of lockdowns. The paper sheds light on nuanced, and sometimes counter-intuitive, relationships. It rationalizes key but divergent findings in the literature on the extent of substitution and complementarity between lockdowns and testing. It also demonstrates remarkable robustness in lockdown policy to changes in its efficiency.</p>","PeriodicalId":51380,"journal":{"name":"Economic Inquiry","volume":"63 2","pages":"523-544"},"PeriodicalIF":1.7,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143629751","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"经济学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}