{"title":"Comments on: A Re-analysis of Repeatability and Reproducibility in the Ames-USDOE-FBI Study, by Dorfman and Valliant","authors":"Max D. Morris","doi":"10.1080/2330443x.2023.2188069","DOIUrl":"https://doi.org/10.1080/2330443x.2023.2188069","url":null,"abstract":"","PeriodicalId":43397,"journal":{"name":"Statistics and Public Policy","volume":null,"pages":null},"PeriodicalIF":1.6,"publicationDate":"2023-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48341026","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Statistical Understanding of Disability in the LGBT Community","authors":"Christopher R. Surfus","doi":"10.1080/2330443x.2023.2188056","DOIUrl":"https://doi.org/10.1080/2330443x.2023.2188056","url":null,"abstract":"","PeriodicalId":43397,"journal":{"name":"Statistics and Public Policy","volume":null,"pages":null},"PeriodicalIF":1.6,"publicationDate":"2023-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45465276","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
C. Citro, Jonathan Auerbach, Katherine Smith Evans, E. Groshen, J. Landefeld, J. Mulrow, Tom Petska, Steve Pierson, N. Potok, C. Rothwell, John Thompson, James L. Woodworth, Edward Wu
{"title":"What Protects the Autonomy of the Federal Statistical Agencies? An Assessment of the Procedures in Place to Protect the Independence and Objectivity of Official U.S. Statistics","authors":"C. Citro, Jonathan Auerbach, Katherine Smith Evans, E. Groshen, J. Landefeld, J. Mulrow, Tom Petska, Steve Pierson, N. Potok, C. Rothwell, John Thompson, James L. Woodworth, Edward Wu","doi":"10.1080/2330443x.2023.2188062","DOIUrl":"https://doi.org/10.1080/2330443x.2023.2188062","url":null,"abstract":"The Abstract We assess the professional autonomy of the 13 principal U.S. federal statistical agencies. We define six components or measures of such autonomy and evaluate each of the 13 principal statistical agencies according to each measure. Our assessment yields three main findings: 1. Challenges to the objectivity, credibility, and utility of federal statistics arise largely as a consequence of insufficient autonomy. 2. There is remarkable variation in autonomy protections and a surprising lack of statutory protections for many agencies for many of the proposed measures. 3. Many existing autonomy rules and guidelines are weakened by unclear or unactionable","PeriodicalId":43397,"journal":{"name":"Statistics and Public Policy","volume":null,"pages":null},"PeriodicalIF":1.6,"publicationDate":"2023-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45997765","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Bayesian Spatio-temporal Model to Optimize Allocation of Buprenorphine in North Carolina.","authors":"Qianyu Dong, David Kline, Staci A Hepler","doi":"10.1080/2330443x.2023.2218448","DOIUrl":"10.1080/2330443x.2023.2218448","url":null,"abstract":"<p><p>The opioid epidemic is an ongoing public health crisis. In North Carolina, overdose deaths due to illicit opioid overdose have sharply increased over the last 5-7 years. Buprenorphine is a U.S. Food and Drug Administration approved medication for treatment of opioid use disorder and is obtained by prescription. Prior to January 2023, providers had to obtain a waiver and were limited in the number of patients that they could prescribe buprenorphine. Thus, identifying counties where increasing buprenorphine would yield the greatest overall reduction in overdose death can help policymakers target certain geographical regions to inform an effective public health response. We propose a Bayesian spatiotemporal model that relates yearly, county-level changes in illicit opioid overdose death rates to changes in buprenorphine prescriptions. We use our model to forecast the statewide count and rate of illicit opioid overdose deaths in future years, and we use nonlinear constrained optimization to identify the optimal buprenorphine increase in each county under a set of constraints on available resources. Our model estimates a negative relationship between death rate and increasing buprenorphine after accounting for other covariates, and our identified optimal single-year allocation strategy is estimated to reduce opioid overdose deaths by over 5.</p>","PeriodicalId":43397,"journal":{"name":"Statistics and Public Policy","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10398789/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9961651","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Shining a Light on Forensic Black-Box Studies","authors":"Kori Khan, A. Carriquiry","doi":"10.1080/2330443x.2023.2216748","DOIUrl":"https://doi.org/10.1080/2330443x.2023.2216748","url":null,"abstract":"Forensic science plays a critical role in the United States criminal justice system. For decades, many feature-based fields of forensic science, such as firearm and toolmark identification, developed outside the scientific community's purview. The results of these studies are widely relied on by judges nationwide. However, this reliance is misplaced. Black-box studies to date suffer from inappropriate sampling methods and high rates of missingness. Current black-box studies ignore both problems in arriving at the error rate estimates presented to courts. We explore the impact of each type of limitation using available data from black-box studies and court materials. We show that black-box studies rely on non-representative samples of examiners. Using a case study of a popular ballistics study, we find evidence that these unrepresentative samples may commit fewer errors than the wider population from which they came. We also find evidence that the missingness in black-box studies is non-ignorable. Using data from a recent latent print study, we show that ignoring this missingness likely results in systematic underestimates of error rates. Finally, we offer concrete steps to overcome these limitations.","PeriodicalId":43397,"journal":{"name":"Statistics and Public Policy","volume":null,"pages":null},"PeriodicalIF":1.6,"publicationDate":"2022-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44806864","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Marginal Structural Models to Estimate Causal Effects of Right-to-Carry Laws on Crime","authors":"W. M. van der Wal","doi":"10.1080/2330443X.2022.2120136","DOIUrl":"https://doi.org/10.1080/2330443X.2022.2120136","url":null,"abstract":"Abstract Right-to-carry (RTC) laws allow the legal carrying of concealed firearms for defense, in certain states in the United States. I used modern causal inference methodology from epidemiology to examine the effect of RTC laws on crime over a period from 1959 up to 2016. I fitted marginal structural models (MSMs), using inverse probability weighting (IPW) to correct for criminological, economic, political and demographic confounders. Results indicate that RTC laws significantly increase violent crime by 7.5% and property crime by 6.1%. RTC laws significantly increase murder and manslaughter, robbery, aggravated assault, burglary, larceny theft and motor vehicle theft rates. Applying this method to this topic for the first time addresses methodological shortcomings in previous studies such as conditioning away the effect, overfit and the inappropriate use of county level measurements. Data and analysis code for this article are available online.","PeriodicalId":43397,"journal":{"name":"Statistics and Public Policy","volume":null,"pages":null},"PeriodicalIF":1.6,"publicationDate":"2022-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48333451","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Signal Weighted Teacher Value-Added Models","authors":"Edward Kim","doi":"10.1080/2330443X.2022.2105769","DOIUrl":"https://doi.org/10.1080/2330443X.2022.2105769","url":null,"abstract":"Abstract This study introduces the signal weighted teacher value-added model (SW VAM), a value-added model that weights student-level observations based on each student’s capacity to signal their assigned teacher’s quality. Specifically, the model leverages the repeated appearance of a given student to estimate student reliability and sensitivity parameters, whereas traditional VAMs represent a special case where all students exhibit identical parameters. Simulation study results indicate that SW VAMs outperform traditional VAMs at recovering true teacher quality when the assumption of student parameter invariance is met but have mixed performance under alternative assumptions of the true data generating process depending on data availability and the choice of priors. Evidence using an empirical dataset suggests that SW VAM and traditional VAM results may disagree meaningfully in practice. These findings suggest that SW VAMs have promising potential to recover true teacher value-added in practical applications and, as a version of value-added models that attends to student differences, can be used to test the validity of traditional VAM assumptions in empirical contexts.","PeriodicalId":43397,"journal":{"name":"Statistics and Public Policy","volume":null,"pages":null},"PeriodicalIF":1.6,"publicationDate":"2022-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42748072","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Re-Analysis of Repeatability and Reproducibility in the Ames-USDOE-FBI Study","authors":"A. Dorfman, R. Valliant","doi":"10.1080/2330443X.2022.2120137","DOIUrl":"https://doi.org/10.1080/2330443X.2022.2120137","url":null,"abstract":"Abstract Forensic firearms identification, the determination by a trained firearms examiner as to whether or not bullets or cartridges came from a common weapon, has long been a mainstay in the criminal courts. Reliability of forensic firearms identification has been challenged in the general scientific community, and, in response, several studies have been carried out aimed at showing that firearms examination is accurate, that is, has low error rates. Less studied has been the question of consistency, of whether two examinations of the same bullets or cartridge cases come to the same conclusion, carried out by an examiner on separate occasions—intrarater reliability or repeatability—or by two examiners—interrater reliability or reproducibility. One important study, described in a 2020 Report by the Ames Laboratory-USDOE to the Federal Bureau of Investigation, went beyond considerations of accuracy to investigate firearms examination repeatability and reproducibility. The Report’s conclusions were paradoxical. The observed agreement of examiners with themselves or with other examiners appears mediocre. However, the study concluded repeatability and reproducibility are satisfactory, on grounds that the observed agreement exceeds a quantity called the expected agreement. We find that appropriately employing expected agreement as it was intended does not suggest satisfactory repeatability and reproducibility, but the opposite.","PeriodicalId":43397,"journal":{"name":"Statistics and Public Policy","volume":null,"pages":null},"PeriodicalIF":1.6,"publicationDate":"2022-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44790182","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
N. Hwang, S. Chatterjee, Y. Di, Sharmodeep Bhattacharyya
{"title":"Observational Study of the Effect of the Juvenile Stay-At-Home Order on SARS-CoV-2 Infection Spread in Saline County, Arkansas","authors":"N. Hwang, S. Chatterjee, Y. Di, Sharmodeep Bhattacharyya","doi":"10.1080/2330443X.2022.2050326","DOIUrl":"https://doi.org/10.1080/2330443X.2022.2050326","url":null,"abstract":"Abstract We assess the treatment effect of juvenile stay-at-home orders (JSAHO) on reducing the rate of SARS-CoV-2 infection spread in Saline County (“Saline”), Arkansas, by examining the difference between Saline’s and control Arkansas counties’ changes in daily and mean log infection rates of pretreatment (March 28–April 5, 2020) and treatment periods (April 6–May 6, 2020). A synthetic control county is constructed based on the parallel-trends assumption, least-squares fitting on pretreatment and socio-demographic covariates, and elastic-net-based methods, from which the counterfactual outcome is predicted and the treatment effect is estimated using the difference-in-differences, the synthetic control, and the changes-in-changes methodologies. Both the daily and average treatment effects of JSAHO are shown to be significant. Despite its narrow scope and lack of enforcement for compliance, JSAHO reduced the rate of the infection spread in Saline. Supplementary materials for this article are available online.","PeriodicalId":43397,"journal":{"name":"Statistics and Public Policy","volume":null,"pages":null},"PeriodicalIF":1.6,"publicationDate":"2022-03-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43216255","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Wartime Fatalities in the Nuclear Era","authors":"L. Ice, J. Scouras, E. Toton","doi":"10.1080/2330443X.2022.2038744","DOIUrl":"https://doi.org/10.1080/2330443X.2022.2038744","url":null,"abstract":"Abstract Senior leaders in the U.S. Department of Defense, as well as nuclear strategists and academics, have argued that the advent of nuclear weapons is associated with a dramatic decrease in wartime fatalities. This assessment is often supported by an evolving series of figures that show a marked drop in wartime fatalities as a percentage of world population after 1945 to levels well below those of the prior centuries. The goal of this article is not to ascertain whether nuclear weapons are associated with or have led to a decrease in wartime fatalities, but rather to critique the supporting statistical evidence. We assess these wartime fatality figures and find that they are both irreproducible and misleading. We perform a more rigorous and traceable analysis and discover that post-1945 wartime fatalities as a percentage of world population are consistent with those of many other historical periods. Supplementary materials for this article are available online.","PeriodicalId":43397,"journal":{"name":"Statistics and Public Policy","volume":null,"pages":null},"PeriodicalIF":1.6,"publicationDate":"2022-02-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49250579","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}