Counternull Sets in Randomized Experiments

IF 1.8 4区 数学 Q1 STATISTICS & PROBABILITY
M.-A. C. Bind, D. B. Rubin
{"title":"Counternull Sets in Randomized Experiments","authors":"M.-A. C. Bind, D. B. Rubin","doi":"10.1080/00031305.2024.2432884","DOIUrl":null,"url":null,"abstract":"Consider a study whose primary results are “not statistically significant”. How often does it lead to the following published conclusion that “there is no effect of the treatment/exposure on the outcome”? We believe too often and that the requirement to report counternull values could help to avoid this! In statistical parlance, the null value of an estimand is a value that is distinguished in some way from other possible values, for example a value that indicates no difference between the general health status of those treated with a new drug versus a traditional drug. A counternull value is a nonnull value of that estimand that is supported by the same amount of evidence that supports the null value. Of course, such a definition depends critically on how “evidence” is defined. Here, we consider the context of a randomized experiment where evidence is summarized by the randomization-based <i>p</i>-value associated with a specified sharp null hypothesis. Consequently, a counternull value has the same <i>p</i>-value from the randomization test as does the null value; the counternull value is rarely unique, but rather comprises a <i>set</i> of values. We explore advantages to reporting a counternull set in addition to the <i>p</i>-value associated with a null value; a first advantage is pedagogical, in that reporting it avoids the mistake of implicitly accepting a not-rejected null hypothesis; a second advantage is that the effort to construct a counternull set can be scientifically helpful by encouraging thought about nonnull values of estimands. Two examples are used to illustrate these ideas.","PeriodicalId":50801,"journal":{"name":"American Statistician","volume":"1 1","pages":""},"PeriodicalIF":1.8000,"publicationDate":"2025-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"American Statistician","FirstCategoryId":"100","ListUrlMain":"https://doi.org/10.1080/00031305.2024.2432884","RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"STATISTICS & PROBABILITY","Score":null,"Total":0}
引用次数: 0

Abstract

Consider a study whose primary results are “not statistically significant”. How often does it lead to the following published conclusion that “there is no effect of the treatment/exposure on the outcome”? We believe too often and that the requirement to report counternull values could help to avoid this! In statistical parlance, the null value of an estimand is a value that is distinguished in some way from other possible values, for example a value that indicates no difference between the general health status of those treated with a new drug versus a traditional drug. A counternull value is a nonnull value of that estimand that is supported by the same amount of evidence that supports the null value. Of course, such a definition depends critically on how “evidence” is defined. Here, we consider the context of a randomized experiment where evidence is summarized by the randomization-based p-value associated with a specified sharp null hypothesis. Consequently, a counternull value has the same p-value from the randomization test as does the null value; the counternull value is rarely unique, but rather comprises a set of values. We explore advantages to reporting a counternull set in addition to the p-value associated with a null value; a first advantage is pedagogical, in that reporting it avoids the mistake of implicitly accepting a not-rejected null hypothesis; a second advantage is that the effort to construct a counternull set can be scientifically helpful by encouraging thought about nonnull values of estimands. Two examples are used to illustrate these ideas.
随机实验中的反空集
考虑一项研究,其主要结果“在统计上不显著”。有多少次会得出以下发表的结论:“治疗/暴露对结果没有影响”?我们经常认为报告反空值的要求可以帮助避免这种情况!用统计学术语来说,估计值的空值是在某种程度上与其他可能的值有所区别的值,例如,一个值表明接受新药治疗的患者的一般健康状况与接受传统药物治疗的患者的一般健康状况没有区别。反空值是该估计的非空值,它得到与支持空值相同数量的证据的支持。当然,这样的定义主要取决于如何定义“证据”。在这里,我们考虑随机实验的背景,其中证据由与特定尖锐零假设相关的基于随机化的p值总结。因此,反空值与空值具有相同的随机化检验p值;反空值很少是唯一的,而是由一组值组成。除了报告与空值相关的p值之外,我们还探讨了报告反空集的优点;第一个优势是教学上的,在报告中它避免了隐含地接受一个未被拒绝的零假设的错误;第二个优点是,构建反空集的努力可以通过鼓励思考估计的非空值而在科学上有所帮助。下面用两个例子来说明这些观点。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
American Statistician
American Statistician 数学-统计学与概率论
CiteScore
3.50
自引率
5.60%
发文量
64
审稿时长
>12 weeks
期刊介绍: Are you looking for general-interest articles about current national and international statistical problems and programs; interesting and fun articles of a general nature about statistics and its applications; or the teaching of statistics? Then you are looking for The American Statistician (TAS), published quarterly by the American Statistical Association. TAS contains timely articles organized into the following sections: Statistical Practice, General, Teacher''s Corner, History Corner, Interdisciplinary, Statistical Computing and Graphics, Reviews of Books and Teaching Materials, and Letters to the Editor.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信