Yanjiao Shen, Parpia Sameer, Xin Xia, Yuqing Zhang, Jinhui Ma, Qingyang Shi, Qiukui Hao, Xianlin Gu, Wenbo He, Yamin Chen, Na Zhang, Le Wang, Yating Zeng, Xiaoyi Su, Qiang Zong, Qiao Zhi, Sitong Liu, Xinyao Wang, Xinyu Zou, Ying He, Qiong Guo, Borong Wang, Liang Du, Zhengchi Li, Jin Huang, Guyatt Gordon
{"title":"随机对照试验模拟研究中处理缺失二值结果的最优策略的系统综述。","authors":"Yanjiao Shen, Parpia Sameer, Xin Xia, Yuqing Zhang, Jinhui Ma, Qingyang Shi, Qiukui Hao, Xianlin Gu, Wenbo He, Yamin Chen, Na Zhang, Le Wang, Yating Zeng, Xiaoyi Su, Qiang Zong, Qiao Zhi, Sitong Liu, Xinyao Wang, Xinyu Zou, Ying He, Qiong Guo, Borong Wang, Liang Du, Zhengchi Li, Jin Huang, Guyatt Gordon","doi":"10.1111/jebm.70058","DOIUrl":null,"url":null,"abstract":"<p><strong>Aim: </strong>To summarize the optimal strategies for dealing with missing binary outcome data (MBOD) in randomized controlled trials (RCTs) as informed by simulation studies, and to summarize the quality of reporting in these studies.</p><p><strong>Methods: </strong>To identify simulation studies comparing at least two strategies to deal with MBOD and evaluating their performance (bias, coverage and power), we searched MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials via Ovid, Web of Science, and JSTOR from their inception up to December 20, 2023. We evaluated reporting quality using established criteria for simulation studies in medical statistics. We summarized data using descriptive statistics and a narrative synthesis.</p><p><strong>Results: </strong>Our search identified 29,460 citations, of which five proved eligible. Multiple imputation (MI), investigated in five studies, showed consistently good performance in all domains tested for missing completely at random (MCAR) and missing at random (MAR) but with important limitations in missing not at random (MNAR). Complete case analysis (CCA), investigated in four studies of which three addressed model-based CCA, performed well in bias and coverage under MAR and MCAR, but less well for MNAR. One study reported that non-model-based CCA performed poorly with respect to bias under MAR. Non-model-based single imputation, investigated in two studies, showed consistently poor performance across all domains tested for MAR, MCAR and MNAR. One study reported that model-based single imputation performed well with respect to bias under MAR. Regarding reporting quality, all studies reported the aims, dependence of simulated data sets, scenarios and statistical methods evaluated, number of simulations performed, justification of data generation and criteria used to evaluate the simulation performance. None of the studies reported the starting seeds, random number generators and failures occurring during simulation.</p><p><strong>Conclusions: </strong>Simulation studies address methods to deal with MBOD in RCTs, provided evidence that the MI approach is superior with respect to bias and coverage compared with CCA. Non-model-based single imputation generally performed poorly.</p>","PeriodicalId":16090,"journal":{"name":"Journal of Evidence‐Based Medicine","volume":" ","pages":"e70058"},"PeriodicalIF":3.5000,"publicationDate":"2025-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A Systematic Survey of the Optimal Strategy for Dealing With Missing Binary Outcomes in Simulation Studies of Randomized Controlled Trials.\",\"authors\":\"Yanjiao Shen, Parpia Sameer, Xin Xia, Yuqing Zhang, Jinhui Ma, Qingyang Shi, Qiukui Hao, Xianlin Gu, Wenbo He, Yamin Chen, Na Zhang, Le Wang, Yating Zeng, Xiaoyi Su, Qiang Zong, Qiao Zhi, Sitong Liu, Xinyao Wang, Xinyu Zou, Ying He, Qiong Guo, Borong Wang, Liang Du, Zhengchi Li, Jin Huang, Guyatt Gordon\",\"doi\":\"10.1111/jebm.70058\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Aim: </strong>To summarize the optimal strategies for dealing with missing binary outcome data (MBOD) in randomized controlled trials (RCTs) as informed by simulation studies, and to summarize the quality of reporting in these studies.</p><p><strong>Methods: </strong>To identify simulation studies comparing at least two strategies to deal with MBOD and evaluating their performance (bias, coverage and power), we searched MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials via Ovid, Web of Science, and JSTOR from their inception up to December 20, 2023. We evaluated reporting quality using established criteria for simulation studies in medical statistics. We summarized data using descriptive statistics and a narrative synthesis.</p><p><strong>Results: </strong>Our search identified 29,460 citations, of which five proved eligible. Multiple imputation (MI), investigated in five studies, showed consistently good performance in all domains tested for missing completely at random (MCAR) and missing at random (MAR) but with important limitations in missing not at random (MNAR). Complete case analysis (CCA), investigated in four studies of which three addressed model-based CCA, performed well in bias and coverage under MAR and MCAR, but less well for MNAR. One study reported that non-model-based CCA performed poorly with respect to bias under MAR. Non-model-based single imputation, investigated in two studies, showed consistently poor performance across all domains tested for MAR, MCAR and MNAR. One study reported that model-based single imputation performed well with respect to bias under MAR. Regarding reporting quality, all studies reported the aims, dependence of simulated data sets, scenarios and statistical methods evaluated, number of simulations performed, justification of data generation and criteria used to evaluate the simulation performance. None of the studies reported the starting seeds, random number generators and failures occurring during simulation.</p><p><strong>Conclusions: </strong>Simulation studies address methods to deal with MBOD in RCTs, provided evidence that the MI approach is superior with respect to bias and coverage compared with CCA. Non-model-based single imputation generally performed poorly.</p>\",\"PeriodicalId\":16090,\"journal\":{\"name\":\"Journal of Evidence‐Based Medicine\",\"volume\":\" \",\"pages\":\"e70058\"},\"PeriodicalIF\":3.5000,\"publicationDate\":\"2025-08-26\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Evidence‐Based Medicine\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1111/jebm.70058\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"MEDICINE, GENERAL & INTERNAL\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Evidence‐Based Medicine","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1111/jebm.70058","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MEDICINE, GENERAL & INTERNAL","Score":null,"Total":0}
引用次数: 0
摘要
目的:总结模拟研究中处理随机对照试验(RCTs)中缺失二元结局数据(MBOD)的最佳策略,并总结这些研究的报告质量。方法:为了确定比较至少两种处理MBOD策略并评估其性能(偏倚、覆盖率和功率)的模拟研究,我们通过Ovid、Web of Science和JSTOR检索了MEDLINE、EMBASE、Cochrane Central Register of Controlled Trials,从它们成立到2023年12月20日。我们使用医学统计学模拟研究的既定标准评估报告质量。我们使用描述性统计和叙述性综合来总结数据。结果:我们的搜索确定了29,460条引用,其中5条被证明是合格的。在五项研究中,多重输入(MI)在完全随机缺失(MCAR)和随机缺失(MAR)测试的所有领域中表现出一致的良好表现,但在非随机缺失(MNAR)中存在重要局限性。完整案例分析(CCA)在四项研究中进行了调查,其中三项研究涉及基于模型的CCA,在MAR和MCAR下的偏倚和覆盖率方面表现良好,但在MNAR上表现不佳。一项研究报告称,在MAR下,非基于模型的CCA在偏倚方面表现不佳。两项研究调查了非基于模型的单一imputation,结果显示,在MAR、MCAR和MNAR测试的所有领域中,CCA的表现都很差。一项研究报告称,基于模型的单次代入在mar下的偏倚方面表现良好。关于报告质量,所有研究都报告了目标、模拟数据集的依赖性、评估的情景和统计方法、执行的模拟次数、数据生成的合理性和用于评估模拟性能的标准。没有研究报告在模拟过程中发生的启动种子、随机数生成器和故障。结论:模拟研究解决了随机对照试验中处理MBOD的方法,提供的证据表明,与CCA相比,MI方法在偏倚和覆盖方面优于CCA。非基于模型的单次输入通常表现不佳。
A Systematic Survey of the Optimal Strategy for Dealing With Missing Binary Outcomes in Simulation Studies of Randomized Controlled Trials.
Aim: To summarize the optimal strategies for dealing with missing binary outcome data (MBOD) in randomized controlled trials (RCTs) as informed by simulation studies, and to summarize the quality of reporting in these studies.
Methods: To identify simulation studies comparing at least two strategies to deal with MBOD and evaluating their performance (bias, coverage and power), we searched MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials via Ovid, Web of Science, and JSTOR from their inception up to December 20, 2023. We evaluated reporting quality using established criteria for simulation studies in medical statistics. We summarized data using descriptive statistics and a narrative synthesis.
Results: Our search identified 29,460 citations, of which five proved eligible. Multiple imputation (MI), investigated in five studies, showed consistently good performance in all domains tested for missing completely at random (MCAR) and missing at random (MAR) but with important limitations in missing not at random (MNAR). Complete case analysis (CCA), investigated in four studies of which three addressed model-based CCA, performed well in bias and coverage under MAR and MCAR, but less well for MNAR. One study reported that non-model-based CCA performed poorly with respect to bias under MAR. Non-model-based single imputation, investigated in two studies, showed consistently poor performance across all domains tested for MAR, MCAR and MNAR. One study reported that model-based single imputation performed well with respect to bias under MAR. Regarding reporting quality, all studies reported the aims, dependence of simulated data sets, scenarios and statistical methods evaluated, number of simulations performed, justification of data generation and criteria used to evaluate the simulation performance. None of the studies reported the starting seeds, random number generators and failures occurring during simulation.
Conclusions: Simulation studies address methods to deal with MBOD in RCTs, provided evidence that the MI approach is superior with respect to bias and coverage compared with CCA. Non-model-based single imputation generally performed poorly.
期刊介绍:
The Journal of Evidence-Based Medicine (EMB) is an esteemed international healthcare and medical decision-making journal, dedicated to publishing groundbreaking research outcomes in evidence-based decision-making, research, practice, and education. Serving as the official English-language journal of the Cochrane China Centre and West China Hospital of Sichuan University, we eagerly welcome editorials, commentaries, and systematic reviews encompassing various topics such as clinical trials, policy, drug and patient safety, education, and knowledge translation.