Design Parameters for Planning the Sample Size of Individual-Level Randomized Controlled Trials in Community Colleges.

IF 3 4区 社会学 Q1 SOCIAL SCIENCES, INTERDISCIPLINARY
Marie-Andrée Somers, Michael J Weiss, Colin Hill
{"title":"Design Parameters for Planning the Sample Size of Individual-Level Randomized Controlled Trials in Community Colleges.","authors":"Marie-Andrée Somers,&nbsp;Michael J Weiss,&nbsp;Colin Hill","doi":"10.1177/0193841X221121236","DOIUrl":null,"url":null,"abstract":"<p><p>The last two decades have seen a dramatic increase in randomized controlled trials (RCTs) conducted in community colleges. Yet, there is limited empirical information on the design parameters necessary to plan the sample size for RCTs in this context. For a blocked student-level random assignment research design, key design parameters for the minimum detectable true effect (MDTE) are the within-block outcome standard deviation <math><mrow><mo>(</mo><msub><mi>σ</mi><mrow><mo>|</mo><mi>S</mi></mrow></msub><mo>)</mo></mrow></math> and the within-block outcome variance explained by baseline covariates like student characteristics <math><mrow><mo>(</mo><msubsup><mi>R</mi><mrow><mo>|</mo><mi>S</mi></mrow><mn>2</mn></msubsup><mo>)</mo></mrow></math>. We provide empirical estimates of these key design parameters, discussing the pattern of estimates across outcomes (enrollment, credits earned, credential attainment, and grade point average), semesters, and studies. The main analyses use student-level data from 8 to 14 RCTs including 5,649-7,099 students (depending on the outcome) with follow-up data for 3 years. The following patterns are observed: the within-block standard deviation <math><mrow><mo>(</mo><msub><mi>σ</mi><mrow><mo>|</mo><mi>S</mi></mrow></msub><mo>)</mo></mrow></math> and therefore the MDTE can be much larger in later semesters for enrollment outcomes and cumulative credits earned; there is substantial variation across studies in <math><mrow><msub><mi>σ</mi><mrow><mo>|</mo><mi>S</mi></mrow></msub></mrow></math> for degree attainment; and baseline covariates explain less than 10% of the variation in student outcomes. These findings indicate that when planning the sample size for a study, researchers should be mindful of the follow-up period, use a range of values to calculate the MDTE for outcomes that vary across studies, and assume a value of <math><mrow><msubsup><mi>R</mi><mrow><mo>|</mo><mi>S</mi></mrow><mn>2</mn></msubsup></mrow></math> between 0 and 0.05. A public database created for this paper includes parameter estimates for additional RCTs and students.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":"47 4","pages":"599-629"},"PeriodicalIF":3.0000,"publicationDate":"2023-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/ee/45/10.1177_0193841X221121236.PMC10278387.pdf","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Evaluation Review","FirstCategoryId":"90","ListUrlMain":"https://doi.org/10.1177/0193841X221121236","RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"SOCIAL SCIENCES, INTERDISCIPLINARY","Score":null,"Total":0}
引用次数: 0

Abstract

The last two decades have seen a dramatic increase in randomized controlled trials (RCTs) conducted in community colleges. Yet, there is limited empirical information on the design parameters necessary to plan the sample size for RCTs in this context. For a blocked student-level random assignment research design, key design parameters for the minimum detectable true effect (MDTE) are the within-block outcome standard deviation (σ|S) and the within-block outcome variance explained by baseline covariates like student characteristics (R|S2). We provide empirical estimates of these key design parameters, discussing the pattern of estimates across outcomes (enrollment, credits earned, credential attainment, and grade point average), semesters, and studies. The main analyses use student-level data from 8 to 14 RCTs including 5,649-7,099 students (depending on the outcome) with follow-up data for 3 years. The following patterns are observed: the within-block standard deviation (σ|S) and therefore the MDTE can be much larger in later semesters for enrollment outcomes and cumulative credits earned; there is substantial variation across studies in σ|S for degree attainment; and baseline covariates explain less than 10% of the variation in student outcomes. These findings indicate that when planning the sample size for a study, researchers should be mindful of the follow-up period, use a range of values to calculate the MDTE for outcomes that vary across studies, and assume a value of R|S2 between 0 and 0.05. A public database created for this paper includes parameter estimates for additional RCTs and students.

Abstract Image

Abstract Image

Abstract Image

社区大学个体水平随机对照试验样本量规划的设计参数。
在过去的二十年里,在社区大学进行的随机对照试验(rct)急剧增加。然而,在这种情况下,关于规划随机对照试验样本量所需的设计参数的经验信息有限。对于分组学生水平随机分配研究设计,最小可检测真效应(MDTE)的关键设计参数是分组内结果标准差(σ|S)和分组内结果方差(R|S2),如学生特征(R|S2)。我们提供了这些关键设计参数的实证估计,讨论了结果(入学、获得的学分、获得的证书和平均绩点)、学期和研究的估计模式。主要分析使用8至14项随机对照试验的学生水平数据,包括5,649-7,099名学生(取决于结果),随访数据为3年。观察到以下模式:块内标准差(σ|S),因此,在以后的学期中,入学结果和累积学分的MDTE可能会大得多;在不同的研究中,学位获得的σ|S有很大的差异;而基线协变量只能解释不到10%的学生成绩差异。这些发现表明,在规划研究的样本量时,研究人员应注意随访期,使用一个范围的值来计算不同研究结果的MDTE,并假设R|S2在0到0.05之间。为本文创建的公共数据库包括额外随机对照试验和学生的参数估计。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Evaluation Review
Evaluation Review SOCIAL SCIENCES, INTERDISCIPLINARY-
CiteScore
2.90
自引率
11.10%
发文量
80
期刊介绍: Evaluation Review is the forum for researchers, planners, and policy makers engaged in the development, implementation, and utilization of studies aimed at the betterment of the human condition. The Editors invite submission of papers reporting the findings of evaluation studies in such fields as child development, health, education, income security, manpower, mental health, criminal justice, and the physical and social environments. In addition, Evaluation Review will contain articles on methodological developments, discussions of the state of the art, and commentaries on issues related to the application of research results. Special features will include periodic review essays, "research briefs", and "craft reports".
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信