Development of VAScoR: A rubric to qualify and score responses to the views of nature of science (VNOS) questionnaire

IF 3.6 1区 教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH
Fouad Abd-El-Khalick, Ryan Summers, Jeanne L. Brunner, Jeremy Belarmino, John Myers
{"title":"Development of VAScoR: A rubric to qualify and score responses to the views of nature of science (VNOS) questionnaire","authors":"Fouad Abd-El-Khalick,&nbsp;Ryan Summers,&nbsp;Jeanne L. Brunner,&nbsp;Jeremy Belarmino,&nbsp;John Myers","doi":"10.1002/tea.21916","DOIUrl":null,"url":null,"abstract":"<p>We report on the development of a rubric to reliably qualify and score responses to the <i>Views of Nature of Science Questionnaire</i> (VNOS): The <i>VNOS Analysis and Scoring Rubric</i> (VAScoR). The <i>VAScoR</i> is designed to (a) provide systematic guidance for the qualitative analysis, and score assignment to nuanced categories, of <i>VNOS</i> responses, (b) explicitly scaffold qualitative inferencing and standardize score assignment to substantially lessen the burden of, and variance in, analyzing and scoring the <i>VNOS</i>, and (c) improve the viability and meaningfulness of cross-study comparisons drawing on <i>VNOS</i> data. The rubric adopted the <i>VNOS's</i> consensus NOS framework and further delineated core and related elements across 10 target NOS aspects. The <i>VAScoR's</i> reliability was examined in two studies that drew on <i>VNOS</i> questionnaires completed by 185 preservice secondary science teachers (58% female; 126 undergraduate and 59 graduate students) enrolled over several years in a combined undergraduate and graduate licensure program in a large U.S. Midwestern university. In Study I, <i>VAScoR</i> analyses of 86 <i>VNOS</i> questionnaires undertaken by a single author were used to examine the rubric's intra-rater reliability, which resulted in a robust Cronbach's alpha value of 0.81. In Study II, analyses by four authors of a randomly generated, overlapping set of 18 questionnaires were used to examine inter-rater reliability, which was supported with substantial consensus among raters as indicated by a Cohen's kappa of 0.71. Further evidence for the <i>VAScoR's</i> inter-rater reliability was indicated by moderate to strong consistency among four raters with an overall Pearson's correlation coefficient of 0.82, and coefficient values ranging from 0.77 to 0.89 for six possible rater pairings.</p>","PeriodicalId":48369,"journal":{"name":"Journal of Research in Science Teaching","volume":"61 7","pages":"1641-1688"},"PeriodicalIF":3.6000,"publicationDate":"2023-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Research in Science Teaching","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/tea.21916","RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0

Abstract

We report on the development of a rubric to reliably qualify and score responses to the Views of Nature of Science Questionnaire (VNOS): The VNOS Analysis and Scoring Rubric (VAScoR). The VAScoR is designed to (a) provide systematic guidance for the qualitative analysis, and score assignment to nuanced categories, of VNOS responses, (b) explicitly scaffold qualitative inferencing and standardize score assignment to substantially lessen the burden of, and variance in, analyzing and scoring the VNOS, and (c) improve the viability and meaningfulness of cross-study comparisons drawing on VNOS data. The rubric adopted the VNOS's consensus NOS framework and further delineated core and related elements across 10 target NOS aspects. The VAScoR's reliability was examined in two studies that drew on VNOS questionnaires completed by 185 preservice secondary science teachers (58% female; 126 undergraduate and 59 graduate students) enrolled over several years in a combined undergraduate and graduate licensure program in a large U.S. Midwestern university. In Study I, VAScoR analyses of 86 VNOS questionnaires undertaken by a single author were used to examine the rubric's intra-rater reliability, which resulted in a robust Cronbach's alpha value of 0.81. In Study II, analyses by four authors of a randomly generated, overlapping set of 18 questionnaires were used to examine inter-rater reliability, which was supported with substantial consensus among raters as indicated by a Cohen's kappa of 0.71. Further evidence for the VAScoR's inter-rater reliability was indicated by moderate to strong consistency among four raters with an overall Pearson's correlation coefficient of 0.82, and coefficient values ranging from 0.77 to 0.89 for six possible rater pairings.

VAScoR的开发:一个对科学本质观点(VNOS)问卷的回答进行限定和评分的标准
我们报告了一个标准的开发,以可靠地限定和评分对科学的本质问卷(VNOS)的回答:VNOS分析和评分标准(VAScoR)。VAScoR旨在(a)为VNOS反应的定性分析和细致分类评分提供系统指导,(b)明确地支撑定性推理和标准化评分,以大大减轻VNOS分析和评分的负担和差异,以及(c)提高利用VNOS数据进行交叉研究比较的可行性和意义。该主题采用了VNOS的共识NOS框架,并进一步划定了10个目标NOS方面的核心和相关要素。VAScoR的可靠性在两项研究中进行了检验,该研究利用185名职前中学科学教师(58%女性;126名本科生和59名研究生)在美国中西部一所大型大学注册了几年的本科和研究生联合执照计划。在研究I中,对86份由单一作者进行的VNOS问卷进行VAScoR分析,以检验标题的内部信度,结果得出稳健的Cronbach's alpha值为0.81。在研究II中,四位作者对随机生成的重叠的18份问卷进行了分析,以检验评级者之间的信度,这得到了评级者之间的大量共识的支持,科恩的kappa为0.71。进一步的证据表明,四个评分者之间的一致性中等到强,总体Pearson相关系数为0.82,6个可能的评分者配对的系数值在0.77到0.89之间。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Journal of Research in Science Teaching
Journal of Research in Science Teaching EDUCATION & EDUCATIONAL RESEARCH-
CiteScore
8.80
自引率
19.60%
发文量
96
期刊介绍: Journal of Research in Science Teaching, the official journal of NARST: A Worldwide Organization for Improving Science Teaching and Learning Through Research, publishes reports for science education researchers and practitioners on issues of science teaching and learning and science education policy. Scholarly manuscripts within the domain of the Journal of Research in Science Teaching include, but are not limited to, investigations employing qualitative, ethnographic, historical, survey, philosophical, case study research, quantitative, experimental, quasi-experimental, data mining, and data analytics approaches; position papers; policy perspectives; critical reviews of the literature; and comments and criticism.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信