Considering Multiple Sources of Validity Evidence Can Help to Address Challenges in the Development of Pedagogical Content Knowledge (PCK) Multiple-Choice Items

IF 2.2 3区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Research in Science Education Pub Date : 2024-12-26 DOI:10.1007/s11165-024-10227-9

Tobias Lieberei, Leroy Großmann, Virginia Deborah Elaine Welter, Dirk Krüger, Moritz Krell

{"title":"Considering Multiple Sources of Validity Evidence Can Help to Address Challenges in the Development of Pedagogical Content Knowledge (PCK) Multiple-Choice Items","authors":"Tobias Lieberei, Leroy Großmann, Virginia Deborah Elaine Welter, Dirk Krüger, Moritz Krell","doi":"10.1007/s11165-024-10227-9","DOIUrl":null,"url":null,"abstract":"The use of multiple-choice (MC) instruments to assess pedagogical content knowledge (PCK) has advantages in terms of test economy and objectivity, but it also poses challenges, for example, in terms of adequately capturing the intended construct. To help address these challenges, we developed and evaluated a new instrument to assess science teachers’ PCK of scientific reasoning in biology contexts (PCKSR-bio), considering multiple sources of validity evidence. First, 12 MC items were developed to assess crucial PCK components for three scientific reasoning skills. Subsequently, the correlation of corresponding content knowledge (CK) with the PCKSR-bio score was tested with 67 master’s students. In addition, the instrument was used in a cross-sectional study with 165 students (n = 29 bachelor, n = 115 master, n = 21 school-based preservice teachers), and the internal consistency as well as the correlation of the test score with the educational level was determined. An analysis of the response processes of 10 bachelor’s students showed that they more often referred to PCK when selecting an attractor and more often (rather intuitively) to other knowledge when selecting a distractor. In the cross-sectional study, the internal consistency was relatively low but increased with higher educational level. A correlation was found between the test score and CK but not between the test score and the educational level. Our results show that considering multiple sources of validity evidence can help to address common challenges in developing MC–PCK instruments. The results and limitations are discussed, and recommendations are made for the development of MC instruments to assess PCK in general.","PeriodicalId":47988,"journal":{"name":"Research in Science Education","volume":"34 1","pages":""},"PeriodicalIF":2.2000,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Research in Science Education","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1007/s11165-024-10227-9","RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

The use of multiple-choice (MC) instruments to assess pedagogical content knowledge (PCK) has advantages in terms of test economy and objectivity, but it also poses challenges, for example, in terms of adequately capturing the intended construct. To help address these challenges, we developed and evaluated a new instrument to assess science teachers’ PCK of scientific reasoning in biology contexts (PCK_SR-bio), considering multiple sources of validity evidence. First, 12 MC items were developed to assess crucial PCK components for three scientific reasoning skills. Subsequently, the correlation of corresponding content knowledge (CK) with the PCK_SR-bio score was tested with 67 master’s students. In addition, the instrument was used in a cross-sectional study with 165 students (n = 29 bachelor, n = 115 master, n = 21 school-based preservice teachers), and the internal consistency as well as the correlation of the test score with the educational level was determined. An analysis of the response processes of 10 bachelor’s students showed that they more often referred to PCK when selecting an attractor and more often (rather intuitively) to other knowledge when selecting a distractor. In the cross-sectional study, the internal consistency was relatively low but increased with higher educational level. A correlation was found between the test score and CK but not between the test score and the educational level. Our results show that considering multiple sources of validity evidence can help to address common challenges in developing MC–PCK instruments. The results and limitations are discussed, and recommendations are made for the development of MC instruments to assess PCK in general.

查看原文本刊更多论文

求助全文

约1分钟内获得全文求助全文

来源期刊

Research in Science Education EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

6.40

自引率

8.70%

发文量

期刊介绍： 2020 Five-Year Impact Factor: 4.021 2020 Impact Factor: 5.439 Ranking: 107/1319 (Education) – Scopus 2020 CiteScore 34.7 – Scopus Research in Science Education (RISE ) is highly regarded and widely recognised as a leading international journal for the promotion of scholarly science education research that is of interest to a wide readership. RISE publishes scholarly work that promotes science education research in all contexts and at all levels of education. This intention is aligned with the goals of Australasian Science Education Research Association (ASERA), the association connected with the journal. You should consider submitting your manscript to RISE if your research: Examines contexts such as early childhood, primary, secondary, tertiary, workplace, and informal learning as they relate to science education; and Advances our knowledge in science education research rather than reproducing what we already know. RISE will consider scholarly works that explore areas such as STEM, health, environment, cognitive science, neuroscience, psychology and higher education where science education is forefronted. The scholarly works of interest published within RISE reflect and speak to a diversity of opinions, approaches and contexts. Additionally, the journal’s editorial team welcomes a diversity of form in relation to science education-focused submissions. With this in mind, RISE seeks to publish empirical research papers. Empircal contributions are: Theoretically or conceptually grounded; Relevant to science education theory and practice; Highlight limitations of the study; and Identify possible future research opportunities. From time to time, we commission independent reviewers to undertake book reviews of recent monographs, edited collections and/or textbooks. Before you submit your manuscript to RISE, please consider the following checklist. Your paper is: No longer than 6000 words, including references. Sufficiently proof read to ensure strong grammar, syntax, coherence and good readability; Explicitly stating the significant and/or innovative contribution to the body of knowledge in your field in science education; Internationalised in the sense that your work has relevance beyond your context to a broader audience; and Making a contribution to the ongoing conversation by engaging substantively with prior research published in RISE. While we encourage authors to submit papers to a maximum length of 6000 words, in rare cases where the authors make a persuasive case that a work makes a highly significant original contribution to knowledge in science education, the editors may choose to publish longer works.