{"title":"Notetaking as validity evidence: A mixed-methods investigation of question preview in EAP listening assessment","authors":"Rebecca Yeager , GoMee Park , Ray J.T. Liao","doi":"10.1016/j.jeap.2024.101346","DOIUrl":null,"url":null,"abstract":"<div><p>Recent scholarship has questioned the cognitive validity of listening tests with preview, in which test-takers can see test questions before listening. This study mined student notes for evidence of cognitive processes in listening tests with and without preview, using a mixed-methods design that explored the effect of test format on notetaking behaviors. Qualitative analysis indicated that students who previewed items were more likely to systematically omit information, highlight previewed keywords, and engage in shallower structural representation. Conversely, Kruskal-Wallis tests revealed that students who listened without preview took more notes, especially of main ideas and details, and had better coverage of the lecture. However, correlation and hierarchical linear regression analyses found these notetaking achievements did not predict higher scores in the no-preview condition, while in the preview condition, only note quantity and focus on minor ideas predicted scores. Both strands of data suggest that students' cognitive processes were shaped by the format of the exam they experienced. These findings may bear on validity arguments for listening assessment and inform the way that language instructors prepare their students for academic listening.</p></div>","PeriodicalId":47717,"journal":{"name":"Journal of English for Academic Purposes","volume":null,"pages":null},"PeriodicalIF":3.1000,"publicationDate":"2024-01-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of English for Academic Purposes","FirstCategoryId":"98","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1475158524000146","RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0
Abstract
Recent scholarship has questioned the cognitive validity of listening tests with preview, in which test-takers can see test questions before listening. This study mined student notes for evidence of cognitive processes in listening tests with and without preview, using a mixed-methods design that explored the effect of test format on notetaking behaviors. Qualitative analysis indicated that students who previewed items were more likely to systematically omit information, highlight previewed keywords, and engage in shallower structural representation. Conversely, Kruskal-Wallis tests revealed that students who listened without preview took more notes, especially of main ideas and details, and had better coverage of the lecture. However, correlation and hierarchical linear regression analyses found these notetaking achievements did not predict higher scores in the no-preview condition, while in the preview condition, only note quantity and focus on minor ideas predicted scores. Both strands of data suggest that students' cognitive processes were shaped by the format of the exam they experienced. These findings may bear on validity arguments for listening assessment and inform the way that language instructors prepare their students for academic listening.
期刊介绍:
The Journal of English for Academic Purposes provides a forum for the dissemination of information and views which enables practitioners of and researchers in EAP to keep current with developments in their field and to contribute to its continued updating. JEAP publishes articles, book reviews, conference reports, and academic exchanges in the linguistic, sociolinguistic and psycholinguistic description of English as it occurs in the contexts of academic study and scholarly exchange itself.