Language Teaching Research Quarterly最新文献_第2页

Another Reason for CDA in Language Assessment: A Critical Synthesis from a Perspective of Validating Test Constructs 语言评价中使用批评性话语分析的另一个原因:从验证测试构念的角度进行批判性综合

Language Teaching Research Quarterly Pub Date : 2023-11-01 DOI: 10.32038/ltrq.2023.37.14

Yeon-Sook Yi

{"title":"Another Reason for CDA in Language Assessment: A Critical Synthesis from a Perspective of Validating Test Constructs","authors":"Yeon-Sook Yi","doi":"10.32038/ltrq.2023.37.14","DOIUrl":"https://doi.org/10.32038/ltrq.2023.37.14","url":null,"abstract":"Cognitive diagnostic assessment is known for its capacity to provide detailed feedback to test-takers, which can inform instruction and guide learning. Besides this core strength of the assessment approach, the measurement procedure also works as a construct validation method in itself. Previous studies do not call attention to this aspect of cognitive diagnostic assessment and how the CDA validation can be different from existing construct validation methods. Thus, the purpose of this article is to synthesize cognitive diagnostic assessment research from a standpoint of seeing it as a construct validation procedure with a focus on its strengths as such. In doing so, the article closely examines studies that applied two earlier cognitive diagnostic models to learner data because they established the research foundations of CDA in language testing. Before looking into the details of the procedure of cognitive diagnostic assessment, the paper critically reviews non-CDA studies on construct validation in language testing. It then scrutinizes the methods and objects of construct validation in CDA research to find any significant differences. The paper concludes with several avenues of future research in CDA, which are related to the matter of verifying test constructs.","PeriodicalId":350461,"journal":{"name":"Language Teaching Research Quarterly","volume":"136 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135566306","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Program Evaluation and Triangulation: What Three Data Sets Show about How Participants Respond to the Course Teaching Academic English with the TOEFL iBT® Test 项目评估和三角测量:三个数据集如何显示参与者对托福网考学术英语教学课程的反应

Language Teaching Research Quarterly Pub Date : 2023-11-01 DOI: 10.32038/ltrq.2023.37.15

Gerriet Janssen, Renka Ohta, Jeremy Lee, Michael Suhan

{"title":"Program Evaluation and Triangulation: What Three Data Sets Show about How Participants Respond to the Course Teaching Academic English with the TOEFL iBT® Test","authors":"Gerriet Janssen, Renka Ohta, Jeremy Lee, Michael Suhan","doi":"10.32038/ltrq.2023.37.15","DOIUrl":"https://doi.org/10.32038/ltrq.2023.37.15","url":null,"abstract":"Drawing from a larger CIPP program evaluation (Context, Input, Processes, Products) comprised of 11 analyses of the ETS online teacher training course titled Teaching Academic English with the TOEFL iBT® Test (TAE), this paper triangulates three qualitatively different data sets and seeks to understand how participants responded to the TAE course. The research team considered self-reported data (closed- and open-answer survey questions), engagement data, and learning data. The self-reported data were strongly positive and illustrated the value participants reported having about the course content, especially ETS-created materials. Some gaps were indicated: calls were made for more materials—especially additional resources participants could use when teaching—and for increased interactions during the course workshop. In partial juxtaposition, the engagement data indicated that participants were engaging the course less than anticipated. The learning data simultaneously indicated that some assessments were quite easy for participants while elsewhere participants did not have the desired uptake of some topics. Viewing the three data sets together, the research team arrived at a more nuanced conclusion: despite participant satisfaction, development processes should continue, to ensure the highest quality teacher training program possible, in terms of its content, resources, and interactions.","PeriodicalId":350461,"journal":{"name":"Language Teaching Research Quarterly","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135566515","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Five Golden Rules for Successful Classroom Assessment Based on What We Have Learnt from JD Brown 成功课堂评估的五条黄金法则——从JD Brown那里学到的

Language Teaching Research Quarterly Pub Date : 2023-11-01 DOI: 10.32038/ltrq.2023.37.03

Aek Phakiti, Adam Steinhoff

{"title":"Five Golden Rules for Successful Classroom Assessment Based on What We Have Learnt from JD Brown","authors":"Aek Phakiti, Adam Steinhoff","doi":"10.32038/ltrq.2023.37.03","DOIUrl":"https://doi.org/10.32038/ltrq.2023.37.03","url":null,"abstract":"Classroom language assessment aims to gather various types of information related to student language learning and use and use it to inform a range of formative-summative decisions, including how to provide feedback to help students improve their learning efficacy, what teaching adjustments are needed, and whether students have demonstrated their learning attainment defined by a given syllabus (Turner, 2012). This article presents five golden rules based on the contributions of JD Brown. These golden rules address a need for a practical and accessible assessment approach that helps language teachers effectively assess students’ learning achievement in the language classroom. This article is, therefore, written for language teachers and pre-service TESOL (Teaching English to Speakers of Other Languages) students. The five golden rules presented in this article are: (1) choose the suitable types of language assessment tasks; (2) recognise the two types of assessment interpretations; (3) consider and apply basic classical test theory for analysis of the assessment of learning; (4) develop well-defined assessment criteria and rubrics for judging students’ performance; and (5) use assessment of language learning as a source of feedback to improve assessment tasks and student learning.","PeriodicalId":350461,"journal":{"name":"Language Teaching Research Quarterly","volume":"25 2","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135516059","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An AI Generated Test of Pragmatic Competence and Connected Speech 人工智能生成的语用能力和连接语的测试

Language Teaching Research Quarterly Pub Date : 2023-11-01 DOI: 10.32038/ltrq.2023.37.10

Stefan O’Grady

引用次数: 0

Shiken Across 23 Years: James Dean Brown’s Statistical Advice 跨越23年:詹姆斯·迪恩·布朗的统计建议

Language Teaching Research Quarterly Pub Date : 2023-11-01 DOI: 10.32038/ltrq.2023.37.06

Thom Hudson

引用次数: 0

Leveraging Computational Psychometrics for Language Testing 利用计算心理测量学进行语言测试

Language Teaching Research Quarterly Pub Date : 2023-11-01 DOI: 10.32038/ltrq.2023.37.11

Ardeshir Geranpayeh

引用次数: 0

Fluency in Speaking as a Dynamic Construct 作为动态结构的流利口语

Language Teaching Research Quarterly Pub Date : 2023-11-01 DOI: 10.32038/ltrq.2023.37.09

Nivja H. de Jong

引用次数: 0

James Dean Brown’s 50 Years of Work in Second Language Studies: A Systematic Review 詹姆斯·迪恩·布朗在第二语言研究方面50年的工作:系统回顾

Language Teaching Research Quarterly Pub Date : 2023-11-01 DOI: 10.32038/ltrq.2023.37.02

James Dean Brown, Ali Panahi, Hassan Mohebbi

{"title":"James Dean Brown’s 50 Years of Work in Second Language Studies: A Systematic Review","authors":"James Dean Brown, Ali Panahi, Hassan Mohebbi","doi":"10.32038/ltrq.2023.37.02","DOIUrl":"https://doi.org/10.32038/ltrq.2023.37.02","url":null,"abstract":"Panahi and Mohebbi review James Dean Brown’s 50-years of research in language testing, curriculum development and research statistics with reference to an impressionistic framework for analysis containing two components with their subcomponents: Annotations (i.e., briefing and implications) and main concepts and themes (i.e., testing and teaching terminology, research design, research instruments, data analysis, and domains). The review was carried out in two phases: In Phase I, we (Ali Panahi and Hassan Mohebbi) reviewed Brown’s all works and extracted approximately 1100 main concepts and themes leading to 28 main entries for testing and teaching terminology. The issues he has examined are much more extensive; the first 10 topics and themes most widely investigated, in the order of frequency, are language testing and assessment, research and statistics, curriculum development and language programs, cloze tests, CRTs and NRTs, TESOL, ESL, applied linguistics and language testing, placement, standardized and proficiency tests, connected speech and reduced forms, pragmatics tests and issues, and reliability and validity. In Phase II, JD Brown provides a discussion and his personal reflections on the systematic review.","PeriodicalId":350461,"journal":{"name":"Language Teaching Research Quarterly","volume":"25 4","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135516058","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ESL Teachers’ Perceptions of Effective Classroom Assessment Feedback ESL教师对有效课堂评估反馈的认知

Language Teaching Research Quarterly Pub Date : 2023-11-01 DOI: 10.32038/ltrq.2023.37.04

Alexis A. López

{"title":"ESL Teachers’ Perceptions of Effective Classroom Assessment Feedback","authors":"Alexis A. López","doi":"10.32038/ltrq.2023.37.04","DOIUrl":"https://doi.org/10.32038/ltrq.2023.37.04","url":null,"abstract":"Feedback is integral to classroom assessment. Language development is maximized if teachers and students use feedback more efficiently and effectively (Brown, 2019). This study examined how primary and secondary ESL teachers use classroom assessment feedback effectively to support student learning, actively involve young language learners in the feedback process, and support young language learners in understanding and using classroom assessment feedback. Fifteen ESL teachers were recruited for this study using convenience sampling. All the teachers participated in one-on-one semi-structured interviews through a web-based video conferencing platform. I developed an interview protocol, which helped to guide each interview and allowed me to ask follow-up questions as needed. The main aims of the interviews were to get information about the teachers’ classroom assessment practices, classroom assessment feedback, self-assessment practices, peer assessment practices, and strategies to help their students make sense of the feedback and make use of it. The findings suggest that teachers perceive classroom assessment feedback as a dialogic process in which teachers and students discuss the feedback to help students understand and use it. The teachers actively engaged students in the feedback process using peer and self-assessment. However, teachers must provide much support to help students understand and use the feedback effectively to promote learning. Some implications for practice and suggestions for future research are also provided.","PeriodicalId":350461,"journal":{"name":"Language Teaching Research Quarterly","volume":"29 11","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135565055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Mixed Methods Investigation into Test Score Users’ Perspectives about IELTS Reading Skill Profiles 分数使用者对雅思阅读技能概况看法的混合方法调查

Language Teaching Research Quarterly Pub Date : 2023-11-01 DOI: 10.32038/ltrq.2023.37.13

Eunice Eunhee Jang, Christie Barron, Hyunah Kim, Bruce Russell

{"title":"Mixed Methods Investigation into Test Score Users’ Perspectives about IELTS Reading Skill Profiles","authors":"Eunice Eunhee Jang, Christie Barron, Hyunah Kim, Bruce Russell","doi":"10.32038/ltrq.2023.37.13","DOIUrl":"https://doi.org/10.32038/ltrq.2023.37.13","url":null,"abstract":"Research on the use of standardized test scores in higher education reveals significant variations in attitudes and perceptions of language proficiency tests among test score users. Most test score users have limited knowledge about test score interpretations in terms of what English as additional language (EAL) students typically know and can do at the language proficiency levels associated with admission cut scores. To address this critical gap, the language testing field has actively investigated the potential of Diagnostic Classification Models (DCMs) to offer useful information, facilitating test score users in their decision-making processes. The present two-phase mixed methods study examined the characteristics of reading skill profiles across various IELTS band scores, specifically focusing on the most frequently used admission cut scores: 6.0, 6.5, and 7.0. The study further explored test score users’ perspectives about these admission test scores, challenges encountered by EAL students, and the usefulness of reading skill profiles derived from DCMs. Findings from the application of DCMs to IELTS reading test responses (N = 5,222) showed a lack of advanced skills, such as inferential reasoning, at these commonly employed cut scores. Test score users perceived the skill profiles as instrumental in distinguishing reading abilities across various band scores and discussed the EAL students’ lack of critical reasoning, especially in inferential and synthesis tasks in coursework. The results underscore the potential of DCM-based skill profiles in providing test score users with detailed information about test score interpretations.","PeriodicalId":350461,"journal":{"name":"Language Teaching Research Quarterly","volume":"130 2","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135566313","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0