Research Methods in Applied Linguistics最新文献_第7页

LexJP: A test to efficiently assess vocabulary knowledge of Japanese 一个有效评估日语词汇知识的测试

Research Methods in Applied Linguistics Pub Date : 2025-04-14 DOI: 10.1016/j.rmal.2025.100208

Yui Suzukida , Kazuya Saito

引用次数: 0

Network Analysis for the investigation of rater effects in language assessment: A comparison of ChatGPT vs human raters 语言评估中评价者效应调查的网络分析：ChatGPT与人类评价者的比较

Research Methods in Applied Linguistics Pub Date : 2025-04-11 DOI: 10.1016/j.rmal.2025.100205

Iasonas Lamprianou

{"title":"Network Analysis for the investigation of rater effects in language assessment: A comparison of ChatGPT vs human raters","authors":"Iasonas Lamprianou","doi":"10.1016/j.rmal.2025.100205","DOIUrl":"10.1016/j.rmal.2025.100205","url":null,"abstract":"<div><div>A recent study by Yamashita (2024) showcases the usefulness of Many-Facet Rasch Model (MFRM) for the analysis of rater effects within the context of Automated Essay Scoring (AES). Building upon Yamashita's work, we break new ground by using Network Analysis (NA) to interrogate the same dataset comparing ChatGPT and human raters for the evaluation of 136 essays. We replicate the analysis of the original study and show a near-perfect agreement between the results of NA and MFRM. We extend the original study by providing strong evidence of halo effect in the data (including the ChatGPT ratings) and propose two new statistics to assess the consistency of raters. We also present simulation studies to show that the NA estimation algorithm is robust, even with small and sparse datasets. Finally, we provide practical guidelines for researchers seeking to use NA with their own datasets. We argue that NA can complement established methodologies, such as the MFRM, but can also be used independently, leveraging its strong visual representations. Relevant algorithms and R code are provided in the Online Appendix to support researchers and practitioners in replicating our findings.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100205"},"PeriodicalIF":0.0,"publicationDate":"2025-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143816307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The development of tasks for measuring morphological awareness and word-meaning inference in L2 Russian 二语俄语词形意识和词义推理测试任务的发展

Research Methods in Applied Linguistics Pub Date : 2025-04-09 DOI: 10.1016/j.rmal.2025.100209

Rossina Soyan

引用次数: 0

Making time count: Kairos as temporal, methodological, and ethical innovation in qualitative applied linguistics research 让时间有价值：Kairos在定性应用语言学研究中的时间、方法和伦理创新

Research Methods in Applied Linguistics Pub Date : 2025-04-08 DOI: 10.1016/j.rmal.2025.100207

Jacob Rieker, Meredith Doran

{"title":"Making time count: Kairos as temporal, methodological, and ethical innovation in qualitative applied linguistics research","authors":"Jacob Rieker, Meredith Doran","doi":"10.1016/j.rmal.2025.100207","DOIUrl":"10.1016/j.rmal.2025.100207","url":null,"abstract":"<div><div>Responding to calls to interrogate the role of temporality in applied linguistics research methods, this paper explores the utility and value of <em>kairos</em> as a temporal and methodological logic in qualitative inquiry. In contrast to the predominant understanding of time as <em>chronos</em>, the objective passage of clock time, <em>kairos</em> represents the qualitative experience of time by the individual as personally significant moments that constitute turning points and major transitions in development. To illustrate how <em>kairos</em> can be productively integrated in qualitative research, we present a case study of a research project on faculty development in the context of English as a Medium of Instruction that was designed to create change in participants’ instructional decision making and practices. The analysis illustrates how the cultivation of <em>kairos</em>, defined as opportune, relevant, and meaningful experiences for participants, guided the study design and implementation, and enabled faculty-participants to implement new teaching practices to the benefit of their students’ engagement and learning. We conclude by considering the temporal, methodological, and ethical implications of attending to time as <em>kairos</em>, and by advocating for the potential benefits of a kairic orientation to qualitative inquiry in applied linguistics.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100207"},"PeriodicalIF":0.0,"publicationDate":"2025-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143791359","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The volunteer participation paradox: Ethical tensions between self-selection and targeted sampling 志愿者参与悖论：自我选择与目标抽样之间的伦理矛盾

Research Methods in Applied Linguistics Pub Date : 2025-04-08 DOI: 10.1016/j.rmal.2025.100206

Takaaki Hiratsuka

{"title":"The volunteer participation paradox: Ethical tensions between self-selection and targeted sampling","authors":"Takaaki Hiratsuka","doi":"10.1016/j.rmal.2025.100206","DOIUrl":"10.1016/j.rmal.2025.100206","url":null,"abstract":"<div><div>The voluntary nature of research participation is a cornerstone of ethical practices in studies involving human participants. However, when individuals voluntarily respond to participant calls or are selected through convenient methods, they often share traits like proximity or willingness—possibly leading to data skewed by similar attitudes. Conversely, methods aimed at diversity and inclusion, such as snowballing or maximum variation sampling, might inadvertently compromise the voluntary nature of participation. This tension, which I term the <em>volunteer participation paradox</em>, presents researchers with a methodological dilemma. That is, when participation is entirely voluntary, research may face limitations due to potentially unbalanced data. In contrast, when researchers employ targeted sampling strategies to gather broad-ranging data, the authenticity of voluntary participation may be influenced by factors such as social obligation or researcher relationships. This article explores manifestations of this methodological tension and puts forward guidance for navigating the complex ethical terrain in research methodology.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100206"},"PeriodicalIF":0.0,"publicationDate":"2025-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143791360","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Using researcher designed digital games to examine how game-mechanics impact pragmatic development: Some issues at play 利用研究人员设计的数字游戏来研究游戏机制如何影响实用主义发展：游戏中的一些问题

Research Methods in Applied Linguistics Pub Date : 2025-04-02 DOI: 10.1016/j.rmal.2025.100203

Paul M. Richards

{"title":"Using researcher designed digital games to examine how game-mechanics impact pragmatic development: Some issues at play","authors":"Paul M. Richards","doi":"10.1016/j.rmal.2025.100203","DOIUrl":"10.1016/j.rmal.2025.100203","url":null,"abstract":"<div><div>This article makes the case for increased consideration towards variable manipulation and research design in studies that use researcher-designed digital games (RDDGs) for pragmatics instruction. Specifically, this article argues for future studies to adopt <span><span>Mayer's (2019)</span></span> value-added approach when using RDDGs to examine the influence of game-mechanics on learning outcomes. In the value-added approach, research compares learning between groups that differ according to a single design element. Previous studies that have used RDDGs to examine pragmatic development have examined sets of game-mechanics common to commercial games (e.g., points systems, leveling, hints) in pre-experimental studies (single group, pretest-posttest studies) or (quasi-)experimental studies where two treatments are contrasted without a comparison or control. To illustrate the limitations of these designs, data from Richards (2024) are reexamined according to these designs. These simulated analyses illustrate potential challenges with interpreting findings when comparison or control groups are not used and when treatments differ by multiple instructional features.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100203"},"PeriodicalIF":0.0,"publicationDate":"2025-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143748647","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Structural equation modeling (SEM) in L2 writing research: Simple tutorial and useful recommendations 结构方程建模（SEM）在第二语言写作研究：简单的教程和有用的建议

Research Methods in Applied Linguistics Pub Date : 2025-04-01 DOI: 10.1016/j.rmal.2025.100202

Abdullah Alamer

{"title":"Structural equation modeling (SEM) in L2 writing research: Simple tutorial and useful recommendations","authors":"Abdullah Alamer","doi":"10.1016/j.rmal.2025.100202","DOIUrl":"10.1016/j.rmal.2025.100202","url":null,"abstract":"<div><div>Research in second language (L2) writing has witnessed a surge in the endorsement of structural equation modeling (SEM) applications. This tutorial paper highlights the advantages of using SEM in the field through a showcase of basic as well as advanced SEM analyses. I begin by illustrating how SEM can reproduce basic analyses (i.e., first-generation methods) like correlation and <em>t</em>-test. More importantly, I show how SEM enhances these analyses by effectively handling missing data and deal with non-normality which leads to more valid and unbiased findings. Beyond enhancing basic analyses, SEM is typically used for advanced analyses such as mediation and moderation. Nonetheless, particular emphasis in this paper will be on justifiying the disticntion between two types of constrcuts: (1) <em>latent variables</em> (reflective/common factors) like ‘L2 intrinsic motivation’ where items are interchangeable and similar in meaning, and (2) <em>emergent variables</em> (informative/composites) like ‘L2 writing achievement’ that is formed by distinct, but relevant, elements such as spelling, writing sample, and sentence fluency. After that, I highlight new features of SEM that analysts should be aware of. Also, concise guidelines and recommendations for using and reporting SEM, such as sample size, model estimators, fit indices, and effect sizes of the paths are provided. To enhance the practicality of this article, a step-by-step tutorial using the free software <em>Jamovi</em>, along with a simulated dataset uploaded online, is presented to enable readers to gain hands-on experience and replicate the analyses. Given the increasing accessibility of user-friendly SEM applications, researchers should adopt this powerful methodology and follow the updated guidelines.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100202"},"PeriodicalIF":0.0,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143739752","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Rating scales in L2 speech act research: A systematic review 二语言语行为研究中的评定量表：系统回顾

Research Methods in Applied Linguistics Pub Date : 2025-03-27 DOI: 10.1016/j.rmal.2025.100204

Xiaoping Gao , Liming Deng

{"title":"Rating scales in L2 speech act research: A systematic review","authors":"Xiaoping Gao , Liming Deng","doi":"10.1016/j.rmal.2025.100204","DOIUrl":"10.1016/j.rmal.2025.100204","url":null,"abstract":"<div><div>This paper presents a systematic review of rating scales used in speech act studies within interlanguage pragmatics. It synthesizes the key characteristics of these scales (e.g., formats, rating criteria, theoretical foundations, and development methods) and sources of variation that influence their functionality. Primary studies were identified through keyword searches in major databases, including Google Scholar, Scopus, and Web of Science, using criteria aligned with the study's objectives. The studies in the sample examined the speech act production of second language (L2) learners of English and Asian languages (e.g., Chinese, Japanese, Korean, and Arabic), primarily elicited through written and oral Discourse Completion Tasks (DCTs) and role-plays. Earlier studies employed numerical and defined scales using a priori approaches informed by existing theories and empirical evidence to reflect advancements in the conceptualization of pragmatic competence. More recent research, however, has adopted a data-driven approach, developing and validating scales based on empirical data before implementation in further investigations. Variation in the functionality of rating scales has been examined in terms of internal sources (e.g., band levels and operational types) and external sources (e.g., rater characteristics and rater training). The findings highlight the critical role of rating scales in enhancing the reliability and validity of research methods and the generalizability of findings, thereby strengthening research rigor. This review offers valuable insights for methodological development, research training, and future research directions in L2 pragmatics.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100204"},"PeriodicalIF":0.0,"publicationDate":"2025-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143705454","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Introduction to go-alongs as a qualitative research method in applied linguistics 应用语言学中作为定性研究方法的“随迁”导论

Research Methods in Applied Linguistics Pub Date : 2025-03-24 DOI: 10.1016/j.rmal.2025.100196

Yingqiu Chen, Louisa Buckingham

{"title":"Introduction to go-alongs as a qualitative research method in applied linguistics","authors":"Yingqiu Chen, Louisa Buckingham","doi":"10.1016/j.rmal.2025.100196","DOIUrl":"10.1016/j.rmal.2025.100196","url":null,"abstract":"<div><div>The go-along method, a place-based research approach that integrates the strengths of interviews and observations, presents opportunities for exploring the intersections of language and place, such as language use and performance (e.g., people's engagement with the linguistic landscape or people's choice of language in certain contexts or for particular tasks), and language teaching (e.g., situational language teaching, out-of-class language learning, language teaching through linguistic landscapes). Initially developed in sociology and subsequently adopted in other disciplines, go-alongs remain underutilised in applied linguistics research. Grounded in the sociocognitive approach and public pedagogy (out-of-class learning), this study demonstrates how go-alongs can conceptually link language and place by illustrating the three functions they serve: descriptive, pedagogical, and diagnostic (or evaluative). The descriptive function facilitates the documentation of participants’ language use across various places and settings. The pedagogical function supports go-along language teaching by assigning participants tasks tied to specific real-life places, and the data of place-related language use collected during task-doing is developed into materials for future teaching. The diagnostic (or evaluative) function allows for testing participants’ language skills/performances at different places. Go-alongs can also contribute to triangulating interview and observation data, thereby strengthening methodological validity and reliability. Drawing on our two empirical projects that used go-alongs as the primary research tool, we propose a set of procedural guidelines aimed at researchers or practitioners using go-alongs in applied linguistic research. We close with limitations and suggestions for future studies.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100196"},"PeriodicalIF":0.0,"publicationDate":"2025-03-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143680281","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Spatiotemporal scales in narrative inquiry research on language identity development during study abroad 海外留学期间语言认同发展叙事探究研究的时空尺度

Research Methods in Applied Linguistics Pub Date : 2025-03-22 DOI: 10.1016/j.rmal.2025.100201

Yulia Kharchenko

{"title":"Spatiotemporal scales in narrative inquiry research on language identity development during study abroad","authors":"Yulia Kharchenko","doi":"10.1016/j.rmal.2025.100201","DOIUrl":"10.1016/j.rmal.2025.100201","url":null,"abstract":"<div><div>This paper examines how language identity of international English language students develops in study abroad settings. The methods used in this study stem from different narrative approaches that focus on aspects of identity and narrative form. The analytical focus is on various scales of time and space over which identity developments occur, from historic to the more immediate contexts of interaction. First, autobiographical narratives are constructed using thematic analysis of interview and diary data. To complement the big-picture findings, short stories of critical identity episodes are analysed in terms of content and context. Finally, the small-scale analysis of interview data provides insights into how identity work is interactively constructed. The diverse findings are unified into a rich research narrative by applying the concept of spatiotemporal scales as an overarching, unifying logic. It is argued that a combination of narrative inquiry approaches and the spatiotemporal perspective results in a more nuanced understanding of identity dynamics of language learners. The findings demonstrate a range of language-related identity developments in a single case study and encourage further narrative studies in applied linguistics.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100201"},"PeriodicalIF":0.0,"publicationDate":"2025-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143680341","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0