Research Methods in Applied Linguistics最新文献

{"title":"Genre-based fine-tuning of large language models with self-organizing maps for automated writing evaluation","authors":"Stephanie Link,&nbsp;Robert Redmon,&nbsp;Martin Hagan","doi":"10.1016/j.rmal.2025.100219","DOIUrl":"10.1016/j.rmal.2025.100219","url":null,"abstract":"<div><div>Automated Writing Evaluation (AWE) systems have significantly advanced in providing feedback for academic essay writing. However, their predominant focus on sentence-level features highlights the need for a broader approach to AWE development. While genre-based AWE systems aim to address the socio-rhetorical complexities of writing for specific audiences and purposes, their availability remains limited. This scarcity is largely due to methodological constraints in developing robust feedback engines that effectively support discipline-specific writing needs. This article describes a new method for fine-tuning large-language models (LLM) and evaluating model performance, which we refer to as G-FiT Mapping (Genre-based FIne-Tuning with self-organizing maps). This method utilizes semi-automated annotation of genre-based functional-rhetorical units of text to efficiently fine-tune an LLM and then uses self-organizing maps to evaluate and improve network performance. The G-FiT Mapping method resulted in a new automated feedback engine for an intelligent tutoring system called Dissemity, for DISSeminating research with clarITY, that supports discipline-specific, scientific writers in writing for publication. We demonstrate use of G-Fit Mapping for establishing measurable improvements in network performance, offering implications for network interpretation, genre-based AWE, and AI-based learning systems development.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 3","pages":"Article 100219"},"PeriodicalIF":0.0,"publicationDate":"2025-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144242859","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Advancing language education with ChatGPT: A path to cultivate 21st-century digital skills","authors":"Amir Reza Rahimi ,&nbsp;Ramin Teimouri","doi":"10.1016/j.rmal.2025.100218","DOIUrl":"10.1016/j.rmal.2025.100218","url":null,"abstract":"<div><div>Artificial Intelligence (AI) has become increasingly integrated into education, human life, and work as we continue to enter the 21st century, resulting in an increased expectation that learners and humans should possess more skills, referred to as 21st-century digital skills. These skills have become essential for learning, working, and living with Artificial Intelligence and the latest generation of Information and Communication Technologies (ICTs). While recent studies have examined Artificial Intelligence, particularly ChatGPT, in terms of its ability to assist language learners in developing their language skills as well as their subskills, they have not explored its role in the development of their 21st-century digital skills. For this sake, in this study the researchers integrated ChatGPT into the language classroom procedure in three high schools in Tehran, where the language teacher tried to cultivate 21st-century digital skills with ChatGPT and led students to take advantage of its features to improve their 21st-century digital skills throughout the academic year of 2023–24 and then filled out the study survey. The results of the Partial Least Square Modelling Approach (PLS-SEM) showed that ChatGPT's personalization, interactivity, accuracy, and responsiveness, along with its anthropomorphism, significantly shaped language learners' 21st-century digital skills, including critical thinking digital skills, information evaluation skills, creative digital skills, and problem-solving skills. In addition, the study found a sign of digital self-authenticity, where language learners perceived that AI-assisted language learning enhanced their 21st-century digital skills more than previous language learning contexts, albeit at the expense of their digital collaboration skills. Therefore, the study broadened the existing literature by focusing on the 21st-century digital skills of language learners, going beyond their language skills and sub-skills. It suggests that ChatGPT has the potential to enhance and develop language learners' 21st-century digital skills. However, language teachers must devise metrics to encourage learners to engage in both collaborative and personalized language learning with ChatGPT, thereby fostering the development of all 21st-century digital skills.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100218"},"PeriodicalIF":0.0,"publicationDate":"2025-06-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144204939","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Ethics for researching language and education: What the discourse of professional guidelines reveals","authors":"Theresa Austin ,&nbsp;Rosa Alejandra Medina Riveros Ph.D","doi":"10.1016/j.rmal.2025.100221","DOIUrl":"10.1016/j.rmal.2025.100221","url":null,"abstract":"<div><div>How do the guidelines provided by language and literacy professional organizations configure the ethical stances of their members, who are practitioners and researchers? As part of a larger project involving the lived experiences of teacher educators and researchers who work with multilingual populations, we present an initial critical discourse study of four professional organizations’ ethical codes of conduct. This article focuses on ethics as it materializes through contemporary textual analysis of public guidelines readily accessible to educational linguistics professionals during the 2021–2024 timeframe. It examines how ethical statements operate on the imagination of professionals desiring membership. We employ critical discourse analysis (CDA), where Van Dijk’s (1993) definition of discourse conceptualizes how language use creates power to dominate, resist, and build social hierarchies. We characterize each code as within deontic ethics by interpreting how the choice of modality guides members’ actions and values. During 2021–24 our corpus included documents detailing guidelines from AERA (2011), MLA (1992), LRA (2016), and AAAL (2017).</div><div>Our findings highlight the characteristics of these ethical codes and their differences. These documents configure an assemblage that affects members through assumed values and obligations. Furthermore, we identify unexamined ethical needs that arise in actual lived realities of researchers that require more fluid, contingent, and responsive ethics; hence, we propose a critically and dialogically engaged stance open to periodic yet continual reformulations.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100221"},"PeriodicalIF":0.0,"publicationDate":"2025-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144196368","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Using motigraphs to investigate the temporalities of motivation: Illustrating contrasting approaches","authors":"Kathryn Sidaway ,&nbsp;Junjie Li ,&nbsp;Ema Ushioda","doi":"10.1016/j.rmal.2025.100220","DOIUrl":"10.1016/j.rmal.2025.100220","url":null,"abstract":"<div><div>Factors that affect the ability to maintain motivation to learn a language are known to vary across individuals and periods of learning. The temporality of language learning motivation is therefore inherently intertwined with learners’ ability to identify and achieve their language-related goals. A motigraph is a tool that can be used to collect individual data on the temporal dynamics of language learning motivation. Through motigraphs, data is elicited from participants by asking them to self-assess their levels of motivation in real-time as a means of visualising their language learning journeys. Participants are then asked to explain and reflect on their motigraphs in follow-up interviews. In this article, we illustrate contrasting examples of how motigraphs can be used to collect data with participants at different life stages in two longitudinal qualitative studies. In the first example, the participants are adult migrants in England, where the researcher used motigraphs to elicit reflections on the participants’ past education, current struggles in a new country and often-delayed future plans. In the second example, the participants are Chinese school children, whose concept of time is still developing and whose goals are to be achieved far into the future. We discuss how each age group utilised motigraphs to explain their individual relationships with time and motivation, along with the challenges and affordances of such an approach. We conclude with suggestions for future directions for using motigraphs as a qualitative data collection tool in applied linguistics research.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100220"},"PeriodicalIF":0.0,"publicationDate":"2025-05-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144170697","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Scale validation in applied linguistics: Methodological trends and challenges","authors":"Ngo Cong-Lem","doi":"10.1016/j.rmal.2025.100217","DOIUrl":"10.1016/j.rmal.2025.100217","url":null,"abstract":"<div><div>Scale validation plays a critical role in ensuring the reliability and applicability of measurement instruments in applied linguistics research. This review examines scale validation studies published in <em>Research Methods in Applied Linguistics</em>, with the aim of identifying methodological trends and areas in need of refinement. Fourteen studies were analyzed with attention to their validation frameworks, statistical techniques, and approaches to reliability and external validation. Findings reveal a growing use of Exploratory Structural Equation Modeling (ESEM) and bifactor modeling to support construct validation in multidimensional scales, while Confirmatory Factor Analysis (CFA) remains prevalent for theory-driven, unidimensional constructs. However, external validation—particularly predictive validity and independent sample cross-validation—remains limited, reducing the generalizability of many validated instruments in domains such as language assessment and second language acquisition. Although reliability assessment is evolving through the use of Rasch modeling and Generalizability Theory, Cronbach’s alpha continues to dominate, despite its known limitations in complex constructs. Given the practical constraints faced by researchers, the review advocates for a flexible, goal-aligned approach to validation that emphasizes foundational steps such as construct conceptualization, pre-validation, and structural modeling. Enhancing predictive validity, incorporating independent sample cross-validation, and improving methodological transparency can further strengthen the rigor and practical relevance of scale development. While the scope is limited to a single journal, this review offers a roadmap for improving validation practices in applied linguistics and contributes to more robust research in language assessment, teacher education, and second language acquisition.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100217"},"PeriodicalIF":0.0,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144107675","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Theorizing and practicing the discourse-based interview in research on multilingual writers","authors":"Debra A. Friedman","doi":"10.1016/j.rmal.2025.100214","DOIUrl":"10.1016/j.rmal.2025.100214","url":null,"abstract":"<div><div>Discourse-based interviews are a method developed by composition researchers Lee Odell, Dixie Goswami, and Anne Herrington (Odell et al., 1983) in which research participants are shown a text and asked to comment on certain features. They have since been adopted by applied linguistics writing researchers to explore multilingual writers’ perspectives as both writers and readers of texts; however, they have received little attention in the applied linguistics methodological literature. This article explores how discourse-based interviews have been conceptualized and used in research on multilingual writers though a review of 45 articles and book chapters that cited Odell et al. (1983) as the source of the interview method. These studies were reviewed to identify (a) the topic or focus of the research, (b) researchers’ stated purpose for using this method, (c) how and by whom texts and focal features were selected, (d) how interviews were conducted, and (e) how researchers dealt with subjectivity. Findings indicate that studies of multilingual writers have used the term <em>discourse-based interview</em> to refer to a wide range of interview purposes and procedures that often deviate considerably from those of Odell et al. The article concludes with a reflection on areas requiring further attention if applied linguistics writing researchers are to best exploit the potential of discourse-based interviews as a research method.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100214"},"PeriodicalIF":0.0,"publicationDate":"2025-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144071879","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Generating better understandings of linguistic dissociation using multiscalar temporal accounts","authors":"Ashley R. Moore","doi":"10.1016/j.rmal.2025.100212","DOIUrl":"10.1016/j.rmal.2025.100212","url":null,"abstract":"<div><div><em>Linguistic dissociation</em> is “a relatively enduring psychosocial process in which an individual or group distances themselves from a set of linguistic practices already within their repertoire because those practices have come to connote a state of significant intersubjective disharmony, or <em>contrasubjectivity</em>” (Moore, 2023, p. 1152). Generating data to theorise the nature and causes of linguistic dissociation among particular kinds of language users represents a methodological challenge; as intersubjective phenomena, linguistic dissociation and contrasubjectivity emerge across multiple converging, connected temporal scales.</div><div>With reference to data generated with two participants from a larger critical realist grounded theory method investigation into the nature and causes of L1 dissociation among some Japanese-English late plurilinguals, I show how I combined two data generation activities with distinct temporal scales—a multimodal timeline through which participants constructed an account of their affective relationship to Japanese over their lifespan and a two-day language use journal—with follow-up interviews to produce different types of data about linguistic dissociation, which in turn made it possible for the participants and me to construct different forms of knowledge about the phenomenon. Further, I contend that we used those knowledges to arrive at better (i.e., more verisimilitudinous) understandings of the nature and causes of the L1 dissociation they had experienced. I finish by sharing methodological implications for applied linguists investing various intersubjective phenomena that shape the linguistic repertoire and reflecting on the limits of my own knowledge of linguistic dissociation.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100212"},"PeriodicalIF":0.0,"publicationDate":"2025-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144070908","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Timelines: Visualizing time in narrative research","authors":"Eric K. Ku","doi":"10.1016/j.rmal.2025.100215","DOIUrl":"10.1016/j.rmal.2025.100215","url":null,"abstract":"<div><div>Timelines have been increasingly recognized as valuable tools in narrative research for visualizing the temporal dimension of participants' lived experiences. This paper offers a comprehensive overview of the application of timelines in qualitative narrative research, particularly within applied linguistics. Narrative research methods inherently focus on time, with timelines serving to highlight the temporal aspects of critical life events, transitions, and personal transformations. By organizing and representing time visually, timelines facilitate deeper reflexivity among participants, offering insights that may not emerge through verbal interviews alone. This study synthesizes existing research on timelines, exploring their flexible application and impact on the quality of narrative data. In applied linguistics, timelines have been employed to investigate various aspects of language teaching and learning, from tracking long-term language learning trajectories to supporting teacher professional development. The paper also discusses methodological considerations, such as who constructs the timeline, when timelines should be constructed, and the medium used. Additionally, it addresses the limitations of timelines, including their tendency to present time linearly, which may not capture the complex, cyclical nature of human experiences. Drawing on examples from within and beyond applied linguistics as well as research with teachers of multiple langauges, this paper underscores the potential of timelines to enhance the rigor and depth of qualitative narrative research while also acknowledging the challenges and limitations inherent in their use.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100215"},"PeriodicalIF":0.0,"publicationDate":"2025-05-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144069308","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Research methods in L2 writing: Interdisciplinary approaches, cutting-edge methods, and practical applications","authors":"Peter Crosthwaite","doi":"10.1016/j.rmal.2025.100211","DOIUrl":"10.1016/j.rmal.2025.100211","url":null,"abstract":"","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100211"},"PeriodicalIF":0.0,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143891536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Large language models fall short in classifying learners’ open-ended responses","authors":"Atsushi Mizumoto ,&nbsp;Mark Feng Teng","doi":"10.1016/j.rmal.2025.100210","DOIUrl":"10.1016/j.rmal.2025.100210","url":null,"abstract":"<div><div>Generative Artificial Intelligence (GenAI), based on large language models (LLMs), excels in various language comprehension tasks and is increasingly utilized in applied linguistics research. This study examines the accuracy and methodological implications of using LLMs to classify open-ended responses from learners. We surveyed 143 Japanese university students studying English as a foreign language (EFL) about their essay-writing process. Two human evaluators independently classified the students’ responses based on self-regulated learning processes: planning, monitoring, and evaluation. At the same time, several LLMs performed the same classification task, and their results were compared with those of the human evaluators using Cohen’s kappa coefficient. We established κ ≥ 0.8 as the threshold for strong agreement based on rigorous methodological standards. Our findings revealed that even the best-performing model (DeepSeek-V3) achieved only moderate agreement (κ = 0.68), while other models demonstrated fair-to-moderate agreement (κ = 0.37–0.61). Surprisingly, open-source models outperformed several commercial counterparts. These results highlight the necessity of expert oversight when integrating GenAI as a support tool in qualitative data analysis. The paper concludes by discussing the methodological implications for using LLMs in qualitative research and proposing specific prompt engineering strategies to enhance their reliability in applied linguistics.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100210"},"PeriodicalIF":0.0,"publicationDate":"2025-04-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143852020","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}