Research Methods in Applied Linguistics最新文献

Using researcher designed digital games to examine how game-mechanics impact pragmatic development: Some issues at play

Research Methods in Applied Linguistics Pub Date : 2025-04-02 DOI: 10.1016/j.rmal.2025.100203

Paul M. Richards

{"title":"Using researcher designed digital games to examine how game-mechanics impact pragmatic development: Some issues at play","authors":"Paul M. Richards","doi":"10.1016/j.rmal.2025.100203","DOIUrl":"10.1016/j.rmal.2025.100203","url":null,"abstract":"<div><div>This article makes the case for increased consideration towards variable manipulation and research design in studies that use researcher-designed digital games (RDDGs) for pragmatics instruction. Specifically, this article argues for future studies to adopt <span><span>Mayer's (2019)</span></span> value-added approach when using RDDGs to examine the influence of game-mechanics on learning outcomes. In the value-added approach, research compares learning between groups that differ according to a single design element. Previous studies that have used RDDGs to examine pragmatic development have examined sets of game-mechanics common to commercial games (e.g., points systems, leveling, hints) in pre-experimental studies (single group, pretest-posttest studies) or (quasi-)experimental studies where two treatments are contrasted without a comparison or control. To illustrate the limitations of these designs, data from Richards (2024) are reexamined according to these designs. These simulated analyses illustrate potential challenges with interpreting findings when comparison or control groups are not used and when treatments differ by multiple instructional features.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100203"},"PeriodicalIF":0.0,"publicationDate":"2025-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143748647","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Structural equation modeling (SEM) in L2 writing research: Simple tutorial and useful recommendations

Research Methods in Applied Linguistics Pub Date : 2025-04-01 DOI: 10.1016/j.rmal.2025.100202

Abdullah Alamer

{"title":"Structural equation modeling (SEM) in L2 writing research: Simple tutorial and useful recommendations","authors":"Abdullah Alamer","doi":"10.1016/j.rmal.2025.100202","DOIUrl":"10.1016/j.rmal.2025.100202","url":null,"abstract":"<div><div>Research in second language (L2) writing has witnessed a surge in the endorsement of structural equation modeling (SEM) applications. This tutorial paper highlights the advantages of using SEM in the field through a showcase of basic as well as advanced SEM analyses. I begin by illustrating how SEM can reproduce basic analyses (i.e., first-generation methods) like correlation and <em>t</em>-test. More importantly, I show how SEM enhances these analyses by effectively handling missing data and deal with non-normality which leads to more valid and unbiased findings. Beyond enhancing basic analyses, SEM is typically used for advanced analyses such as mediation and moderation. Nonetheless, particular emphasis in this paper will be on justifiying the disticntion between two types of constrcuts: (1) <em>latent variables</em> (reflective/common factors) like ‘L2 intrinsic motivation’ where items are interchangeable and similar in meaning, and (2) <em>emergent variables</em> (informative/composites) like ‘L2 writing achievement’ that is formed by distinct, but relevant, elements such as spelling, writing sample, and sentence fluency. After that, I highlight new features of SEM that analysts should be aware of. Also, concise guidelines and recommendations for using and reporting SEM, such as sample size, model estimators, fit indices, and effect sizes of the paths are provided. To enhance the practicality of this article, a step-by-step tutorial using the free software <em>Jamovi</em>, along with a simulated dataset uploaded online, is presented to enable readers to gain hands-on experience and replicate the analyses. Given the increasing accessibility of user-friendly SEM applications, researchers should adopt this powerful methodology and follow the updated guidelines.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100202"},"PeriodicalIF":0.0,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143739752","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Rating scales in L2 speech act research: A systematic review

Research Methods in Applied Linguistics Pub Date : 2025-03-27 DOI: 10.1016/j.rmal.2025.100204

Xiaoping Gao , Liming Deng

{"title":"Rating scales in L2 speech act research: A systematic review","authors":"Xiaoping Gao , Liming Deng","doi":"10.1016/j.rmal.2025.100204","DOIUrl":"10.1016/j.rmal.2025.100204","url":null,"abstract":"<div><div>This paper presents a systematic review of rating scales used in speech act studies within interlanguage pragmatics. It synthesizes the key characteristics of these scales (e.g., formats, rating criteria, theoretical foundations, and development methods) and sources of variation that influence their functionality. Primary studies were identified through keyword searches in major databases, including Google Scholar, Scopus, and Web of Science, using criteria aligned with the study's objectives. The studies in the sample examined the speech act production of second language (L2) learners of English and Asian languages (e.g., Chinese, Japanese, Korean, and Arabic), primarily elicited through written and oral Discourse Completion Tasks (DCTs) and role-plays. Earlier studies employed numerical and defined scales using a priori approaches informed by existing theories and empirical evidence to reflect advancements in the conceptualization of pragmatic competence. More recent research, however, has adopted a data-driven approach, developing and validating scales based on empirical data before implementation in further investigations. Variation in the functionality of rating scales has been examined in terms of internal sources (e.g., band levels and operational types) and external sources (e.g., rater characteristics and rater training). The findings highlight the critical role of rating scales in enhancing the reliability and validity of research methods and the generalizability of findings, thereby strengthening research rigor. This review offers valuable insights for methodological development, research training, and future research directions in L2 pragmatics.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100204"},"PeriodicalIF":0.0,"publicationDate":"2025-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143705454","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Introduction to go-alongs as a qualitative research method in applied linguistics

Research Methods in Applied Linguistics Pub Date : 2025-03-24 DOI: 10.1016/j.rmal.2025.100196

Yingqiu Chen, Louisa Buckingham

{"title":"Introduction to go-alongs as a qualitative research method in applied linguistics","authors":"Yingqiu Chen, Louisa Buckingham","doi":"10.1016/j.rmal.2025.100196","DOIUrl":"10.1016/j.rmal.2025.100196","url":null,"abstract":"<div><div>The go-along method, a place-based research approach that integrates the strengths of interviews and observations, presents opportunities for exploring the intersections of language and place, such as language use and performance (e.g., people's engagement with the linguistic landscape or people's choice of language in certain contexts or for particular tasks), and language teaching (e.g., situational language teaching, out-of-class language learning, language teaching through linguistic landscapes). Initially developed in sociology and subsequently adopted in other disciplines, go-alongs remain underutilised in applied linguistics research. Grounded in the sociocognitive approach and public pedagogy (out-of-class learning), this study demonstrates how go-alongs can conceptually link language and place by illustrating the three functions they serve: descriptive, pedagogical, and diagnostic (or evaluative). The descriptive function facilitates the documentation of participants’ language use across various places and settings. The pedagogical function supports go-along language teaching by assigning participants tasks tied to specific real-life places, and the data of place-related language use collected during task-doing is developed into materials for future teaching. The diagnostic (or evaluative) function allows for testing participants’ language skills/performances at different places. Go-alongs can also contribute to triangulating interview and observation data, thereby strengthening methodological validity and reliability. Drawing on our two empirical projects that used go-alongs as the primary research tool, we propose a set of procedural guidelines aimed at researchers or practitioners using go-alongs in applied linguistic research. We close with limitations and suggestions for future studies.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100196"},"PeriodicalIF":0.0,"publicationDate":"2025-03-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143680281","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Spatiotemporal scales in narrative inquiry research on language identity development during study abroad

Research Methods in Applied Linguistics Pub Date : 2025-03-22 DOI: 10.1016/j.rmal.2025.100201

Yulia Kharchenko

{"title":"Spatiotemporal scales in narrative inquiry research on language identity development during study abroad","authors":"Yulia Kharchenko","doi":"10.1016/j.rmal.2025.100201","DOIUrl":"10.1016/j.rmal.2025.100201","url":null,"abstract":"<div><div>This paper examines how language identity of international English language students develops in study abroad settings. The methods used in this study stem from different narrative approaches that focus on aspects of identity and narrative form. The analytical focus is on various scales of time and space over which identity developments occur, from historic to the more immediate contexts of interaction. First, autobiographical narratives are constructed using thematic analysis of interview and diary data. To complement the big-picture findings, short stories of critical identity episodes are analysed in terms of content and context. Finally, the small-scale analysis of interview data provides insights into how identity work is interactively constructed. The diverse findings are unified into a rich research narrative by applying the concept of spatiotemporal scales as an overarching, unifying logic. It is argued that a combination of narrative inquiry approaches and the spatiotemporal perspective results in a more nuanced understanding of identity dynamics of language learners. The findings demonstrate a range of language-related identity developments in a single case study and encourage further narrative studies in applied linguistics.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100201"},"PeriodicalIF":0.0,"publicationDate":"2025-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143680341","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Assigning CEFR-J levels to English learners’ writing: An approach using lexical metrics and generative AI

Research Methods in Applied Linguistics Pub Date : 2025-03-19 DOI: 10.1016/j.rmal.2025.100199

Satoru Uchida , Masashi Negishi

{"title":"Assigning CEFR-J levels to English learners’ writing: An approach using lexical metrics and generative AI","authors":"Satoru Uchida , Masashi Negishi","doi":"10.1016/j.rmal.2025.100199","DOIUrl":"10.1016/j.rmal.2025.100199","url":null,"abstract":"<div><div>This study presents the CEFR-based Writing Level Analyzer (CWLA), a novel automated system for assessing English learners’ writing proficiency. It assesses proficiency according to CEFR-J levels, a finely graded adaptation of the CEFR framework tailored to the English as a Foreign Language context, particularly in Japan. CWLA generates scores by combining vocabulary scores with AI-based analytical scores, allowing for a sophisticated, regression-based alignment with CEFR-J proficiency levels. By leveraging both straightforward lexical metrics and advanced AI scoring, CWLA provides accurate and interpretable assessments accessible via a user-friendly web interface. To evaluate CWLA's effectiveness, we conducted validation using the ICNALE GRA dataset, which showed a strong correlation of 0.88 between CWLA scores and human ratings. Additionally, entropy analysis indicated that CWLA's scoring distribution closely resembles human rater patterns, capturing the variability expected in human assessments. Three CEFR/CEFR-J specialists also performed expert validation, resulting in an agreement rate of 83.33 %, thereby supporting the system's alignment with expert judgment. These results suggest that CWLA is a reliable system for detailed CEFR-J level assessment, offering promising applications in language education and learner assessment across proficiency levels.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100199"},"PeriodicalIF":0.0,"publicationDate":"2025-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143644269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Systematic comparison of computational measures of linguistic synchrony in online educational environments

Research Methods in Applied Linguistics Pub Date : 2025-03-13 DOI: 10.1016/j.rmal.2025.100195

Jinnie Shin, A. Pauline Aguinalde

{"title":"Systematic comparison of computational measures of linguistic synchrony in online educational environments","authors":"Jinnie Shin, A. Pauline Aguinalde","doi":"10.1016/j.rmal.2025.100195","DOIUrl":"10.1016/j.rmal.2025.100195","url":null,"abstract":"<div><div>Linguistic synchrony, the alignment of linguistic features between conversational partners, is a key indicator for understanding students’ learning outcomes in computer-mediated learning environments, where communication quality directly influences success. In online tutoring, synchrony fosters better comprehension and engagement. Despite numerous methods to measure synchrony—such as lexical, syntactic, and semantic alignment—systematic comparisons across these approaches remain limited, impeding a full understanding of how synchrony operates across different educational contexts. This study addresses this gap by analyzing tutoring conversations from diverse contexts, particularly English as a Second Language (ESL) tutoring and algebra instruction, using seven computational models. These models yielded 29 synchrony indices, ranging from traditional lexical overlap to advanced natural language processing-based embedding models, capturing synchrony across multiple dimensions. Our results show that while these methods generally align with theoretical dimensions of synchrony, variations in embedding methods and distance measures can significantly impact their ability to capture meaningful interactions. Additionally, different subject domains influenced the finer-grained dimensions of synchrony observed, particularly in temporal aspects.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100195"},"PeriodicalIF":0.0,"publicationDate":"2025-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143610744","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Centering Pacific knowledge of time, space, and relationality in community-based translanguaging research: An engaged and fluid temporal methodology

Research Methods in Applied Linguistics Pub Date : 2025-03-13 DOI: 10.1016/j.rmal.2025.100198

Honiara Amosa , Sam Amosa , Corinne A. Seals

{"title":"Centering Pacific knowledge of time, space, and relationality in community-based translanguaging research: An engaged and fluid temporal methodology","authors":"Honiara Amosa , Sam Amosa , Corinne A. Seals","doi":"10.1016/j.rmal.2025.100198","DOIUrl":"10.1016/j.rmal.2025.100198","url":null,"abstract":"<div><div>In Western perspectives, space and place are often viewed as distinct concepts. Place refers to a specific physical location, whereas space describes the interactions that take place within that location. Time is typically understood as a fixed moment or the interval between moments. In contrast, Pacific perspectives see time, space, and place as interconnected with each other and with individuals. In this worldview, a space or place continues to resonate even after one leaves it, and time is seen as circular rather than linear, linking the past, present, and future. This interconnectedness is embodied in the concept of 'va', which emphasizes the importance of understanding relationships and responsibilities within the community involved in research (cf. Airini et al., 2010). This article argues for the importance of recognizing Indigenous epistemologies and ontologies as valid in their own right, which in this case means recognizing the importance of va in Pacific research. We also present our theorization of an Engaged and Fluid Temporal Methodology, which outlines via five principles how Pacific understandings guided our work with Pacific communities. We then walk through each of the five principles as a methodological guide for community-based research, providing illustrative examples from the Wellington Translanguaging Project. By focusing on principles to research that are imbued with respective Pacific understandings of time, space, and relationships (e.g. va), we argue that this supports the research to remain in and for the communities with which we worked, a core component of Indigenous research methodology.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 1","pages":"Article 100198"},"PeriodicalIF":0.0,"publicationDate":"2025-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143610704","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Assessing Whisper automatic speech recognition and WER scoring for elicited imitation: Steps toward automation

Research Methods in Applied Linguistics Pub Date : 2025-03-11 DOI: 10.1016/j.rmal.2025.100197

Michael McGuire , Jenifer Larson-Hall

{"title":"Assessing Whisper automatic speech recognition and WER scoring for elicited imitation: Steps toward automation","authors":"Michael McGuire , Jenifer Larson-Hall","doi":"10.1016/j.rmal.2025.100197","DOIUrl":"10.1016/j.rmal.2025.100197","url":null,"abstract":"<div><div>Elicited imitation (EI) has received considerable attention in the field of SLA as a psycholinguistic method for oral proficiency assessment. However, EI tests are perhaps underutilized because of the need for time-consuming scoring by hand. A clear solution to this is computer automation, which requires two key components to work in parallel: (1) an accurate automatic speech recognition (ASR) system to transcribe EI response speech into text, and (2) a computational scoring metric to assess the resulting text. In this paper, we test the feasibility of automating EI scoring with the open-source Whisper ASR system and the Word Error Rate (<em>WER</em>) scoring metric which measures deviation from the original prompt. 30 Japanese L1 participants took a 30-item test, resulting in 900 English L2 learner EI sentence responses which were transcribed by the two authors and Whisper ASR. The intraclass correlation coefficient (ICC) between the error rates of the human raters and Whisper across all items was ICC = 0.929, 95 % CI [0.921, 0.936] indicating very strong alignment. We then compared automated test scores using <em>WER</em> to those done manually by human raters using a traditional ordinal-scale scoring method developed by Ortega et al. (2002) and found a robust correlation of <em>r</em> = 0.969, 95 % CI [0.935, 0.985] across overall participant scores. These findings show that the combination of the Whisper ASR system and the <em>WER</em> scoring metric result in EI test scores that align almost perfectly with and are arguably an improvement on currently accepted methods.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 1","pages":"Article 100197"},"PeriodicalIF":0.0,"publicationDate":"2025-03-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143592104","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Narrative trajectory in identity and subjectivity research in applied linguistics

Research Methods in Applied Linguistics Pub Date : 2025-03-05 DOI: 10.1016/j.rmal.2025.100189

Steven Yeung

{"title":"Narrative trajectory in identity and subjectivity research in applied linguistics","authors":"Steven Yeung","doi":"10.1016/j.rmal.2025.100189","DOIUrl":"10.1016/j.rmal.2025.100189","url":null,"abstract":"<div><div>Studying the trajectory of a social actor has attracted the attention of applied linguists. The construct of trajectory connects one's biographical histories, situated practices and experiences and their ensuing consequences vis-à-vis wider social, cultural, political and economic spaces. Trajectory contributes to identity and subjectivity research by allowing for the examination of evolving identities and subjectivities over time, thereby shedding light on the process of becoming. In identity and subjectivity research, stance is often used as a heuristic. However, stance only captures identity and subjectivity at a particular point in time; it does not help to trace identity trajectories that take place across timescales and events. This paper therefore introduces the idea of <em>narrative trajectory</em> as an analytical approach, drawing on sociolinguistic and linguistic anthropological work on stancetaking, chronotope and interdiscursivity. Social actors take stances through narrated events and narrating events, which can form narrated and narrating trajectories respectively if interdiscursive links between events can be traced. The tracing of such links can be accomplished by attending to various types of signs, models or social positions and relevant spatial-temporal configurations (chronotopes) across multiple narrated and narrating trajectories. Taken together, this multiplicity of trajectories forms an overall narrative trajectory. This paper discusses the tracing of a narrative trajectory with an ethnographic case study of an English learner in Hong Kong. The concept was applied to show how her identity as a struggling learner emerged and evolved across time and space. Recommendations for identity and subjectivity research in applied linguistics are also provided.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 1","pages":"Article 100189"},"PeriodicalIF":0.0,"publicationDate":"2025-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143549577","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0