Research Methods in Applied Linguistics最新文献_第2页

Towards more appropriate modelling of linguistic complexity measures: Beyond traditional regression models

Research Methods in Applied Linguistics Pub Date : 2025-01-16 DOI: 10.1016/j.rmal.2025.100182

Akira Murakami

{"title":"Towards more appropriate modelling of linguistic complexity measures: Beyond traditional regression models","authors":"Akira Murakami","doi":"10.1016/j.rmal.2025.100182","DOIUrl":"10.1016/j.rmal.2025.100182","url":null,"abstract":"<div><div>Despite a recent emphasis on appropriate quantitative data analysis as part of the methodological reform, applied linguists often overlook scrutinising the adequacy of their analytical methods, risking model misspecification. This article critiques the use of regression models assuming normal error distributions for modelling linguistic complexity measures. It examines two alternative approaches: weighted linear regression with a log-transformed outcome variable and negative binomial regression, demonstrating how they mitigate associated limitations. Normal error regression models are inadequate for modelling count-based ratio variables due to two main issues: (i) count variables are theoretically lower-bounded, a constraint not addressed by normal error regression models, and (ii) variations in count quantities lead to differences in sampling variability, violating the homoscedasticity assumption and potentially inflating the Type I error rate. Analysis of 14 syntactic complexity measures and an artificial-data simulation show that the lower bound of the prediction intervals for normal error regression models often falls below the theoretical minimum in realistic scenarios. Moreover, the denominator count of syntactic complexity measures negatively correlates with variability, causing heteroscedasticity, a higher false-positive rate, and reduced true value coverage by 80% confidence intervals. Both alternative approaches outperform normal error regression models in these criteria. These findings challenge the suitability of normal error regressions for modelling syntactic complexity measures and caution against their use for other count-based measures in second language research and corpus linguistics. This necessitates reevaluating the widespread use of normal error regression in applied linguistics, urging methodologists to develop and validate more structurally faithful modelling approaches.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 1","pages":"Article 100182"},"PeriodicalIF":0.0,"publicationDate":"2025-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143101667","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

CA and MCA to advance diversity, equity, and inclusion (DEI): A case for motivated looking

Research Methods in Applied Linguistics Pub Date : 2025-01-16 DOI: 10.1016/j.rmal.2025.100185

Steven Talmy

引用次数: 0

Embracing temporality: Reflexive insights into positionality and relational dynamics in intercultural research

Research Methods in Applied Linguistics Pub Date : 2025-01-13 DOI: 10.1016/j.rmal.2025.100183

Angie Baily

{"title":"Embracing temporality: Reflexive insights into positionality and relational dynamics in intercultural research","authors":"Angie Baily","doi":"10.1016/j.rmal.2025.100183","DOIUrl":"10.1016/j.rmal.2025.100183","url":null,"abstract":"<div><div>Reflexivity has increasingly become a focal point in qualitative research as researchers embrace their evolving roles and presence in the research process. In this article, I explore the complexities of reflexive practice, highlighting notions of relationality, intercultural research and hierarchical structures. Through personal reflexive diaries, I investigate how nuanced shifts and transformations in my identity and positionality were negotiated over time. Specifically, I reflect on my transition from researcher to <em>Dasao</em> (sister-in-law) during my Ph.D. fieldwork, where cultural and linguistic immersion simultaneously guided and challenged my inquiry. This journey confronted me with unforeseen complexities, prompting deeper engagement with reflexivity and positionality. I discuss the fear of becoming too involved in the research and with the participants, potentially (mis)leading the data. I describe my struggles moving in and out of fluid researcher roles, alternating between researcher/participant, insider/outsider, learned/learner - positions interwoven with, and inseparable from my own life experience. While time was central to gaining, understanding and developing positionality, it also highlighted my own research insecurities, which became further exposed by unplanned timescales, leading to unexpected friendships, fluid relational boundaries and an all-consuming reflexivity which thrived on time. By focussing on temporal dimensions, this study offers an innovative perspective on the challenges of reflexivity, the negotiation of roles and blurring of boundaries, researcher vulnerability and the dilemmas around accepting one's presence in the data.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 1","pages":"Article 100183"},"PeriodicalIF":0.0,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143101665","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Vocabulary list generator: A digital tool to generate frequency-based word lists adjusted for dispersion

Research Methods in Applied Linguistics Pub Date : 2025-01-09 DOI: 10.1016/j.rmal.2025.100180

Ashleigh Cox , Daniel Dixon , Tülay Dixon

{"title":"Vocabulary list generator: A digital tool to generate frequency-based word lists adjusted for dispersion","authors":"Ashleigh Cox , Daniel Dixon , Tülay Dixon","doi":"10.1016/j.rmal.2025.100180","DOIUrl":"10.1016/j.rmal.2025.100180","url":null,"abstract":"<div><div>In second and foreign language teaching, helping students learn new vocabulary is an important goal, and many researchers aim to help teachers determine which vocabulary items should be prioritized by making predictions about the words that specific learner populations are likely to need. Such predictions can be made by examining the words that frequently appear in corpora that are representative of the learners’ target language use domains. While early vocabulary lists tended to consider only frequency and range when ranking important words in a domain, more recently, researchers have argued for more robust measures of dispersion, including <em>evenness</em> (i.e., how evenly spread a word is across texts) and <em>pervasiveness</em> (i.e., the proportion of texts using a word, which is similar to range) (Egbert & Burch, 2023). However, the current tools and methods that are commonly used to form corpus-based vocabulary lists often do not include these dispersion measures, especially the measure of <em>evenness</em>. To make it easier for researchers and teachers to inform language learning goals with vocabulary lists that consider both frequency and recommended dispersion measures, this report describes and demonstrates a new digital tool, the <em>Vocabulary List Generator</em>. The tool was piloted with a corpus of psychology journal articles, and the vocabulary list generated from the corpus demonstrates its use and output.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 1","pages":"Article 100180"},"PeriodicalIF":0.0,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143101664","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Making sense of non-comprehension issues while listening: A data-based coding scheme

Research Methods in Applied Linguistics Pub Date : 2025-01-09 DOI: 10.1016/j.rmal.2025.100181

Joseph Siegel, Maria Kuteeva, Aki Siegel

引用次数: 0

Challenges and opportunities of design-based research in applied linguistics: Insights from a scoping review

Research Methods in Applied Linguistics Pub Date : 2025-01-08 DOI: 10.1016/j.rmal.2024.100178

Zehui Yang, Yan-Yi Lee

引用次数: 0

Exploring the application of keystroke logging techniques to research in second language (L2) writing

Research Methods in Applied Linguistics Pub Date : 2025-01-04 DOI: 10.1016/j.rmal.2024.100179

Yu Tian , Sara T. Cushing

引用次数: 0

Gauging the validity of machine learning-based temporal feature annotation to measure fluency in speech automatically

Research Methods in Applied Linguistics Pub Date : 2024-12-31 DOI: 10.1016/j.rmal.2024.100177

Ryuki Matsuura , Shungo Suzuki , Kotaro Takizawa , Mao Saeki , Yoichi Matsuyama

{"title":"Gauging the validity of machine learning-based temporal feature annotation to measure fluency in speech automatically","authors":"Ryuki Matsuura , Shungo Suzuki , Kotaro Takizawa , Mao Saeki , Yoichi Matsuyama","doi":"10.1016/j.rmal.2024.100177","DOIUrl":"10.1016/j.rmal.2024.100177","url":null,"abstract":"<div><div>Machine learning (ML) techniques allow for automatically annotating various temporal speech features, particularly by the cascade connection of ML-based modules. Although such systems are expected to enhance scalability of second language (L2) speech research, their annotation accuracy is potentially moderated by speaking tasks and proficiency levels due to the mismatch between training and real-world data. Accordingly, we developed and validated an ML-based temporal feature annotation system on L2 English datasets split by speaking tasks (monologic vs. dialogic tasks) and proficiency levels, operationalized as overall fluency levels (low, mid vs. high). We compared the annotations by experts and the system in terms of the agreement between manual and automatic annotations, correlations between manual and automatic measures, and the predictive power for listener-based fluency judgments. Results showed a substantial degree of agreement in the annotations for monologic tasks and a general tendency of strong correlations between manual and automatic measures regardless of tasks and overall fluency levels. Furthermore, automatic measures yielded substantial predictive power of fluency scores in monologic tasks. These findings suggest the substantial applicability of ML-based annotation systems to monologic tasks possibly without biases by holistic levels of fluency.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 1","pages":"Article 100177"},"PeriodicalIF":0.0,"publicationDate":"2024-12-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143101737","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Applying lexical sophistication models to wordlist development: A proof-of-concept study

Research Methods in Applied Linguistics Pub Date : 2024-12-17 DOI: 10.1016/j.rmal.2024.100175

Christopher Nicklin , Daniel Bailey , Stuart McLean , Young Ae Kim , Hyeonah Kang , Joseph P. Vitta

{"title":"Applying lexical sophistication models to wordlist development: A proof-of-concept study","authors":"Christopher Nicklin , Daniel Bailey , Stuart McLean , Young Ae Kim , Hyeonah Kang , Joseph P. Vitta","doi":"10.1016/j.rmal.2024.100175","DOIUrl":"10.1016/j.rmal.2024.100175","url":null,"abstract":"<div><div>Language teaching stakeholders generally rely on frequency-derived wordlists to determine words for pedagogical purposes. However, words that are instinctively easier for many learners, such as “pizza”, occur less frequently in reference corpora than words that might be considered more difficult, such as “physics”. Furthermore, research demonstrates that modeling frequency alongside other lexical sophistication variables predicts word difficulty better than frequency alone. This study constitutes a proof-of-concept; the concept being that a lexical sophistication-based approach to wordlist construction can produce lists that outperform frequency as word difficulty predictors. The method resulted in lexical sophistication-derived difficulty scores for 14,054 of the 20,000 most frequent Corpus of Contemporary American English lemmas. When compared with other commonly used wordlists, these scores successfully addressed the “pizza/physics” problem in that “pizza” was ranked easier than “physics”, and they also displayed larger correlations with word difficulty than other lists across two linguistic domains. More importantly, the scores also performed comparably to a knowledge-based vocabulary list, but contained almost three times as many lemmas for a fraction of the time and financial costs. We envisage that the present study's methodology can be used by researchers and language teaching stakeholders to create bespoke wordlists for a range of contexts.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 1","pages":"Article 100175"},"PeriodicalIF":0.0,"publicationDate":"2024-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143101736","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Introducing fluency measures to the elicited imitation task

Research Methods in Applied Linguistics Pub Date : 2024-12-12 DOI: 10.1016/j.rmal.2024.100176

Hui Sun , Dagmar Divjak , Petar Milin

{"title":"Introducing fluency measures to the elicited imitation task","authors":"Hui Sun , Dagmar Divjak , Petar Milin","doi":"10.1016/j.rmal.2024.100176","DOIUrl":"10.1016/j.rmal.2024.100176","url":null,"abstract":"<div><div>The elicited imitation (EI) task has been widely used as a measure of automatized L2 knowledge. However, the scoring of the task has relied exclusively on product-based measures (i.e., accuracy of L2 production), without considering any process-based indices of automatization, such as fluency. To fill this gap, our study develops a written version of the EI task and innovatively draws on keystroke logging techniques to introduce new measures of fluency in EI production. To test whether the addition of fluency measures improves task sensitivity, we examined the degree to which fluency and accuracy predicted L2 proficiency among 40 L1 Polish speakers of English, living in the UK (Mage = 31, 20–60). The participants were late learners of English at intermediate-to-advanced level (CEFR B1–C2) with varying lengths of residence (0.5–18 years). Their L2 proficiency was measured through self-evaluation according to CEFR scales and through test-evaluation by DIALANG English grammar and vocabulary tests. Their written production of English article and tense-aspect target structures was coded for grammatical accuracy, speed and pausing fluency, and consistency in fluency. Generalized Additive Modelling revealed a nonlinear interaction between speed and pausing fluency and grammatical accuracy as predictors of selfevaluated (but not test-evaluated) proficiency, which suggests that participants tended to be more accurate and fluent from low to average proficiency, after which their accuracy plateaued while fluency continued to improve. The results support the importance of assessing fluency to maintain the sensitivity of the EI task especially among advanced learners.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 1","pages":"Article 100176"},"PeriodicalIF":0.0,"publicationDate":"2024-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143101733","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0