Research Methods in Applied Linguistics最新文献_第2页

Examining refusal and acceptance sequences employing a data-driven binary rating approach 采用数据驱动的二元评级法研究拒绝和接受顺序

Research Methods in Applied Linguistics Pub Date : 2024-11-05 DOI: 10.1016/j.rmal.2024.100161

Yunwen Su , Xi Chen

{"title":"Examining refusal and acceptance sequences employing a data-driven binary rating approach","authors":"Yunwen Su , Xi Chen","doi":"10.1016/j.rmal.2024.100161","DOIUrl":"10.1016/j.rmal.2024.100161","url":null,"abstract":"<div><div>This paper presents the development and preliminary validation of a data-driven analytic binary rating rubric for assessing roleplay-elicited pragmatic performance of learners. The roleplay task consists of two items—one lunch invitation to elicit a refusal and one gift offer to elicit an acceptance in Chinese, both of which involve initiating proposals to return an earlier favor. Responses to return-to-favor invitations or offers in Chinese are challenging for L2 learners as they involve complicated discursive features. To create a practical, reliable tool for evaluating L2 performance, the researchers conducted a discourse analysis of L1 Chinese performance data (<em>N</em> = 22) to identify key discursive features in two domains— Intention (i.e., refusing the return-for-favor intention) and Decision (i.e., refusing/accepting the offer/invitation). A roleplay can receive up to 4 points in both domains, including one point each for orientation/directness, position, modification, and justification, for 8 total possible points. Scores assigned to both L1 and L2 performance according to the rubric reflected high inter-rater reliability. The effect of proficiency was significant on learner scores and interacted with rating domains and scenarios. Lower-level learners scored lower with addressing the interlocutor's return-for-favor intention than with the invitation or offer itself; they also scored lower with refusing than accepting an offer. By focusing on key discursive features rather than exhaustive interactional details, the rubric streamlines the evaluation process while maintaining accuracy and reliability. Future studies may extend the use of the rubric to other speech act sequences and languages and apply it to automated grading systems.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100161"},"PeriodicalIF":0.0,"publicationDate":"2024-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142593789","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Profiling learner development with a computerized dynamic assessment of a Japanese learner's L2 email writing 利用计算机动态评估日语学习者的第二语言电子邮件写作，剖析学习者的发展情况

Research Methods in Applied Linguistics Pub Date : 2024-10-30 DOI: 10.1016/j.rmal.2024.100164

Allan Nicholas, John Blake

{"title":"Profiling learner development with a computerized dynamic assessment of a Japanese learner's L2 email writing","authors":"Allan Nicholas, John Blake","doi":"10.1016/j.rmal.2024.100164","DOIUrl":"10.1016/j.rmal.2024.100164","url":null,"abstract":"<div><div>This study evaluates a computerized dynamic assessment (C-DA) method's potential to build a detailed pragmatics-focused developmental profile of a Japanese L2 English learner's L2 email writing. L2 email writing studies typically separate learning and assessment, using holistic scales to evaluate the pragmatic elements of learner texts. C-DA, grounded in sociocultural theory, unifies learning and assessment, administering email tasks with varying social contexts and providing immediate mediation within the learner's zone of proximal development (ZPD). Further, a diagnostic capacity allows for identification of specific pragmatics-related infelicities. The C-DA employs a dual-layered framework: an identification layer uses a coding scheme to automatically identify specific instances of perceived pragmatic inappropriateness, while the mediation layer provides ZPD-sensitive mediation. By evaluating the frequency and explicitness of mediation engaged in, in combination with qualitative examination of elicited email texts, the C-DA enables insights into learner development within the ZPD. We focus on an individual learner's interaction with the C-DA, evaluating the program's effectiveness in enabling a detailed learner diagnostic and developmental profile.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100164"},"PeriodicalIF":0.0,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142540289","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Incidental vocabulary learning: A scientometric review 偶然词汇学习：科学计量学回顾

Research Methods in Applied Linguistics Pub Date : 2024-10-22 DOI: 10.1016/j.rmal.2024.100160

Sofiya Shahiwala , D. R. Rahul , John R. Baker

{"title":"Incidental vocabulary learning: A scientometric review","authors":"Sofiya Shahiwala , D. R. Rahul , John R. Baker","doi":"10.1016/j.rmal.2024.100160","DOIUrl":"10.1016/j.rmal.2024.100160","url":null,"abstract":"<div><div>Research on incidental vocabulary learning has gained increasing attention in language acquisition. With growing empirical evidence, traditional reviews have attempted to provide incidental vocabulary learning narrative and summary effects. However, a detailed analysis of publication trends and their dynamics is necessary to comprehensively understand the field's developments. Consequently, a scientometric review was conducted using 547 journal articles published between 2003 and 2023 from the Web of Science and Scopus databases. The analysis explored developments through three lenses: performance, document co-citation, and structural variation. Through performance analysis, prominent journals, affiliate universities, and authors who increasingly contributed to the field were identified. The document co-citation analysis revealed several major clusters, of which the top six were scrutinised. Findings highlighted recurrent themes such as the importance of topic familiarity, glosses, and input modality. Moreover, influential articles and their linkages were evaluated through structural variation analysis, and their key findings and citation bursts revealed frontier trends. Finally, limitations in empirical data were discussed to elicit future research directions. As an early attempt at a scientometric analysis of incidental vocabulary acquisition, these findings will be highly beneficial to language researchers, teachers, and teacher educators, not only for identifying potential research areas but also for adapting strategies for vocabulary instruction.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100160"},"PeriodicalIF":0.0,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142529689","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Ordinal response scales: Psychometric grounding for design and analysis 序数反应量表：设计和分析的心理基础

Research Methods in Applied Linguistics Pub Date : 2024-10-21 DOI: 10.1016/j.rmal.2024.100156

Lukas Sönning

{"title":"Ordinal response scales: Psychometric grounding for design and analysis","authors":"Lukas Sönning","doi":"10.1016/j.rmal.2024.100156","DOIUrl":"10.1016/j.rmal.2024.100156","url":null,"abstract":"<div><div>Ordinal response scales are commonly used in applied linguistics. To summarize the distribution of ratings or judgments provided by informants, these are usually converted into numbers and then averaged or analyzed with ordinary regression models. This approach has been criticized in the literature; one caveat (among others) is the assumption that distances between categories are known. The present paper illustrates how empirical insights into the perception of response labels may inform the design and analysis stage of a study. We start with a review of how ordinal scales are used in linguistic research. Our survey offers insights into typical scale layouts and analysis strategies, and it allows us to identify three commonly used rating dimensions (agreement, intensity, and frequency). We take stock of the experimental literature on the perception of relevant scale point labels and then demonstrate how psychometric insights may direct scale design and data analysis. This includes a careful consideration of measurement-theoretic and statistical issues surrounding the numeric-conversion approach to ordinal data. We focus on the consequences of these drawbacks for the interpretation of empirical findings, which will enable researchers to make informed decisions and avoid drawing false conclusions from their data. We present a case study on <em>yous(e)</em> in two varieties of English, which shows that reliance on psychometric scale values can alter statistical conclusions, while also giving due consideration to the key limitations of the numeric-conversion approach to ordinal data analysis.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100156"},"PeriodicalIF":0.0,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142529688","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Developing and using an adapted mosaic approach to explore children's foreign language learning experiences in primary school 开发和使用经调整的马赛克方法，探索小学儿童的外语学习经验

Research Methods in Applied Linguistics Pub Date : 2024-10-17 DOI: 10.1016/j.rmal.2024.100155

Victoria Schulze

{"title":"Developing and using an adapted mosaic approach to explore children's foreign language learning experiences in primary school","authors":"Victoria Schulze","doi":"10.1016/j.rmal.2024.100155","DOIUrl":"10.1016/j.rmal.2024.100155","url":null,"abstract":"<div><div>This article describes an adapted mosaic approach, developed and used in a longitudinal, ethnographic study that explored children's foreign language learning (FLL) experiences in primary school in England. As a methodological innovation within the field of instructed early foreign language learning, the adapted approach addresses an identified need for a wider range of research methodologies and more inductive approaches. The original mosaic approach was initially considered to help address methodological questions that arose during the ethnography. That approach was subsequently adapted and developed in four main ways: firstly, for use within the longitudinal, ethnographic study; second, for use with its older, primary school-aged children; third, for development and use <em>together with</em> children in school rather than a tool designed for use <em>on</em> children and finally, where the focus was upon the <em>process</em> of completion rather than the end product. The adapted approach met the needs identified within the ethnography by further enhancing children's voices, their agency and perspectives within its data collection and analysis. Recommendations for its use in language learning contexts are made to support further research together with children, to help strengthen what is currently known and understood about children's experiences of instructed foreign language learning. This is based on the premise that knowing and understanding more about these will help inform future practices and initiatives.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100155"},"PeriodicalIF":0.0,"publicationDate":"2024-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142445797","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Research methods for investigating young learners’ engagement with foreign languages outside of school 调查青少年学生在校外学习外语情况的研究方法

Research Methods in Applied Linguistics Pub Date : 2024-10-13 DOI: 10.1016/j.rmal.2024.100159

Vanessa De Wilde

引用次数: 0

Network science in a dynamic usage-based approach 基于动态使用方法的网络科学

Research Methods in Applied Linguistics Pub Date : 2024-09-30 DOI: 10.1016/j.rmal.2024.100150

Susanne DeVore , Marjolijn Verspoor

{"title":"Network science in a dynamic usage-based approach","authors":"Susanne DeVore , Marjolijn Verspoor","doi":"10.1016/j.rmal.2024.100150","DOIUrl":"10.1016/j.rmal.2024.100150","url":null,"abstract":"<div><div>In this study, we test the ability of network science to capture linguistic development over time in individuals from a dynamic usage-based perspective, a combination of a complex dynamic systems theory (CDST) approach and usage-based (UB) linguistics. Network science is designed to quantitatively analyze entire systems and captures complex interrelationships between components of those systems. We select network science measures that have potential to represent theoretically predicted individual language learning processes, specifically focusing on the development from prototype to schematized constructions at the macro- (network), micro- (word), and meso‑level (syntactic structure). To test this approach, we traced the beginning L2 Chinese development of two English speakers. The results are generally aligned to dynamic usage-based predictions in terms of prototypicality, schematization variability, and variation. Additionally, the network science approach allows us to identify a core set of words that emerges at the early stage of learning and is highly connected, seeming to drive development; it also suggests that variability plays different roles at different levels of analysis. Although it is preliminary to make strong conclusions about this set of words, it suggests that, as with other areas of inquiry, network science can reveal previously unidentified information about linguistic development and should be further explored.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100150"},"PeriodicalIF":0.0,"publicationDate":"2024-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142358790","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A methodological exploration of the P-frame approach: Implications for developing phrase lists 对 P 框架方法的方法论探索：对编制短语表的影响

Research Methods in Applied Linguistics Pub Date : 2024-09-19 DOI: 10.1016/j.rmal.2024.100154

Aysel Şahin Kızıl , Benet Vincent , Lee McCallum

{"title":"A methodological exploration of the P-frame approach: Implications for developing phrase lists","authors":"Aysel Şahin Kızıl , Benet Vincent , Lee McCallum","doi":"10.1016/j.rmal.2024.100154","DOIUrl":"10.1016/j.rmal.2024.100154","url":null,"abstract":"<div><p>The widespread availability of corpora and corpus tools has led to considerable advances in our understanding of phraseology and how it can be investigated. An increasingly influential approach to investigating phraseology is the retrieval and analysis of phrase frames (p-frames), recurrent word sequences with a variable slot. Although the p-frame approach is quite well-established, pedagogically oriented research in this area is still quite new, in particular the application of phrase frames to generate useful lists of phrases. This paper aims to contribute to methodological efforts in this direction by presenting a case study to illustrate decision-making throughout the process from corpus compilation to the generation of a final pedagogical list of phrases. Using a corpus of research article (RA) introductions in Health Sciences, this study shows how key decisions including p-frame length, frequency and range thresholds were arrived at. It also discusses exclusion criteria focusing on variability and predictability of p-frame fillers, and the fine-tuning of the resulting list, focusing on semantic coherence and pedagogical usefulness. Finally, we present the results of an initial evaluation of the list by stakeholders. The main contributions of the study to p-frame research methodology lie in the clarification of threshold settings, the potential contribution of variability and predictability to decision-making and the proposal of more in-depth concordance analysis of co-text to aid in the identification of phrases.</p></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100154"},"PeriodicalIF":0.0,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142270721","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Building custom NLP tools to annotate discourse-functional features for second language writing research: A tutorial 为第二语言写作研究建立自定义 NLP 工具以标注话语功能特征：教程

Research Methods in Applied Linguistics Pub Date : 2024-09-18 DOI: 10.1016/j.rmal.2024.100153

Masaki Eguchi , Kristopher Kyle

引用次数: 0

Moving beyond binary language status in research: Investigating early foreign language learning and linguistic distance 在研究中超越二元语言地位：调查早期外语学习和语言距离

Research Methods in Applied Linguistics Pub Date : 2024-09-09 DOI: 10.1016/j.rmal.2024.100151

Nils Jaekel , Michael Schurig , Eliane Lorenz

{"title":"Moving beyond binary language status in research: Investigating early foreign language learning and linguistic distance","authors":"Nils Jaekel , Michael Schurig , Eliane Lorenz","doi":"10.1016/j.rmal.2024.100151","DOIUrl":"10.1016/j.rmal.2024.100151","url":null,"abstract":"<div><p>Globalization and migration continue to shape our societies, including educational contexts such as school classrooms. In response to young learners’ linguistic needs, particularly in the context of foreign language learning in Germany, educational approaches need to be adapted to meet the needs of multilingual students. Current, binary approaches accounting for diverse linguistic backgrounds of students in research assume a high degree of homogeneity among multilingual students. Linguistic distance measures may provide alternative, more fine-grained, continuous tools to account for linguistic diversity. This study employs lexical linguistic distance to account for young language learners’ linguistic diversity in a reanalysis of Jaekel et al. (2017). Additionally, mixed-effects modeling was employed to factor in within-class effects for within-class factors versus structural equation modeling, which was previously used. The results outline that linguistic distance provides additional information beyond binary language status. Mixed effects modeling renders comparable results with the same tendencies, but yields more nuanced perspectives on the data.</p></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100151"},"PeriodicalIF":0.0,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2772766124000570/pdfft?md5=9492363d23de861a48c0f3725af5df5f&pid=1-s2.0-S2772766124000570-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142162284","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0