Bernadette O'Rourke , Qunshan Zhao , Alejandro Dayán-Fernández , Euan Dickson , Ruiyu Wang , Mingkang Wang , Lei Zhang , Muhammad Imran
{"title":"Integrated geospatial methods for multilingual cities: Combining GPS, electronically activated recorder, and map-based ethnographic interviews","authors":"Bernadette O'Rourke , Qunshan Zhao , Alejandro Dayán-Fernández , Euan Dickson , Ruiyu Wang , Mingkang Wang , Lei Zhang , Muhammad Imran","doi":"10.1016/j.rmal.2024.100147","DOIUrl":"10.1016/j.rmal.2024.100147","url":null,"abstract":"<div><p>Multilingual speakers’ linguistic practices are typically investigated via self-report measures, often relying on subjective memory recall. This article examines the potential of an interdisciplinary methodology which would allow us to examine the experiences of multilingual speakers as they are played out in real time and space through a combination of GPS monitoring, the use of the Electronically Activated Recorder (EAR), and map-based ethnographic interviews. GPS and EAR offer promising opportunities to investigate the spatial dynamics of multilingual practices as well as the acquisition of a new language and its subsequent use. Despite frequent applications in adjacent fields such as Human Geography, Urban Analytics, Sociology and Psychology, the use of these methods in Applied Linguistics is still relatively scarce. In this article, we examine the potential benefits and limitations of using automated processes for observing human language behaviour, such as those provided by GPS and EAR technologies. We address timely ethical questions around tracked and recorded data collection processes and present solutions used to devise privacy-preserving tools when using GPS and EAR technologies in a pilot study of Gaelic multilingual speakers in the city of Glasgow. By optimising the potential of these technologies when combined with qualitative and ethnographic approaches, our article proposes a multidisciplinary methodological approach that can provide a more nuanced understanding of the spatial dynamics of multilingual speakers in cities.</p></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100147"},"PeriodicalIF":0.0,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2772766124000533/pdfft?md5=d6f9b6b88d828f37d0366064b24e25d0&pid=1-s2.0-S2772766124000533-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142150212","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"More accurate estimation for nonrandom sampling surveys: A post hoc correction method","authors":"Takunori Terasawa","doi":"10.1016/j.rmal.2024.100152","DOIUrl":"10.1016/j.rmal.2024.100152","url":null,"abstract":"<div><p>Nonprobability sample surveys are commonly used in applied linguistics research, but their lack of representativeness makes it difficult to generalize their results to the wider population. To address this issue, this study introduces a post hoc statistical correction method that uses an existing probability sample survey as reference data. The paper begins by outlining the importance of such corrections and provides details of a practical correction procedure that employs propensity scores as a balancing factor. The method is then applied to real-world data from the 2022 Survey of Workers’ English Use, a nonprobability web panel survey investigating English use among Japanese people as an international language. The corrected estimates are subsequently compared to those from a nationally representative survey. The results showed that the proposed correction method effectively adjusted the proportions and mean values of self-reported English proficiency and replicated the population. However, the method was less effective in adjusting the mean values for attitudes toward globalization as well as correlation coefficients, although it did not worsen these estimates. Based on these findings, the paper advocates for the adoption of this correction method in applied linguistic studies where feasible, with some critical considerations and caveats highlighted.</p></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100152"},"PeriodicalIF":0.0,"publicationDate":"2024-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142150211","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Assessing interactional competence through group discussion: A mixed methods validation","authors":"John Syquia , Paul Leeming","doi":"10.1016/j.rmal.2024.100144","DOIUrl":"10.1016/j.rmal.2024.100144","url":null,"abstract":"<div><p>The purpose of this mixed methods study was to assess the validity and functionality of an analytic rating scale for the assessment of interactional competence (IC). The participants were 79 low- to high-proficiency Japanese university students who completed 10-minute small-group discussions. Video recordings of the discussions were assessed by raters using the rating scale. The rater scores were then analyzed using many-facet Rasch measurement (MFRM) which indicated a very good fit to the model. The data were subsequently analyzed using generalizability theory in the form of a G-study and d-study. Those studies showed that the rating scale could be used with fewer raters, therefore increasing practicality without a substantial decrease in reliability. In addition to quantitative data, qualitative data were also collected in the form of interviews with raters and comments they made during assessment. Several raters noted unexpected participant behaviors which were difficult to evaluate using the rating scale, as well as ambiguous language in some category descriptors. The qualitative data provided an invaluable supplement to the quantitative analyses which did not indicate major issues with the rubric. Both forms of data were used to revise the original rating scale and those changes are discussed. This study adds to the limited but growing number of mixed methods studies on IC assessment.</p></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100144"},"PeriodicalIF":0.0,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142128901","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Random-item Rasch models and explanatory extensions: A worked example using L2 vocabulary test item responses","authors":"Karen J. Dunn","doi":"10.1016/j.rmal.2024.100143","DOIUrl":"10.1016/j.rmal.2024.100143","url":null,"abstract":"<div><p>This paper describes the application and relevance of random-item Rasch models in second language (L2) vocabulary research and testing scenarios, aiming to increase understanding of this statistical method amongst researchers and academics working in the L2 assessment field and more broadly in applied linguistics. A step-by-step description of the links between Generalized Linear Mixed Models (GLMMs) and Rasch models is given. It is then demonstrated how random-item-random-person (RPRI) Rasch models (De Boeck, 2008) can be built within a GLMM framework, and the modelling of an explanatory extension is presented in which the role of word and item characteristics are tested as fixed effect covariates in explaining item difficulty in an L2 vocabulary test completed by Hungarian school-age learners of English.</p></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100143"},"PeriodicalIF":0.0,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2772766124000491/pdfft?md5=fa19df9bec4ef16985819dbdeedaefe0&pid=1-s2.0-S2772766124000491-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142096397","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Analyzing metadiscourse in L2 writing for academic purposes: Models and approaches","authors":"Shuyi Amelia Sun, Feng (Kevin) Jiang","doi":"10.1016/j.rmal.2024.100149","DOIUrl":"10.1016/j.rmal.2024.100149","url":null,"abstract":"<div><p>Successful texts rely on writers’ ability to “control the level of personality in their texts, claiming solidarity with readers, evaluating their material, and acknowledging alternative views” (Hyland, 2004b, p. 133). The rhetorical ability to achieve this social interaction is now acknowledged to be both a key feature of academic writing and an important aspect of advanced academic literacy. Over the past three decades, metadiscourse has emerged as a robust model that has garnered increasing attention in applied linguistics, but prevailing views suggest different models and approaches open to us (Ädel & Mauranen, 2010). The openness warrants a review of these methodological issues. Therefore, this study explicates integrative and non-integrative models, delineates thin and thick methodological approaches, and demonstrates how these perspectives are operationalised through two tutorials. Our discussion concludes by highlighting potential avenues of the above-mentioned frameworks and methods in the future research of second language writing and academic discourse.</p></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100149"},"PeriodicalIF":0.0,"publicationDate":"2024-08-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142050169","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hildegunn Dirdal , Stine H. Johansen , Philip Durrant
{"title":"Representativeness and metadata presentation in learner/child corpora: Lessons from the GiG and TRAWL corpora","authors":"Hildegunn Dirdal , Stine H. Johansen , Philip Durrant","doi":"10.1016/j.rmal.2024.100145","DOIUrl":"10.1016/j.rmal.2024.100145","url":null,"abstract":"<div><p>Representativeness is a key requirement in corpus linguistics, and the evaluation of the representativeness of an existing corpus depends on the provision of metadata. The present paper discusses challenges to both representativeness and metadata presentation based on our experiences in compiling corpora of school writing from young learners. Our discussion lends support to the calls for more transparent documentation and standardization, but also highlights some dangers that need to be kept in mind when attempting to standardize metadata.</p></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100145"},"PeriodicalIF":0.0,"publicationDate":"2024-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S277276612400051X/pdfft?md5=1ed8c130a1342c5680cdfbc31e83db9a&pid=1-s2.0-S277276612400051X-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142050168","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An introduction to CIABATTA: Increasing accessibility to corpus creation for non-experts","authors":"Larissa Goulart , Shelley Staples , Aleksey Novikov , Adriana Picoral , Randi Reppen","doi":"10.1016/j.rmal.2024.100146","DOIUrl":"10.1016/j.rmal.2024.100146","url":null,"abstract":"<div><p>This methodological report introduces CIABATTA (Corpus In A Box: Automated Tools, Tutorials, & Advising), a suite of tools that provides novice corpus compilers with a starting point to develop new corpora. These tools are a distillation of guiding principles and templates for corpus building developed by the Corpus & Repository of Writing (CROW) team during the compilation of the CROW corpus. We start by discussing the best practices and ethical issues that need to be considered when compiling a corpus. We then focus on the steps of corpus data processing, introducing two tools that support corpus building. The first tool, the Corpus Text Processor, is used to convert, encode, and standardize texts in the corpus. The second, the deidentifying tool, replaces identifying information with special annotation tags in corpus files, such as [name] and [place]. Both can be downloaded from the CIABATTA wiki and do not require programming experience. In the final part of this paper, we focus on customizable Python scripts developed for CIABATTA that aid in the processing of metadata. We believe that CIABATTA can aid those without corpus building expertise to build corpora for research purposes, thus expanding the use of corpora to a wider group of researchers.</p></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100146"},"PeriodicalIF":0.0,"publicationDate":"2024-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142050167","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Duoethnography and English for research publication purposes: Promises and challenges","authors":"Pejman Habibie , Richard D. Sawyer","doi":"10.1016/j.rmal.2024.100148","DOIUrl":"10.1016/j.rmal.2024.100148","url":null,"abstract":"<div><p>In this paper, we explain how to conduct and apply duoethnography innovatively, given its lack of prescribed methodological steps. We discuss the theoretical underpinnings of duoethnography, its central tenets, its methodological constraints, and its challenges and limitations. More specifically, we explain how this promising innovative methodology can be adopted in the fast-growing field of English for Research Publication Purposes (ERPP) within Applied Linguistics to examine and investigate rhetorical, socio-political, and contextual aspects of the production and dissemination of knowledge and writing for scholarly publication practices. This paper aims to provide important pedagogical and scholarly implications for researchers in Applied Linguistics in general and ERPP in particular, especially novice scholars and doctoral students that are interested in reflective and reflexive methodologies and orientations.</p></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100148"},"PeriodicalIF":0.0,"publicationDate":"2024-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2772766124000545/pdfft?md5=3ed7fb3725f2de8554ca4cf56d7bb500&pid=1-s2.0-S2772766124000545-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142045029","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dana Gablasova , Luke Harding , Raffaella Bottini , Vaclav Brezina , Haoshan (Sally) Ren , Giovanni Iamartino , Yingyu Li , Tanjun Liu , Laura Poggesi , Kristof Savski , Anuchit Toomaneejinda , Angela Zottola
{"title":"Building a corpus of student academic writing in EMI contexts: Challenges in corpus design and data collection across international higher education settings","authors":"Dana Gablasova , Luke Harding , Raffaella Bottini , Vaclav Brezina , Haoshan (Sally) Ren , Giovanni Iamartino , Yingyu Li , Tanjun Liu , Laura Poggesi , Kristof Savski , Anuchit Toomaneejinda , Angela Zottola","doi":"10.1016/j.rmal.2024.100140","DOIUrl":"10.1016/j.rmal.2024.100140","url":null,"abstract":"<div><p>The article discusses methodological procedures and challenges in a project requiring multi-site, transnational data collection for the construction of a corpus of academic writing in EMI higher education contexts. Drawing on our decision-making experiences as a research team, together with empirical data generated through data collection logs recorded by a network of researchers involved in the project, we reflect on key issues in conducting the project and the solutions we found to address specific challenges. After describing the background to the project and the current status of the corpus, we focus on four broad challenges: (1) selecting partners and managing a multi-site project; (2) defining a working construct of academic writing; (3) categorising data according to disciplinary areas; and (4) managing data collection “on the ground”. Throughout, we provide descriptions of our solutions to the challenges identified, and we conclude with a call for further publication of <em>corpus construction records</em> to provide greater transparency and detail around decisions and judgements made at all stages of a corpus construction project.</p></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100140"},"PeriodicalIF":0.0,"publicationDate":"2024-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2772766124000466/pdfft?md5=22dbae89d7b098a34a5a52ba7dc7818d&pid=1-s2.0-S2772766124000466-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141964364","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Comparative judgement for advancing research in applied linguistics","authors":"Peter Thwaites , Magali Paquot","doi":"10.1016/j.rmal.2024.100142","DOIUrl":"10.1016/j.rmal.2024.100142","url":null,"abstract":"<div><p>Comparative judgement (CJ) is a data collection method in which judges are presented with two items, side-by-side, and asked to decide which is “better”. By compiling the results of many such decisions, a scale can be developed to rank each item from best to worst. Though most commonly used for educational assessment, CJ is fundamentally a method for generating holistic, perceptually grounded measurements of hard-to-define constructs. This capability gives CJ broad potential in the field of applied linguistics, as it can address the need for more accurate measurement and definition of various applied linguistic constructs. In this tutorial, we provide a step-by-step guide on how to set up CJ studies and analyse the resulting data. We also discuss some of the method's strengths and weaknesses, and explore ways in which it might enhance and broaden the methodological toolkit of applied linguistic research.</p></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100142"},"PeriodicalIF":0.0,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141961192","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}