Rating scales in L2 speech act research: A systematic review

Abstract

This paper presents a systematic review of rating scales used in speech act studies within interlanguage pragmatics. It synthesizes the key characteristics of these scales (e.g., formats, rating criteria, theoretical foundations, and development methods) and sources of variation that influence their functionality. Primary studies were identified through keyword searches in major databases, including Google Scholar, Scopus, and Web of Science, using criteria aligned with the study's objectives. The studies in the sample examined the speech act production of second language (L2) learners of English and Asian languages (e.g., Chinese, Japanese, Korean, and Arabic), primarily elicited through written and oral Discourse Completion Tasks (DCTs) and role-plays. Earlier studies employed numerical and defined scales using a priori approaches informed by existing theories and empirical evidence to reflect advancements in the conceptualization of pragmatic competence. More recent research, however, has adopted a data-driven approach, developing and validating scales based on empirical data before implementation in further investigations. Variation in the functionality of rating scales has been examined in terms of internal sources (e.g., band levels and operational types) and external sources (e.g., rater characteristics and rater training). The findings highlight the critical role of rating scales in enhancing the reliability and validity of research methods and the generalizability of findings, thereby strengthening research rigor. This review offers valuable insights for methodological development, research training, and future research directions in L2 pragmatics.