Assessing Korean writing ability through a scenario-based assessment approach

Advocating for measuring a broadened construct of writing ability, scenario-based assessment (SBA) has recently been used to assess writing ability. While SBA for writing has been explored in ESL contexts (e.g., Banerjee, 2019; Purpura, 2021), it has been rarely used or examined in the context of languages other than English, especially those that are typologically distant from English, such as Korean. This study explores the feasibility of designing a scenario-based Korean writing assessment (K-SBA). A pilot study was conducted with 51 participants from a Korean as a foreign language program at a US university. Through a goal-oriented scenario of a study abroad program in South Korea, examinees were presented with a collaborative problem-solving task where they were expected to learn about two potential class trip destinations and write a summary of the pros and cons of each destination based on what they learned. The results indicated that the K-SBA was psychometrically sound, providing adequate evidence of reliability. It elicited construct-relevant performances reflecting features unique to the Korean language, such as sociolinguistic competence through the use of honorifics or rhetorical control through a range of cohesive devices. Additionally, variations in examinee performances were observed across the different course levels.