Employing Automatic Speech Recognition for Quantitative Oral Corrective Feedback in Japanese Second or Foreign Language Education

Abstract

In Second or Foreign Language (SFL) education, a number of studies in applied linguistics have addressed a common issue of how teachers can provide effective feedback to correct learner's erroneous utterances during a classroom hour. Oral Corrective Feedback (OCF) is generally time-consuming and labor-intensive work for teachers. The use of ASR (Automatic Speech Recognition) in SFL education has drawn attention from both teachers and learners to increase the learning effect and efficiency. We designed and integrated Quantitative OCF using Google Cloud Speech-to-Text as a part of the oral assessment using an LMS (Learning Management System) for Japanese SFL courses. The level of learners is a starter's level without any prerequisite knowledge of Japanese language. Preliminary experiments using a total of 214 audio datasets by non-native speakers exhibited that 37.4% of the datasets were recognized properly as Japanese sentences. However, as the remainder of the datasets contains erroneous utterances, characteristics of intonation, or noise, ASR successfully detected word-based errors with high accuracy (82.4%) but low precision (28.1%). Oral assessment employing ASR is highly promising as a complementary system for teachers on partially automating the assessment of audio data from learners with evidence and priority orders as well as significantly reducing teachers' scoring workload and time spent on the most problematic part of the students' speech. While our implementation still requires teachers to double-check, such overhead is small and affordable.