Research on A Machine Scoring Method of Role-Play Section in English Oral Test

Abstract

Computer-assisted instruction has been widely implemented in language learning since the remarkable development of speech technology and natural language processing. In English teaching, the traditional manual evaluation mode has been replaced by the computer. In this paper, we introduce a machine scoring method of Role-Play Section in the English oral test. Specifically, we design a scoring method of Role-Play section in Computer-based English Listening and Speaking Test (CELST) of the Guangdong college entrance examination (GDCEE), which can simulate human scoring. The role-play section is also called Three Questions (TQ) and Five Answers (FA), so two scoring modules are established, respectively. According to the features of the role-play section and the given manual scoring scale, the scoring task can be regarded as a short text similarity evaluation task, and we propose a corresponding multi-index text similarity evaluation method. Statistic-based word matching and keyword-focused semantic similarity evaluations are adopted in the TQ and FA scoring modules. Additionally, we consider the grammar factor and utilize semantic similarity-combined dependency parsing evaluation for TQ scoring. Based on the above five evaluative indicators, we linearly integrate them and corresponding expertise-based weights with linear regression optimization, thus constructing a comprehensive scoring model of the role-play Section in CELST. Experiments were conducted using real oral data of examinees in GDCEE. Results have indicated that the machine scoring method achieves impressive data consistency with human scoring.