Spoken English fluency scoring using convolutional neural networks

In this paper, we propose a spoken English fluency scoring using Convolutional Neural Network (CNN) to learn feature extraction and scoring model jointly from raw time-domain signal input. In general, automatic spoken English fluency scoring is composed feature extraction and a scoring model. Feature extraction is used to compute the feature vectors that are assumed to represent spoken English fluency, and the scoring model predicts the fluency score of an input feature vector. Although the conventional approach works well, there are issues regarding feature extraction and model parameter optimization. First, because the fluency features are computed based on human knowledge, some crucial representations that are included in a raw data corpus can be missed. Second, each parameter of the model is optimized separately, which can lead to suboptimal performance. To address these issues, we propose a CNN-based approach to extract fluency features directly from a raw data corpus without hand-crafted engineering and optimizes all model parameters jointly. The effectiveness of the proposed approach is evaluated using Korean-Spoken English Corpus.