Investigating the potential of NLP-driven linguistic and acoustic features for predicting human scores of children’s oral language proficiency

IF 2.7 3区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Assessment in Education-Principles Policy & Practice Pub Date : 2021-07-04 DOI:10.1080/0969594X.2021.1999209

Melissa R. Hunte, Samantha McCormick, Maitree Shah, Clarissa Lau, E. Jang

{"title":"Investigating the potential of NLP-driven linguistic and acoustic features for predicting human scores of children’s oral language proficiency","authors":"Melissa R. Hunte, Samantha McCormick, Maitree Shah, Clarissa Lau, E. Jang","doi":"10.1080/0969594X.2021.1999209","DOIUrl":null,"url":null,"abstract":"ABSTRACT Children’s oral language proficiency (OLP) is integral for developing literacy skills. Storytelling or retelling is often used by parents and educators to elicit children’s OLP, yet it is less commonly used for assessment purposes. Leveraged by natural language processing and machine learning, this study examined the extent to which computational linguistic and acoustic indices predict human ratings of children’s (n=184 aged 9 to 11) OLP using two story retell stimuli presented in written and aural forms. Human raters scored children’s OLP on five oral proficiency criteria: vocabulary, grammar, idea development, task-fulfilment, and speech delivery, using a 4-point scale, and linguistic and acoustic features were used to predict each criterion. Results showed the efficacy of automated indices to predict human scores of children’s OLP. This study calls for attention to discrepancies in human and machine speech delivery scores and stimulus effects on story retelling performance among children of different language backgrounds.","PeriodicalId":51515,"journal":{"name":"Assessment in Education-Principles Policy & Practice","volume":"16 1","pages":"477 - 505"},"PeriodicalIF":2.7000,"publicationDate":"2021-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Assessment in Education-Principles Policy & Practice","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1080/0969594X.2021.1999209","RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 1

Abstract

ABSTRACT Children’s oral language proficiency (OLP) is integral for developing literacy skills. Storytelling or retelling is often used by parents and educators to elicit children’s OLP, yet it is less commonly used for assessment purposes. Leveraged by natural language processing and machine learning, this study examined the extent to which computational linguistic and acoustic indices predict human ratings of children’s (n=184 aged 9 to 11) OLP using two story retell stimuli presented in written and aural forms. Human raters scored children’s OLP on five oral proficiency criteria: vocabulary, grammar, idea development, task-fulfilment, and speech delivery, using a 4-point scale, and linguistic and acoustic features were used to predict each criterion. Results showed the efficacy of automated indices to predict human scores of children’s OLP. This study calls for attention to discrepancies in human and machine speech delivery scores and stimulus effects on story retelling performance among children of different language backgrounds.

查看原文本刊更多论文

研究nlp驱动的语言和声学特征在预测儿童口语能力的人类得分方面的潜力

儿童的口语能力(OLP)是发展读写能力不可或缺的一部分。父母和教育工作者经常使用讲故事或复述的方法来引出孩子的OLP，但很少用于评估目的。利用自然语言处理和机器学习，本研究考察了计算语言和声学指数在多大程度上预测了人类对儿童(n=184名9至11岁)的OLP评分，使用两种以书面和听觉形式呈现的故事复述刺激。人类评分者对儿童的OLP在五个口语熟练度标准上进行评分:词汇量、语法、想法发展、任务完成和演讲，使用4分制，并使用语言和声学特征来预测每个标准。结果显示了自动指数预测儿童OLP得分的有效性。本研究呼吁关注不同语言背景的儿童在人类和机器语音传递得分的差异和刺激效应对故事复述的影响。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Assessment in Education-Principles Policy & Practice EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

5.70

自引率

3.10%

发文量

期刊介绍： Recent decades have witnessed significant developments in the field of educational assessment. New approaches to the assessment of student achievement have been complemented by the increasing prominence of educational assessment as a policy issue. In particular, there has been a growth of interest in modes of assessment that promote, as well as measure, standards and quality. These have profound implications for individual learners, institutions and the educational system itself. Assessment in Education provides a focus for scholarly output in the field of assessment. The journal is explicitly international in focus and encourages contributions from a wide range of assessment systems and cultures. The journal''s intention is to explore both commonalities and differences in policy and practice.