Hai Li, Wanli Xing, Chenglu Li, Wangda Zhu, Hyunju Oh
{"title":"简单的数学故事更好吗？参与验证的ai生成的多模态数学故事的自动可读性评估","authors":"Hai Li, Wanli Xing, Chenglu Li, Wangda Zhu, Hyunju Oh","doi":"10.1111/bjet.13554","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <p>Mathematical stories can enhance students' motivation and interest in learning mathematics, thereby positively impacting their academic performance. However, due to resource constraints faced by the creators, generative artificial intelligence (GAI) is employed to create mathematical stories accompanied by images. This study introduces a method for automatically assessing the quality of these multimodal stories by evaluating text-image coherence and textual readability. Using GAI-generated stories for grades 3 to 5 from the US math story learning platform <i>Read Solve Create</i> (RSC), we extracted features related to multimodal semantics and text readability. We then analysed the correlation between these features and student engagement levels, measured by average reading time per story (behavioural engagement) and average drawing tool usage per story (cognitive engagement), derived from browsing logs and interaction metrics on the platform. Our findings reveal that textual features such as conjunctive adverbs, sentence connectors, causal connectives and simplified vocabulary positively correlate with behavioural engagement. Additionally, higher semantic similarity between text and images, as well as the number of operators in the stories, is associated with increased cognitive engagement. This study advances the application of GAI in mathematics education and offers novel insights for instructional material design.</p>\n </section>\n \n <section>\n \n <div>\n \n <div>\n \n <h3>Practitioner notes</h3>\n <p>What is already known about this topic\n </p><ul>\n \n <li>Mathematical stories can enhance students' motivation and interest in mathematics, leading to improved academic performance.</li>\n \n <li>Generative artificial intelligence (GAI) has been increasingly employed to create multimodal educational content, including mathematical stories with accompanying images, to address content creators' resource constraints.</li>\n \n <li>Prior readability research has primarily focused on the analysis of text-based educational content, with less emphasis on the integration and analysis of visual elements.</li>\n </ul>\n \n <p>What this paper adds\n </p><ul>\n \n <li>Introduces a novel automated multimodal readability assessment method that evaluates the coherence between text and images and the readability of text in GAI-generated mathematical stories.</li>\n \n <li>Identifies specific story features, such as the more frequent use of three types of conjunctions (adversative conjunctions, common sentence conjunctions and logical conjunctions) and vocabulary simplicity that correlate with student engagement.</li>\n </ul>\n \n <p>Implications for practice and/or policy\n </p><ul>\n \n <li>Educators and curriculum developers are encouraged to utilise automated multimodal readability assessment tools to analyse and refine GAI-generated educational content, aiming to enhance student engagement and learning experience.</li>\n \n <li>Suggestions for the design of educational content includes the consideration of identified readability features that correlate with higher engagement. Caution should be exercised in handling the association between images and text considering the cognitive load of the instructional materials.</li>\n </ul>\n \n </div>\n </div>\n </section>\n </div>","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":"56 3","pages":"1092-1117"},"PeriodicalIF":6.7000,"publicationDate":"2024-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Are simpler math stories better? Automatic readability assessment of GAI-generated multimodal mathematical stories validated by engagement\",\"authors\":\"Hai Li, Wanli Xing, Chenglu Li, Wangda Zhu, Hyunju Oh\",\"doi\":\"10.1111/bjet.13554\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div>\\n \\n \\n <section>\\n \\n <p>Mathematical stories can enhance students' motivation and interest in learning mathematics, thereby positively impacting their academic performance. However, due to resource constraints faced by the creators, generative artificial intelligence (GAI) is employed to create mathematical stories accompanied by images. This study introduces a method for automatically assessing the quality of these multimodal stories by evaluating text-image coherence and textual readability. Using GAI-generated stories for grades 3 to 5 from the US math story learning platform <i>Read Solve Create</i> (RSC), we extracted features related to multimodal semantics and text readability. We then analysed the correlation between these features and student engagement levels, measured by average reading time per story (behavioural engagement) and average drawing tool usage per story (cognitive engagement), derived from browsing logs and interaction metrics on the platform. Our findings reveal that textual features such as conjunctive adverbs, sentence connectors, causal connectives and simplified vocabulary positively correlate with behavioural engagement. Additionally, higher semantic similarity between text and images, as well as the number of operators in the stories, is associated with increased cognitive engagement. This study advances the application of GAI in mathematics education and offers novel insights for instructional material design.</p>\\n </section>\\n \\n <section>\\n \\n <div>\\n \\n <div>\\n \\n <h3>Practitioner notes</h3>\\n <p>What is already known about this topic\\n </p><ul>\\n \\n <li>Mathematical stories can enhance students' motivation and interest in mathematics, leading to improved academic performance.</li>\\n \\n <li>Generative artificial intelligence (GAI) has been increasingly employed to create multimodal educational content, including mathematical stories with accompanying images, to address content creators' resource constraints.</li>\\n \\n <li>Prior readability research has primarily focused on the analysis of text-based educational content, with less emphasis on the integration and analysis of visual elements.</li>\\n </ul>\\n \\n <p>What this paper adds\\n </p><ul>\\n \\n <li>Introduces a novel automated multimodal readability assessment method that evaluates the coherence between text and images and the readability of text in GAI-generated mathematical stories.</li>\\n \\n <li>Identifies specific story features, such as the more frequent use of three types of conjunctions (adversative conjunctions, common sentence conjunctions and logical conjunctions) and vocabulary simplicity that correlate with student engagement.</li>\\n </ul>\\n \\n <p>Implications for practice and/or policy\\n </p><ul>\\n \\n <li>Educators and curriculum developers are encouraged to utilise automated multimodal readability assessment tools to analyse and refine GAI-generated educational content, aiming to enhance student engagement and learning experience.</li>\\n \\n <li>Suggestions for the design of educational content includes the consideration of identified readability features that correlate with higher engagement. Caution should be exercised in handling the association between images and text considering the cognitive load of the instructional materials.</li>\\n </ul>\\n \\n </div>\\n </div>\\n </section>\\n </div>\",\"PeriodicalId\":48315,\"journal\":{\"name\":\"British Journal of Educational Technology\",\"volume\":\"56 3\",\"pages\":\"1092-1117\"},\"PeriodicalIF\":6.7000,\"publicationDate\":\"2024-12-13\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"British Journal of Educational Technology\",\"FirstCategoryId\":\"95\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1111/bjet.13554\",\"RegionNum\":1,\"RegionCategory\":\"教育学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"EDUCATION & EDUCATIONAL RESEARCH\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"British Journal of Educational Technology","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/bjet.13554","RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
Are simpler math stories better? Automatic readability assessment of GAI-generated multimodal mathematical stories validated by engagement
Mathematical stories can enhance students' motivation and interest in learning mathematics, thereby positively impacting their academic performance. However, due to resource constraints faced by the creators, generative artificial intelligence (GAI) is employed to create mathematical stories accompanied by images. This study introduces a method for automatically assessing the quality of these multimodal stories by evaluating text-image coherence and textual readability. Using GAI-generated stories for grades 3 to 5 from the US math story learning platform Read Solve Create (RSC), we extracted features related to multimodal semantics and text readability. We then analysed the correlation between these features and student engagement levels, measured by average reading time per story (behavioural engagement) and average drawing tool usage per story (cognitive engagement), derived from browsing logs and interaction metrics on the platform. Our findings reveal that textual features such as conjunctive adverbs, sentence connectors, causal connectives and simplified vocabulary positively correlate with behavioural engagement. Additionally, higher semantic similarity between text and images, as well as the number of operators in the stories, is associated with increased cognitive engagement. This study advances the application of GAI in mathematics education and offers novel insights for instructional material design.
Practitioner notes
What is already known about this topic
Mathematical stories can enhance students' motivation and interest in mathematics, leading to improved academic performance.
Generative artificial intelligence (GAI) has been increasingly employed to create multimodal educational content, including mathematical stories with accompanying images, to address content creators' resource constraints.
Prior readability research has primarily focused on the analysis of text-based educational content, with less emphasis on the integration and analysis of visual elements.
What this paper adds
Introduces a novel automated multimodal readability assessment method that evaluates the coherence between text and images and the readability of text in GAI-generated mathematical stories.
Identifies specific story features, such as the more frequent use of three types of conjunctions (adversative conjunctions, common sentence conjunctions and logical conjunctions) and vocabulary simplicity that correlate with student engagement.
Implications for practice and/or policy
Educators and curriculum developers are encouraged to utilise automated multimodal readability assessment tools to analyse and refine GAI-generated educational content, aiming to enhance student engagement and learning experience.
Suggestions for the design of educational content includes the consideration of identified readability features that correlate with higher engagement. Caution should be exercised in handling the association between images and text considering the cognitive load of the instructional materials.
期刊介绍：
BJET is a primary source for academics and professionals in the fields of digital educational and training technology throughout the world. The Journal is published by Wiley on behalf of The British Educational Research Association (BERA). It publishes theoretical perspectives, methodological developments and high quality empirical research that demonstrate whether and how applications of instructional/educational technology systems, networks, tools and resources lead to improvements in formal and non-formal education at all levels, from early years through to higher, technical and vocational education, professional development and corporate training.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
