SocioMathLLM: A Multimodal Large Language Model Framework for Generating Authentic Mathematical Word Problems Driving Socialization

Abstract

Large language models (LLMs) have demonstrated significant advancements in automatic question generation (AQG), contributing to enhanced learning and teaching efficiency by facilitating the creation of human-like educational content and assessment materials. However, in mathematics education, where generating extensive sets of math word problems (MWPs) requires significant time and effort, the use of LLMs to design tasks that foster socialization competencies such as group collaboration, cooperation, negotiation, and ideas-sharing among students remains unexplored. This study makes the first attempt to develop a specialized multimodal LLM framework, SocioMathLLM, which integrates an innovative prompt engineering strategy to incorporate multimodal authentic contextual information (text and image), group background information, and socialization criteria (i.e., group size, task nature, social goal, social frequency, social-quality, interaction quantity, and social interdependency) to generate MWPs aimed at fostering group socialization. SocioMathLLM was evaluated through automatic metrics, human evaluation, and LLM-based evaluation. The automatic evaluation result demonstrates good performance in generating MWPs that promote socialization with an average precision of 0.906, recall of 0.943, and F1 score of 0.925. A total of 100 evaluation cases by humans and LLM also validated the robustness of SocioMathLLM, highlighting its potential for application in collaborative mathematics learning and assessment contexts.