Multi-objective math problem generation using large language model through an adaptive multi-level retrieval augmentation framework

Abstract

Math problems are an important knowledge carrier and evaluation means in personalized teaching. Their high cost of manual compilation promotes the research of math problem generation. Many previous studies have focused on the generation of math word problems, which are difficult to meet the real teaching needs due to the single task-objective orientation and small differences in generation results. By fusing external knowledge through retrieval-augmented generation (RAG), large language model (LLM) can generate a variety of math problems, but the generated results still have limitations such as poor knowledge consistency, uncontrollability, and high computational cost. In this paper, we propose the task of multi-objective math problem generation (MMPG). This task introduces the triple objectives of generation including “question type, knowledge point and difficulty” in respond to teaching needs in real scene. To the best of our knowledge, this is the first study considering multiple objectives on the process of math problem generation. Based on this, we further design an adaptive multi-level retrieval augmentation framework (AMRAF) for LLM to generate multi-objective math problems. This plug-and-play framework can effectively improve the generation performance without parameter tuning of the target model due to the fine-grained information retrieval and fusion. To verify the effectiveness of the proposed framework and provide a benchmark for subsequent research, we construct an MMPG dataset containing 9,000 samples. Experimental results demonstrate the superiority and effectiveness of our framework.