Towards Updated Standards for Dictionary Examples for English Learners: An Evaluation Framework for ChatGPT-Generated Illustrative Sentences

Abstract

Abstract Developments in GenAI systems like ChatGPT have opened up new possibilities for supporting language education, including the automated generation of illustrative example sentences for learners. One promising application is the generation of dictionary-style examples, but it remains unclear to what extent ChatGPT-generated English sentences can be considered good dictionary examples, or how such illustrative examples can be effectively evaluated. This study presents an improved evaluation framework and a corresponding prototype, extending established criteria for good dictionary examples (Kilgarriff et al., 2008). Designed to assess the linguistic and pedagogical quality of illustrative example sentences, the framework enables systematic comparison between ChatGPT outputs generated via zero-shot prompting and human-curated examples from three authoritative English learner’s dictionaries (ELDs). The proposed framework is multi-dimensional, incorporating key parameters such as completeness, salience, intelligibility, informativeness, typicality, diversity, complexity, and sensitivity. These dimensions align with both lexicographic standards and pedagogical priorities, offering an innovative model for evaluating example sentence quality in language education. Findings reveal that while ELD examples generally demonstrate high standards, they do not consistently represent optimal performance across all dimensions. ChatGPT-generated examples, when assessed using the framework, show promising potential for scalable, quality-controlled use of dictionary example sentences. The prototype operationalises these insights, supporting principled selection and refinement of GenAI outputs for learner-facing applications.