Investigating OpenAI’s ChatGPT Potentials in Generating Chatbot's Dialogue for English as a Foreign Language Learning

Abstract

—Lack of opportunities is a significant hurdle for English as a Foreign Language (EFL) for students during their learning journey. Previous studies have explored the use of chatbots as learning partners to address this issue. However, the success of chatbot implementation depends on the quality of the reference dialogue content, yet research focusing on this subject is still limited. Typically, human experts are involved in creating suitable dialogue materials for students to ensure the quality of such content. Research attempting to utilize artificial intelligence (AI) technologies for generating dialogue practice materials is relatively limited, given the constraints of existing AI systems that may produce incoherent output. This research investigates the potential of leveraging OpenAI's ChatGPT, an AI system known for producing coherent output, to generate reference dialogues for an EFL chatbot system. The study aims to assess the effectiveness of ChatGPT in generating high-quality dialogue materials suitable for EFL students. By employing multiple readability metrics, we analyze the suitability of ChatGPT-generated dialogue materials and determine the target audience that can benefit the most. Our findings indicate that ChatGPT's dialogues are well-suited for students at the Common European Framework of Reference for Languages (CEFR) level A2 (elementary level). These dialogues are easily comprehensible, enabling students at this level to grasp most of the vocabulary used. Furthermore, a substantial portion of the dialogues intended for CEFR B1 (intermediate level) provides ample stimulation for learning new words. The integration of AI-powered chatbots in EFL education shows promise in overcoming limitations and providing valuable learning resources to students.