Jieun Han, Haneul Yoo, Junho Myung, Minsun Kim, Tak Yeon Lee, So-Yeon Ahn, Alice Oh
{"title":"RECIPE4U: Student-ChatGPT Interaction Dataset in EFL Writing Education","authors":"Jieun Han, Haneul Yoo, Junho Myung, Minsun Kim, Tak Yeon Lee, So-Yeon Ahn, Alice Oh","doi":"arxiv-2403.08272","DOIUrl":null,"url":null,"abstract":"The integration of generative AI in education is expanding, yet empirical\nanalyses of large-scale and real-world interactions between students and AI\nsystems still remain limited. Addressing this gap, we present RECIPE4U (RECIPE\nfor University), a dataset sourced from a semester-long experiment with 212\ncollege students in English as Foreign Language (EFL) writing courses. During\nthe study, students engaged in dialogues with ChatGPT to revise their essays.\nRECIPE4U includes comprehensive records of these interactions, including\nconversation logs, students' intent, students' self-rated satisfaction, and\nstudents' essay edit histories. In particular, we annotate the students'\nutterances in RECIPE4U with 13 intention labels based on our coding schemes. We\nestablish baseline results for two subtasks in task-oriented dialogue systems\nwithin educational contexts: intent detection and satisfaction estimation. As a\nfoundational step, we explore student-ChatGPT interaction patterns through\nRECIPE4U and analyze them by focusing on students' dialogue, essay data\nstatistics, and students' essay edits. We further illustrate potential\napplications of RECIPE4U dataset for enhancing the incorporation of LLMs in\neducational frameworks. RECIPE4U is publicly available at\nhttps://zeunie.github.io/RECIPE4U/.","PeriodicalId":501030,"journal":{"name":"arXiv - CS - Computation and Language","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Computation and Language","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2403.08272","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
Abstract
The integration of generative AI in education is expanding, yet empirical
analyses of large-scale and real-world interactions between students and AI
systems still remain limited. Addressing this gap, we present RECIPE4U (RECIPE
for University), a dataset sourced from a semester-long experiment with 212
college students in English as Foreign Language (EFL) writing courses. During
the study, students engaged in dialogues with ChatGPT to revise their essays.
RECIPE4U includes comprehensive records of these interactions, including
conversation logs, students' intent, students' self-rated satisfaction, and
students' essay edit histories. In particular, we annotate the students'
utterances in RECIPE4U with 13 intention labels based on our coding schemes. We
establish baseline results for two subtasks in task-oriented dialogue systems
within educational contexts: intent detection and satisfaction estimation. As a
foundational step, we explore student-ChatGPT interaction patterns through
RECIPE4U and analyze them by focusing on students' dialogue, essay data
statistics, and students' essay edits. We further illustrate potential
applications of RECIPE4U dataset for enhancing the incorporation of LLMs in
educational frameworks. RECIPE4U is publicly available at
https://zeunie.github.io/RECIPE4U/.