Generating multiple-choice questions using reverse engineering techniques.

Abstract

The study is aimed at exploring the potential of using ChatGPT-4 to develop multiple-choice questions that adhere to item development principles through the application of reverse engineering techniques in nursing education. To determine whether ChatGPT-4 could generate multiple-choice questions regarding medical-surgical nursing, the researchers first evaluated ChatGPT-4's level of nursing knowledge. The researchers then developed 20 prompts using reverse engineering techniques; these were subsequently entered into ChatGPT-4, resulting in the generation of 60 multiple-choice questions. All 60 questions were reviewed by five nursing education experts to evaluate compliance with item development principles. ChatGPT-4 demonstrated a high accuracy rate (98.00%) when answering multiple-choice questions about medical-surgical nursing education. AI-generated questions primarily adhered to the principles of item development, with an average compliance score of 7.90/9.00. The least adhered to among the nine-question development principles was 'The question stem should use only essential words or phrases.' Given that the AI-generated questions may serve as preliminary drafts for use in nursing education following expert review, ChatGPT-4 can function as a valuable tool in the design of educational materials. When utilizing ChatGPT-4, it is beneficial for nursing educators to apply reverse engineering techniques that enhance question quality through repeated comparison, as well as to analyze the prompts and corresponding AI-generated questions. Furthermore, including structured outlines and representative sample multiple-choice questions using Markdown syntax in the prompt can enhance the quality of AI-generated questions.