The AI Revolution: Rethinking Assessment in Audiology Training Programs

Abstract

The release of ChatGPT in 2022, a large language model (LLP) trained by Open AI, garnered considerable attention from various sectors, including the technological domain, academic community, and the public.1 In health care settings, the potential for chatbots to enhance patient engagement, facilitate remote care, and improve access to hearing health care services has been increasingly recognized.2 In the realm of higher education, ChatGPT is the subject of extensive research and testing across a range of fields, including law, medicine, and business.3–7 The algorithm has demonstrated impressive adaptability and versatility, positioning itself as a promising tool for teaching and learning, including generating assessments.8Figure 1: Prompts to generate ChatGPT ideas on ways to use it in higher education and specifically Audiology.Table 1: Sample CREATE Prompt for a Specific Audiology Assessment.Table 2: Sample Audiology Assessment Question Co-designed with ChatGPT.Table 3: Example of Assessment Rubric Generated by ChatGPT.Despite the growing recognition of ChatGPT’s potential in higher education, reactions to its use have been mixed due to the increased potential of plagiarism. Institutions have responded to this risk by increasing precautionary measures (i.e., reverting to traditional pen and paper exams and face-to-face oral assessments), prohibition (i.e., banning AI tools with the support of AI detection software), while others have accepted it (i.e., incorporating AI tools into assessment development and design).8–12 Promoting authenticity and academic integrity in assessment continues to present a priority for educational institutions.13–15 Besides providing the foundation for high academic standards and best practice, authentic assessments and academic integrity enrich students with skills that advance their employability, particularly in fields dominated by technological advances, including audiology.16 As a health care profession, most audiology training programs naturally include practical and authentic assessment, such as clinical exams, objective structured clinical exams, simulations, clinical placements, and clinical vivas. These face-to-face assessments reduce the risk of plagiarism, however, they can be time-consuming for staff. In addition, theoretical aspects of the program are usually assessed via written means. It is these written tasks where AI provides both risk and opportunity. CONSIDERATIONS FOR EDUCATORS When creating effective assessment questions using ChatGPT, there are a few key considerations that educators should keep in mind. First, it is important to ensure that the prompts align with the course objectives and the level of complexity appropriate for the intended level of study.15 ChatGPT prompts are a way to enable conversations between users and the AI which can be tailored to specific subjects and can make responses more accurate and efficient. One way to generate a powerful prompt is to use the CREATE model (Character, Request, Examples, Additions, Types of Output, Extras).17 See Table 1. Educators can also use Bloom’s Taxonomy or a similar guide to aid the creation of prompts that require application of knowledge in context of higher-order thinking skills that require students to analyze, synthesise, and evaluate information at a sophisticated level.11,18,19 As a language model, ChatGPT can be used in several ways to design assessments in audiology training including but not limited to: Scenario-based assessments: ChatGPT can be prompted to generate realistic patient scenarios for audiology students to evaluate and diagnose. These scenarios can be tailored to match specific learning objectives and can include various patient profiles, such as children, adults, and the elderly. Interactive simulations: ChatGPT can be used to develop interactive simulations that simulate real-world audiology scenarios. These simulations can allow students to practice clinical skills in a safe and controlled environment (e.g., case history taking). Language and communication assessments: ChatGPT can be used to facilitate students’ communication skills, such as their ability to explain complex audiology concepts to patients in plain language. ChatGPT can generate patient questions and responses, allowing students to practice their communication skills in a realistic context. While ChatGPT is limited in image and audio generation, other open-source AI tools are available to generate audiogram, electrophysiological and other audiological images to suit any assessment as well as audio files.20 As an example, we asked ChatGPT for teaching and learning ideas, starting with general prompts, and moving to more specific tasks. (See Figure 1 for the range of prompts used.) This process involved a back-and-forth conversation, with the authors evaluating responses and asking ChatGPT to expand on ideas of interest, providing it with Bloom’s Taxonomy, and providing existing assessment examples. The following suggestions were deemed useful and relevant, and were rewritten and summarized by the authors below: Critique question and answer sessions: Students can ask ChatGPT to generate audiology information and concepts on a particular topic and critique its accuracy, depth of information provided, and relevance. Case studies: Students can provide ChatGPT with case studies related to audiology and receive assistance in analysing and interpreting the data. ChatGPT can also provide feedback on treatment suggestions that students propose. Language learning: ChatGPT can also be used to help students learn audiology-related terminology in a more interactive and personalized way. Students can input audiology-related vocabulary and receive definitions suited to their level of language and understanding. Patient education: Students can use ChatGPT to create patient education materials that are tailored to individual patient needs. They can input specific patient characteristics, such as age, language, and level of education, and work with ChatGPT to create resources and recommendations that are customized to their patients’ needs. For an example of a specific audiology assessment task and assessment rubric for students using ChatGPT as part of their work, see Table 2 and Table 3, respectively. CONCLUSION While acknowledging the downsides to AI, it is also important for training institutions to recognize the value in using generative AI to promote co-design with students. The collaboration between educators and students to develop high-level assessment prompts and evaluate the quality of ChatGPT-generated responses will not only foster student engagement but student ownership over their learning, thereby improving the relevance and authenticity of assessments. Educators can use AI models like ChatGPT to strengthen assessments and improve assessment literacy for students.14 Additionally, the use of co-designed AI-generated questions can help assess students’ understanding of theoretical knowledge and their ability to apply it in clinical scenarios.10 Reactions to ChatGPT in higher education vary depending on individual perspectives, institutional policies, and specific use cases. Overall, the response to ChatGPT in higher education is likely to be a mix of enthusiasm, curiosity, scepticism, adaptation, and feedback as it continues to be explored and integrated into various educational settings. ACKNOWLEDGEMENT We would like to acknowledge the contribution of ChatGPT in delivering constructive input for this article (See Figure 1). The CDU Audiology Team experimented with specifically designed user prompts to test current post-graduate audiology assessments against model answers and had multiple conversations with ChatGPT to co-design assessments to improve assessment literacy and assess clinical reasoning skills.