PG82 Are remote objective structured clinical examinations (OSCEs) an appropriate method of summative assessment?

Background March 2020 saw the United Kingdom (UK) in chaos as the Covid-19 pandemic led to mass lockdown. Education provision changed dramatically over night with universities rapidly adapting to online teaching. While some universities put postgraduate clinical courses on hold to enable staff to focus on practice, others changed their delivery and assessment methods in order to support student progression and implementation of new skills in practice. This project focuses on evaluating the move to online OSCEs for a post graduate clinical assessment module delivered at a regional university. Summary of Work A case study approach was taken using mixed methodology. A purposive sample of OSCE examiners participated in semi-structured interviews. 56% of examiners (n=5) took part. Transcripts were analysed using template analysis and NVivo software. OSCE check lists were uploaded to CAE Learning Space to ensure parity from two retrospective cohorts. Students from cohort one participated in face to face OSCEs and cohort two participated in online OSCEs. Both cohorts sat the same stations. Descriptive statistics, using both Learning Space and SPSS, were utilised as a method of triangulation focusing on both scores achieved and pass rates comparing similar sized cohorts of learners undertaking face to face (n=171) and online (n=228) OSCEs. Summary of Results Mann-Whitney U tests demonstrated a statistical difference in the range of marks between cohorts but no statistical difference in the pass rates, which were comparable to other organisations. Remote OSCEs appear to be acceptable by faculty as an appropriate method of summative assessment although careful planning needs to be undertaken with regards to student and faculty preparation1 2 including: Faculty examiner training Guidelines for the use of proxy/standardised patients need to be established and incorporate ethical considerations Policy for setting up and communicating Zoom meetings to students, including read receipts and confirmation of attendance A method of inter–examiner communication during OSCEs e.g. WhatsApp Stations need to be fit for purpose and consider multiprofessional practice and service user needs Inter–rater reliability needs to be actively assessed during the OSCE and issues addressed where required Conclusions and Recommendations Further research is required in order to incorporate student views of online OSCEs before they can be considered as an acceptable method of summative assessment. However, initial evaluation is favourable, particularly in relation to OSCEs focusing on communication skills, history taking etc. References Harden RM. Revisiting ‘Assessment of clinical competence using an objective structured clinical examination (OSCE)’. Medical Education. 2016;50(4):376–379. doi:10.1111/medu.12801 Major S, Sawan L, Vognsen J, & Jabre M. COVID-19 pandemic prompts the development of a Web-OSCE using Zoom teleconferencing to resume medical students’ clinical skills training at Weill Cornell Medicine-Qatar. BMJ Simulation & Technology Enhanced Learning 2020; bmjstel-2020-000629. doi:10.1136/bmjstel-2020-000629