Educational utility of observational workplace-based assessment modalities in Australian vocational general practice training: a cross-sectional study.

Abstract

Background: Direct observation, workplace-based assessments (WBAs) are a fundamental component of competency-based postgraduate medical education. In Australian general practice vocational training, external clinical teaching visits (ECTVs) are key observation-based WBAs. Traditionally, ECTVs are conducted face-to-face, but the COVID-19 pandemic saw the development and implementation of remote ECTV modalities. It remains unknown if perceived educational utility of remote ECTVs differs from traditional face-to-face ECTVs. This study explored the educational utility of ECTVs, including face-to-face and remote formats.

Methods: General practice trainees ('registrars') and external clinical teaching visitors ('ECT visitors', who are independent experienced GP observers) each completed a cross-sectional questionnaire following individual ECTVs undertaken in 2020. Outcomes included overall educational utility of the ECTV as perceived by registrars, registrar ratings of likelihood to change their clinical practice as a result of the ECTV, registrar ratings of likelihood to change their approach to learning/training as a result of the ECTV, and overall educational utility of the ECTV as perceived by the ECT visitor. Educational utility ratings (5-point scales) were analysed descriptively. Univariable and multivariable logistic regression were employed to examine factors associated with dichotomised educational utility ratings.

Results: Response rates were 41% (n = 801) for registrars and 39% (n = 742) for ECT visitors. Most registrars (64.1%) rated ECTV overall educational utility as 'very useful'; 58.5% and 47.9% of registrars rated their likelihood to change practice and approach to learning/training, respectively, as 'very likely'. No statistically significant differences in perceived educational utility ratings were identified between face-to-face and remote video/phone ECTVs (multivariable p-value range: .07-.96). Receiving feedback that was focused/specific/easy to translate into action was consistently associated with registrars' rating overall educational utility as 'very useful' (odds ratio (OR): 12.8, 95% confidence interval (CI): 8.26 to19.9), rating likelihood to change practice as 'very likely' (OR: 2.5, 95%CI: 1.59 to 3.94), and rating likelihood to change learning/training approach as 'very likely' (OR: 3.19, 95%CI: 1.97 to 5.17).

Conclusions: ECTVs are perceived by registrars and ECT visitors to be educationally useful across different delivery modalities and formats. The quality and features of the feedback provided appear most important in ECTVs as an assessment for learning.