Does itt Work?

Abstract

Improving Selection to the Foundation Programme (ISFP) undertook a wide- ranging review of options for allocating new doctors to FY1 posts. It selected the SJT. Whether the SJT works or not depends on whether it can accurately predict ‘good’ doctors. There is no real consensus about how to measure the effectiveness of foundation doctors, and so the SJT question is unlikely to ever be resolved to everyone’s satisfaction. However, variations on the SJT have been used in selection to some specialties (e.g. general practice (GP) and public health training). They are also used by many firms in the commercial sector. The SJT pilots suggested a high degree of internal reliability (α = 0.79– 0.85). It was also shown that SJT performance is positively correlated with extraversion, openness, and achievement. A subsequent study (MacKenzie et al. 2017) has shown that SJT score was predicted by emotional non- defensiveness, aloofness, and empathy, as suggested by the non- cognitive component of the UK Clinical Aptitude Test (UKCAT). The SJT score is also correlated with UKFPO performance (MacKenzie et al. 2016). Interestingly, SJT score is not correlated with performance at medical school as measured by the EPM (Simon et al. 2015). Advantages of the SJT over the previously used ‘white space’ questions include the following: ● invigilated conditions so that no one can seek external help with answers ● less reliance on creative writing skills ● questions directly address prioritization, teamworking, and professionalism— all of which are important qualities for new doctors ● evidence from other sectors suggests that situational judgement questions can effectively predict job performance. Although students are unlikely to relish sitting another high- stakes examination in their final year, earlier selection methods were perceived as both burdensome and unfair. A number of studies have suggested that the SJT is unpopular amongst both medical students and faculty members (Sharma, 2015; Sharma et al. 2016). It is impossible to please everyone and you are most likely to approve of this method in retrospect if your score is high enough. Criticisms of the SJT include: ● As noted on, the SJT and EPM are not equally weighted.