How are SJT Questions Created?

Abstract

The SJT questions were created following the professional attributes identified from the FY1 job analysis. Questions were written by volunteers at a series of dedicated workshops. The volunteers were not all doctors but should have been familiar with the FY1 role and have worked with junior doctors within the previous two years. The ISFP Project Group employed 89 people to write SJT questions, of whom 69 (77.5%) were senior doctors, two (2.2%) were lay representatives, and the remainder were undeclared. In terms of background, 59 (66.3%) were from a range of acute specialties and 12 (13.5%) from community specialties. This team created a bank of 453 possible questions. These were scrutinized by a team of psychologists who accepted 360 questions as passing this initial stage. A select few writers were asked to moderate all questions to ensure that scenarios were realistic and the terminology was in use across the UK. This group eliminated additional questions, leaving a total of 306. A series of focus groups was then held with foundation doctors who scrutinized the test instructions and up to 20 questions each. They proposed a number of amendments and whittled down the total question bank to 275 items. Once a question bank was established, it was trialled using a panel of subject ‘experts’, i.e. people with similar qualifications to the question writers. Questions survived this process if they achieved a satisfactory level of concordance, i.e. enough experts independently arrived at the same answer under test conditions. A total of 200 questions went forward to be used in the SJT pilots. The SJT model underwent two pilots. The second and larger of these took place in 13 UK medical schools, involving 639 final- year students. Students reported that the content seemed relevant to the Foundation Programme (85% agreed) and that the questions were fair (73.3%). The reasons for understanding how questions are created are to appreciate the following: ● A lot of thought has gone into every question. There should be no ambiguities (unless intended) or ‘tricks’. ● They are written (largely) by senior doctors who are presumably interested in medical training and development.