Standardization of objective structured practical examination in terms of validity and reliability in biochemistry: Our first experience

Abstract

Objective: The present study is an effort to standardize and establish the validity and reliability of objective structured practical examination (OSPE) as a formative assessment tool in biochemistry. Materials and Methods: After institutional ethical clearance, brainstorming was done to plan the logistics necessary for the standardization of OSPE in biochemistry at undergraduate level. An assessment blueprint was established, structured checklists for individual procedure stations along with the student's and examiners' instruction manuals were designed. Review and revision of the planned OSPE stations were done and the validity and reliability of the tool were established. The internal reliability of the OSPE stations was determined by calculating Cronbach's alpha from the mean scores obtained by the students in each station during the formative assessments. A pilot study was conducted on a group of 14 volunteer students to establish the homogeneity of the study. The content validity index (CVI) and the coefficient of reliability of the feedback questionnaire for faculty and students were calculated. Feedback taken from students and faculty was analyzed through the Satisfaction Index for 5-point Likert Scale questions and thematic analysis for open-ended questions. Results: Cronbach's alpha of the OSPE questions, during the four formative assessments in sessional examinations, showed high internal consistency of 0.832, 0.861, 0.796, and 0.845. Homogeneity was determined by a split-half estimate of pilot OSPE with Cronbach's alpha value of 0.9. The S-CVI/Ave scores for the students and faculty questionnaire were 0.86 and 0.91 respectively with the internal consistency of 0.83 and 0.89. Conclusions: OSPE is a valid and reliable assessment tool after standardization that provides a structured approach to assessment and can be used for formative assessments of medical undergraduate students in biochemistry.