The Pediatric Ethics & Professionalism Assessment Tool (Pedi-EPAT): Development and implementation of a novel, competency-based assessment

Abstract

Objective

To develop and demonstrate validation of a competency-based formative assessment tool, the Pediatric Ethics & Professionalism Assessment Tool (Pedi-EPAT).

Methods

Using a multi-phased study design, a panel of national experts in pediatrics, ethics, and medical education participated in tool development and established content validity using a Delphi approach. To optimize rating accuracy and minimize inter-rater variation, the panel completed frame-of-reference training, consisting of inter-rater comparisons of 3 recorded, simulated scenarios involving parent-actors and pediatric trainees. Outcomes included degree-of-association of raters' ordinal assessments, agreement of ratings between experts and raters, and median ratings per simulated scenario. Kendall's coefficient was used to estimate consistency across raters, and a structural equation-modeling framework assessed internal reliability

Results

11 experts participated in the 2-round Delphi. The final Pedi-EPAT achieved consensus and contained 34 items, including 2 items independent of the Delphi. During FOR training, consistency among the 6 raters by Kendall's coefficient was low-moderate (w = 0.33–0.35, p < 0.05). The tool demonstrated excellent internal reliability (composite reliability = 0.97)

Conclusion

This novel, competency-based, formative assessment tool demonstrated content validity and internal reliability.

Practice implications

The Pedi-EPAT could aid supervising staff in providing high-quality assessments and feedback on ethical behavior and professionalism for pediatric trainees engaging in conversations with families.