A Comparison of Subject Matter Experts’ Perceptions and Job Analysis Surveys

Abstract

Two common approaches for performing job analysis in credentialing programs are committee-based methods, which rely solely on subject matter experts’ judgments, and task inventory surveys. This study evaluates how well subject matter experts’ perceptions coincide with task inventory survey results for three credentialing programs. Results suggest that subject matter expert ratings differ in systematic ways from task inventory survey results and that task lists generated based solely on subject matter experts’ intuitions generally lead to narrower task lists. Results also indicated that there can be key differences for procedures and non-procedures, with subject matter experts’ judgments often tending to exhibit lower agreement levels with task inventory survey results for procedures than for non-procedures. We recommend that organizations performing job analyses think very carefully before relying solely on subject matter experts’ judgments as their primary method of job analysis.