Professionalism in evaluation: Development and validation of a theory-based assessment tool in the U.S. context

Abstract

While professionalization of program evaluation is garnering increasing attention in the literature, a notable gap exists in the absence of a validated, theory-based framework to both comprehend and empirically assess the attributes of a professional evaluator. This study contributes to the debate on the professionalism of evaluators in two ways: by elaborating a multidimensional, theory-based conceptualization of evaluator professionalism, and by developing and testing a professionalism self-assessment tool for novice and practicing evaluators to assess their levels of professionalism. The five dimensions of the tool developed using the sociological model were: expertise, innovation and research, professional autonomy, ethical dispositions, and credentialing. The model was validated using data from online survey administered to a random sample of 2000 members of the American Evaluation Association drawn from the 2018 membership directory. This study provides evidence that the professionalism assessment tool used with both novice and experienced evaluators produced reliable scores and demonstrated validity in measuring key attributes of evaluator professionalism.