Measuring Agency Attributes with Attitudes Across Time: A Method and Examples Using Large-Scale Federal Surveys

Abstract

Public management researchers are interested in many characteristics of organizations that cannot be directly captured, making aggregated attitudes from surveys an attractive proxy. Yet difficulties in measuring meaningful attributes over time and across organizations have frequently limited statistical designs to a single organization or time. We offer a method for creating such statistical measures across agencies and time using item response theory. Focusing our attention on U.S. federal administrative agencies, we marshal a variety of questions from surveys commissioned by the Office of Personnel Management and Merit Systems Protection Board and employ statistical models to measure three important attributes — autonomy, job satisfaction, and intrinsic motivation — for 71 agencies between 1998-2010. Our study provides a wealth of data for quantitative public management research designs as well as an adaptable framework for measuring a wide range of concepts.