A Comprehensive Alignment Framework for Building Coherence in an Assessment System

Abstract

Traditional alignment studies that are narrowly focused on content standards and assessment items and are conducted postadministration often lack the timeliness and utility needed to improve assessment systems. This paper introduces a flexible alignment framework centered on assessment-system coherence. Our framework broadens alignment evaluation to include all interconnected elements, from domain definition to score use. We advocate for proactive alignment evaluation by defining expectations in advance and gathering both procedural and empirical evidence throughout the assessment life cycle. By addressing three guiding questions regarding system elements that need to be aligned, the purpose of the alignment evidence, and the types of evidence to use, developers can create customized evaluation plans to enhance alignment during development. The paper provides example alignment questions and an example application of the framework. We discuss the framework's potential to enhance the coherence of assessment systems in the service of meaningful policy and practice.