Pragmatic variables in role-play design for the context validity of assessing interactional competence

Assessment task design influences the elicitation of targeted linguistic and communicative demands by creating the conditions under which the task is performed. Through the task design, plausible language use contexts are also created. In this study, I examined the role of pragmatic variables in role-play design to ensure the context validity of assessment of interactional competence. Using conversation analysis (CA), I qualitatively analyzed high-level L2 English learners’ performances elicited from five role-plays that measure pragmatic abilities in interaction. I examined how the learners oriented to the pragmatic variables embedded in the role-plays. The elicited role-play performances shared action sequences (e.g., opening) and generic interactional features. However, the learners utilized sequential organizations and grammatical resources differently to accomplish context-specific actions. For example, the length of turns tended to be longer and sequential organizations were more extended when the learners were engaged in the role-plays of formal pragmatic functions with an interlocutor with more social distance. The learners utilized diverse grammatical formats in a non-uniform manner specific to sequential positions of pragmatic actions. I discuss the importance of assessment task design for context validity and eliciting evidence of interactional competence. Further, I argue how CA’s analytical attention to real-time details of interaction captures the evidence of interactional competence.