Investigating the Construct of a Listening Test to Assess Pilots’ Comprehension: a Step-by-Step Project for Test Developers

Pilots and air traffic controllers must demonstrate their ability to listen and speak the language used in radiotelephony communications demonstrated by completing a language test. In this context, it is crucial to assess both interactive listening, when listening occurs together with speaking, and listening in isolation, when there is no speaking or interaction. The purpose of assessing listening in isolation is to reduce the influence of skills that are not relevant to the construct, that is, to minimize construct irrelevant variance (S. Messick 1994). This article describes a project that can be followed by test developers to address the initial step in the development of a test to assess pilots’ listening in isolation: the construct definition. The project is framed within an interactionalist perspective wherein a test construct is defined based on a combination of the abilities that those taking the test should have and the tasks that they should be able to perform (L. Bachman 2007). It is also informed by the work of L. Bachman/ A. Palmer (2010) and the framework proposed by U. Knock/ S. Macqueen (2020) for the development of language assessments for professional purposes. The project outlined in this article may also be of interest to test developers who wish to investigate different constructs of aeronautical English tests, as well as those involved in the development of other types of language assessments for professional purposes.