Untrained test personnel
Why is language testing for the ICAO LPRs in such a poor state? One reason is that regulators give the job of aviation English assessment to untrained, unqualified personnel.
Three things are necessary for aviation English testing to be effective:
A well-constructed test instrument comprising a range of tasks which together address domain-specific listening and speaking for the target population;
A system of administration including standards and procedures for test maintenance, personnel training and monitoring and test instrument validation; and
A team of trained, qualified personnel (managers, administrators, validation officers, interlocutors and raters).
If one of the above is faulty or missing, measurement will not be effective.
In the UK1 and the USA2 (and no doubt in other countries), Flight Examiners (FEs) and other personnel are permitted to conduct language proficiency assessments for licensing purposes. No test instrument is required. No training is required. No monitoring to ensure validity and reliability is required. Layperson assessments are made - quite literally - on the fly.
To be clear, aviation professionals are essential in aviation language test design and administration. But proficiency in English and an FE rating does not equate to language testing expertise. In an otherwise tightly regulated industry and in an assessment context where careers and safety are at stake, it is astonishing that regulators would permit untrained personnel to operate without monitoring and without a language test instrument.
Until regulators better understand language testing and ensure that reliable assessments are conducted by trained and competent personnel using valid test instruments, pilots and controllers will continue to take poorly-constructed language tests that fail to address aeronautical radiotelephony communication.
2 See: https://www.faa.gov/documentLibrary/media/Advisory_Circular/AC_60-28B.pdf
Download this collection of blog posts here:
