Lack of industry-recognised services
Why is language testing for the ICAO LPRs in such a poor state? One reason is a lack of industry-recognised testing services.
Evaluating a language test requires field-specific expertise which regulators typically don’t have. Consequently, regulators approve sub-standard tests. By and large, national regulatory approval is no indicator of language test quality.
In 2010, ICAO acknowledged that ‘many States still lack the expertise and resources to implement ICAO Guidance regarding the selection and development of appropriate testing tools’1. In 2012, with aviation and language industry partners, ICAO went on to launch the Aviation English Language Testing Service2 with the goal of creating ‘a pool of testing systems from which States can choose’. It was a brilliant concept, but it has sadly failed: of the dozens of aviation English tests that are available today, just one is recognised by ICAO. Why?
In the process of applying for ICAO recognition, the Test Service Provider (TSP) is required to submit evidence for the validity and reliability of their test instrument. The evidence requested3 is perfectly reasonable. Any professional language assessment organisation will have some if not all of this evidence to hand. The problem is that the vast majority of TSPs that apply for ICAO recognition fail because they cannot provide this evidence. This is because:
- Gathering this evidence requires test trialling and analysis which is prohibitively costly (the $5,000 fee for ICAO recognition is trivial in comparison); and/or
- Gathering this evidence requires field-specific expertise which the TSP does not have; and/or
- The test instrument is fundamentally faulty which makes the gathering of evidence a pointless exercise.
This is why so few tests have achieved ICAO recognition, and explains, in part, the complete absence of accountability in aviation language testing.
Although ICAO recognition was established specifically to help authorities select and approve language tests on the basis of quality, it has unfortunately never gained traction. Instead, we see weak approval procedures at the national level. When national regulatory approval is cheap and easy to achieve, why would any TSP volunteer to jump through hoops with ICAO? In Europe, the situation is compounded: a weak approval of a weak test in one EASA member state grants the TSP access to markets across all EASA member states.
Some have been critical of ICAO recognition. Of course, the process is not perfect, but it could and should be a powerful tool for regulating aviation language testing. The reasons ICAO recognition has never had the desired impact are complex and multi-dimensional, but beyond political pressures and commercial challenges, they can be boiled down to two things: 1) poor assessment literacy amongst regulators leading to 2) poor assessment practice. Consequently, pilots and controllers continue to take poorly-constructed language tests that fail to address aeronautical radiotelephony communication.
1 ICAO (2010) Language Proficiency: New Test Endorsement Process ICAO Journal, 65:4, 30-31
Download this collection of blog posts here:
