The Ease (and Emptiness) of "It's not so simple"

Language testing always involves simplification. We know that the interlocutor in an oral proficiency test is not really an auto mechanic. We know that, in "real life", we don't answer multiple choice questions after reading a newspaper article. We know that a 30-minute essay does not really mimic writing an academic paper.

When comparing language tasks and language tests to real life language use, there will always be a gap. The easiest criticism to level against a test, no matter how well designed, is that things are much more complex and nuanced in the real world. In other words, "it's not that simple".

Unfortunately, the criticism of "it's not that simple" does little to suggest ways to improve tests. While it is important for language testers to continually think about ways to improve construct representation, it is equally important for test critics to recognize that many language testing projects do not have the luxury of waiting until all the relevant research is in before taking action. Most of the language testing professionals I have met are well aware of the limitations of their tests (and the most likely to not recommend using scores from a single instrument to make important decisions). In other words, we are aware that "it's not that simple".

What we would really like to hear is, "it's not that simple, but I think you could greatly improve things within the current framework by considering this". That would help us move forward.