Wicked Problems and Proficiency (Part 1)

The notion of "wicked problems" comes from Horst Rittel and Melvin Webber in their 1973 article "Dilemas in a General Theory of Planning" in the journal Policy Sciences. The authors note that unlike the relatively "tame" engineering problems of the past (such as building roadways, sewers, and the like), most social problems are not as clear-cut or defined. They term these problems "wicked" problems and identify a list of characteristics of such problems. Looking at these characteristics in turn, I believe that language proficiency testing can be considered a wicked problem.

Characteristic 1: There is no definitive formulation of a wicked problem
In a typical engineering or math problem, the problem itself is independent of potential solutions. One first completely specifies all of the parameters of the problem, then looks at constraints, then looks at options for the solution. Things are not so straight-forward in language. How would one begin to specify all of the parameters of reading proficiency, for example? (It is not coincidence that Tim McNamara titled on of the chapters of his 1996 book Measuring Second Language Performance "Modeling language performance: opening Pandora's Box".)

Characteristic 2: Wicked problems have no stopping rule
With the driving of the Golden Spike, the transcontinental railway was completed. What represents the "Golden Spike" moment in language test development? A testing project ends for external reasons (lack of time or resources; results "good enough" for particular use) rather than for the logical solution to the original assessment problem. Does a test have 50 items because that number is the "right" number?

Characteristic 3: Solutions to wicked problems are not true-or-false, but good-or-bad
Given two different tests of language proficiency, A and B, can one really say that Test A isthe "correct" solution to the assessment problem while test be is "incorrect"? The best one can do is evaluate one test as better or worse in relation to the other (and even then usually only with reference to intended test purpose).

Characteristic 4: There is no immediate and no ultimate test of a solution to a wicked problem
Any test will have potential consequences, not all of which can be anticipated or even quantified. The literature on washback certainly illustrates the less-than-clear relationship between tests use and test users. When can one be sure that their particular "solution" to the assessment problem is correct? What evidence would be needed to make such a claim?

Characteristic 5: Every solution to a wicked problem is a "one-shot operation"; because there is no opportunity to learn by trial and error, every attempt counts significantly
Though tests certainly go through stages of development, there will be a final implemented solution in the form of an operational test. The introduction of a new version of a test does not "undo" any of the consequences for test-takers who took the previous version.


Next time, the other five characteristics...