I had a chat with a student the other day about how to reconcile emerging understanding of the cognitive processes involved in reading with the assessment of reading in educational and other contexts. Thanks to relatively recent technological interventions such as eye-tracking machines and not-so-recent theoretical concepts such as working memory, the research literature is replete with examples of how slight changes to the context (e.g., different font size) can alter processing and how there are complex interactions between the context and the cognitive profiles of the learner (e.g., vocabulary size, purpose for reading, working memory size). The student was pondering whether we need to "control for" these various individual and contextual differences in our educational assessments.
The answer is not straight-forward. Much of the challenge is definitional -- what do we mean by "assess," by "reading," by "control for," etc.? These words are doing a lot of work and need to be unpacked.
Even at a relatively simplistic level, our terms are shorthand for a whole slew of different things. When we talk about a "reading test," in a language proficiency context, we usually aren't really talking about how the person reads (i.e., what particular cognitive processes are involved) but rather about that person's comprehension of what they have read. This is why there is no push (as far as I know) for incorporating eye-tracking into the Defense Language Proficiency Test (DLPT), for example.
But even comprehension is slippery. Outside of the educational testing world, we virtually never show that we have comprehended something we have read by answering multiple choice questions about it. Rather, we take direct action to accomplish an immediate task (such as reading the instructions on a machine and then using the machine), we deploy nuggets of information that we have learned through reading in conversations with others, etc.
I've been thinking about an analogy with basketball (or any sport, really) that is usually employed when talking about individual differences. (This isn't an original thought.) People have a range of physical attributes (tidal volume of the lungs, vertical leap, height, arm strength, etc.), cognitive attributes (perception, attention, decision-making, reaction time), and background knowledge (understanding of basic positions, familiarity with other team's playbook, knowledge of rules) that we can measure more-or-less in isolation. But being a good basketball player is none of these things; it is an amalgam of all of these things. Good players are those who are able to use their particular collection of abilities in the most effective way to maximize their team's chances of winning the game.
SLA researchers are like sports researchers -- they are always looking for ways of improving their measurements of the various capabilities that undergird performance. To do so, they try to isolate the specific variables that they are interested in and try to make the case for how that particular variable might help explain variations in performance that we see on the court. The sophistication and precision of the research methods increases over time.
Language testers, on the other hand, are more like coaches. They need to determine who to start, who to bench, who to cut, etc. They operate at a more global level of "better" versus "worse" players for a given situation. They are aware that the individual difference variables are implicated in the performance of the players, but their primary concern is a player's functional performance ("proficiency") on the court.
No comments:
Post a Comment