More Learning Outcomes

This is somewhat of a continuation from the previous blog. I came across an article in The Chronicle of Higher Education called "An Insider's Take on Assessment: It May Be Worse Than You Thought" by Erik Gilbert. In it, he describes, among other things, his reading of yet another article called "A Guide for the Perplexed" by David Eubanks,  which appeared in an issue of Intersection, a publication of the Association for the Assessment of Learning in Higher Education (AALHE). 

Gilbert claims that assessment expert Eubanks claims that testing specialists have known for a long time that assessment "doesn't work". After reading Eubanks' article, I think that Gilbert has overstated his position a little. In fact, Eubanks' states his main point in his article: "it is difficult to use assessment results because the methods of gathering and analyzing data are very poor". In other words, it is not the case that assessment, as a field or practice, doesn't and can never work, but rather than it will not work without careful attention to detail and purpose. 

That said,both Gilbert and Eubanks would probably agree that many current assessment programs fall short and that "empirically-based" decisions may actually be driven by flawed analyses and/or previously held agendas rather than empirical insights. In fact, Eubanks illustrates in his article how the same data on student foreign language proficiency can lead to very different policy recommendations by broadening the context of the analysis. 

To paraphrase something that Andrew Gelman (at least, I think it was him...) said, the idea that there are interventions out there with huge effect sizes that people just haven't happened to stumble upon yet is pretty hard to image. Yes, there is good and bad teaching, but there is no silver bullet out there that will "revolutionize" instruction. Program evaluation is hard. 

Gilbert's article includes a nice quote, which I've been mulling over:

[This is perhaps] the single most implausible idea associated with assessment: that grades given by people with disciplinary knowledge and training don’t tell us about student learning, but measurements developed by assessors, who lack specific disciplinary knowledge, do.