Pareto Learning Analytics

 The Pareto Principle (knows also as the 80/20 rule) roughly states that 80% of the benefit/cost comes from 20% of the input. (I'm sure there's a more rigorous definition, but this will do for our purposes.) That got me thinking about learning analytics, particularly in terms of the always elusive "learning dashboard" that will finally (in theory) bring the power of data to the business of teaching and learning.

If we grant that there is a Pareto-like effect for educational analytics, this would suggest that perhaps our dashboard won't need very many metrics on it. (Or, perhaps, no matter how many metrics are presented, only a small subset will be actually useful for anything.) Even though modern automobiles collect all sorts of information about the system, only a small subset are presented to the driver. And, of that subset, the only one we really pay attention to in real-time is probably the speedometer. (When was the last time you drove fixated on the engine temperature gauge or RPMs?)

This, in turn, got me thinking about what we could ever expect from a teacher- or student-oriented dashboard, even with "predictive analytics." Sure, it might be nice to know that historically N% of students who got X grade on test Y ended the course with a B+, but does that really change anything fundamental about what the student and/or instructor need to do to change the situation. Do we really need historical performance data to tell us that not leaving major assignments to the last minute is probably a good idea? 

But, you may argue, data allows us to make mid-course corrections. This is probably more true in theory than in practice and is very context dependent. If you are teaching an introductory session (or one of several sections) with hundreds of students, how much leeway is there really to diverge from the planned material? One can certainly hold an "after action report" at the end of the semester, replete with all sorts of information, to try to tweak the syllabus or materials to provide a better experience for future cohorts, but that may be rather tricky to do on-the-fly.

I'm not saying that one should not alter instruction in response to student performance -- I'm all for greater use of real formative assessment in the curriculum. But, rather, my point is that more than learning analytics, the cultivation of an environment in which that type of reflective, responsive teaching is supported (and even expected) is the real prerequisite to educational change, not the provision of a dashboard.