My Grandmother Knows That (But She Might Be Wrong)

When I was in graduate school, one of my professors would occasionally bring up "the grandmother test" when listening to students present their ideas for or results from research. This is a variant of the "so what?" question.

The grandmother test works like this: You've just conducted (or proposed) a study and found that students who spend more time accessing their online course materials get better end-of-semester grades than students who spend less time. Is that really a surprise? Or is it the case that "my grandmother could have told you that"? If the latter, you've just failed the grandmother test. 

Note that the grandmother test is not about the quality of the research -- the study itself could have been quite well-designed, sufficiently powered, appropriately analyzed, etc. -- but rather on the utility of doing the investigation in the first place. Do you really know any more after the study than you did before?

As a heuristic, it is useful to keep the grandmother test in mind, particularly when the context of the research is new but the underlying question may not be. I'm thinking particularly of online learning. I doubt we have a lot of recently published studies breathlessly seeking to determine the relationship between students time reading course textbooks and final course grades for face-to-face instruction, but is that so different from the relationship between LMS access and course grades?

That said, although thinking about the grandmother test can be a useful self-check when thinking about educational research, there is still utility in providing an empirical backing for things that "everybody knows." As Mark Twain has said:

It ain't what you don't know that gets you into trouble. It's what you know for sure that just isn't so.