A while back, the U.S. Department of Education did a meta-analysis of a number of studies comparing online learning and traditional, face-to-face learning. Very basically, the study found that a mix of classroom and online instruction is the most effective form of education, with strictly online education coming in second and strictly face-to-face instruction bringing up the rear–and the controversy.
Recently, a new report from the Community College Research Center challenged these findings. Perfectly timed, the report’s release coincided with the University of California’s announcement that they are seriously considering offering degrees you can earn entirely online. After reading this report and sifting through the one-sided language, I found that it does a fair job of questioning the validity of the data used in the Dept. of Ed. report. As the report goes on, though, a bias against online education becomes clear.
A Critique of the Response to the Meta-Analysis of Online Learning Studies
Let’s break it down…
Stacking the Odds: Shortening the List in Favor of Face-to-Face Instruction
First, let’s look at how the response chopped the Dept. of Education’s list down to 7 studies. From the report:
“[O]f the 23 hybrid courses that were examined in studies included in the meta-analysis, 20 required the students to physically attend class for the same amount of time that students in a face-to-face course would attend; the online portions of these courses were either in on-campus computer labs or were completed in addition to regular classroom time.”
Okay, great point there. It’s important to distinguish between hybrid programs that require time online outside of a traditional class schedule and those that are split more evenly between face-to-face and online instruction. Now, let’s keep in mind that this response isn’t trying to tackle any further claims about hybrid education–they stop here, and those last three courses that were determined acceptable aren’t mentioned again.
The response is, though, focusing heavily on online education, so they decide to weed through the Dept. of Education list to find the studies “that compared fully online courses to face-to-face courses,” of which there were 28. They focused on seven of these, saying the others were not relevant because “(1) conditions are unrepresentative of typical college courses, or (2) target populations are dissimilar to college students.” Okay, so they’re only interested in semester-long courses, not studies on shorter courses; I can get behind that one.
The target population part of this little formula for whittling down the list of studies they aim to debase, though, doesn’t sit well with me. By only looking at “those studies conducted with undergraduate or graduate students in semester-long online courses”, they ignore the “professionals outside of the college setting”, who are taking online courses because they can’t fit a traditional campus education into their schedules–a good-sized chunk of online students. This one has other implications, too. By limiting their focus to students who are in a college setting, they’re also limiting their focus to studies done in a traditional college setting (i.e., a campus college, not an online one). Why is this important? Because campus-based schools are geared towards campus education not online education–and they should be; it’s their thing. Online instruction isn’t their thing, which means that they may not have done the best job developing the online versions of the classes mentioned in the studies. So the odds are stacked against online programs from the get-go because they’re only looking at campus colleges trying something new, which becomes even more clear when they start critiquing individual studies later on.
Online Instruction Holds Its Own while Getting Downplayed
Now that we know how they determined which bits of data to pay attention to, let’s look at the studies in question. Keep in mind as we go through, that these are online versions of campus courses designed by professors who usually teach in a traditional, face-to-face setting. I’m not saying that courses designed to be taught online by professors experienced in that medium of instruction would be that much better, but, gosh, doesn’t it seem like that would be the case?
Comparing strictly online courses to strictly campus courses (occasionally with a hybrid or additional online version of the course thrown in), the studies found…
- Caldwell (2006): “[...] no significant differences [...].”
- Cavus and Ibrahim (2007): “The advanced-collaboration online course significantly outperformed both the standard-collaboration online and face-to-face courses [...]; there was no significant difference between the standard-collaboration online course and the face-to-face course.”
- Davis, Odell, Abbitt, and Amos (1999): “[...] no significant difference [...]“
- LaRose, Gregg, and Eastin (1998): “[...] no significant difference [...]“
- Mentzer, Cryan, and Teclehaimanot (2007): “[...] students in the online and face-to-face classes had the same test scores, but the online group was less likely to turn in assignments [...].”
- Peterson and Bond (2004): “[...] the online group still scored similarly to the face-to-face group [...].”
- Schoenfeld-Tacher, McConnell, and Graham (2001): “[...] online students showed significantly higher adjusted post-test scores [...].”
For those of you keeping score, that’s six ties and one win for online education (two if you count Cavus and Ibrahim’s advanced collaboration class). Which is why it makes total sense when the response report summarizes by saying that there was a “lack of consistent differences in outcomes between online and face-to-face.” Wait, no, that doesn’t make any sense at all.
Grasping at Straws
My favorite part of this report has to be the way it clings to the following assertion [emph. added]:
In addition, eight students who had taken both an online and a face-to-face teacher education course from the two participating instructors were interviewed, and all eight felt that the face-to-face course had better prepared them for teaching.
I’m going to go out on a limb here and guess that this particular teacher education course wasn’t aimed teaching online courses. This finding only shows that a face-to-face teaching course better prepared students for face-to-face teaching, which isn’t surprising, let alone noteworthy. It’s easier to learn how to stand up in front of people and teach from someone who is standing up in front of you and teaching. The fact that the online students scored the same as their face-to-face counterparts is what should be held up as the primary finding, not that eight of the students who did both felt better prepared.
Some Notes on Another Critique of the Response to the… You Get the Idea
Education news website Inside Higher Ed wrapped up their summary of the report with this quote from John Bourne, executive director of the Sloan Consortium:
“I am exceptionally dubious of studies that tend to compare online education and on-the-ground education without even an attempt to understand the differences in the mechanisms of teaching. The jury is absolutely still out on this, and I don’t believe for a minute that it’s about the delivery mechanism, but what the affordances are of the delivery.”
Bourne added that he thinks both reports in question are flawed but interesting, which seems like a great place to leave things: up in the air (at least until we get some real studies done and have those analyzed by non-biased parties).