Rate my MOOC
MOOC review sites yield little useful data
Anonymous forums are among the Internet’s grimmest landscapes. Academically oriented websites like ratemyprofessors.com are pitched at a more sophisticated level than their non-academic counterparts — such as gossip forum Collegiate ACB, a dark younger cousin of Rate My Professors — but students still post with venom, often to the exasperation of professors who dare to give grades below a B-minus. Students publishing on Rate My Professors evaluate their instructors based on defective criteria, as if easiness and hotness had anything to do with a professor’s quality. Though Rate My Professors is founded on a questionable ethic, and yields results of questionable accuracy, the existence of such sites is understandable. Given college’s skyrocketing costs and the inconvenience dropping a course can entail, consumers want confirmation their professors will deliver the goods, and the grades, they need.
Rate My Professors is nothing new, but coursetalk.org is. A student review site of massive open online courses, Course Talk has in recent months emerged alongside other MOOC review forums such as grademycourse.com and topfreeclasses.com. It’s no surprise that MOOCs, which arose from the same Silicon Valley sensibility behind crowdsourcing, would generate online spaces for students to flip out about their flipped classrooms.
The MOOC review sites have been drawing a disproportionate share of positive comments compared to sites like Rate My Professors. It’s not hard to see why: MOOCs are free, and if you don’t like a class you can drop it with impunity. The MOOC review sites offer a fascinating glimpse into the population of nontraditional learners enrolled in online courses. Students from Stockholm to Macedonia, posting on Course Talk about a University of Pennsylvania MOOC on modern American poetry, say the class has allowed them to “dwell in possibility.”
But websites like Course Talk tell us little about the quality of various MOOCs. Such forums sketch an incomplete picture. Rate My Professors does too, but to a lesser extent. The real indicators of whether a MOOC is good or not lie in the big data: how many of the thousands who initially sign up for a course end up dropping it, how many complete the assignments and how many fail. Student feedback is useful, but a better measure of course quality is student engagement.
Course Talk also attracts a biased sample pool. Students who submit negative reviews on Rate My Professors might be motivated by altruism or spite. They might wish to shield fellow students at their school from a poor class, or they might hope to wreak revenge, however trivial, on a despised faculty member by encouraging others to boycott the course. Neither incentive applies to MOOC-takers. Students in MOOCs are not members of the same tribe the way students at the same school are. And fewer than 10 percent of MOOC users finish courses they begin. Dropping out online is easier, and cheaper, than dropping out in person. Students who lose interest in a MOOC and drop out are unlikely to take the trouble to post a negative review. The students posting are more likely to be the ones who stick with a course and enjoy it.
The sample pool for MOOC reviews is also proportionally much smaller. If three students out of a 30-person University class post on Rate My Professors, that data, while incomplete, might tell you something useful about the class. If 30 students out of a 30,000-person Coursera class submit online reviews, that data is diluted to the point of insignificance.
MOOC review sites also have the potential to amplify the problems of sites such as Rate My Professors. Some academic topics do not lend themselves to the flash and flair that drive enthusiastic Internet traffic. A chemistry explosion is more interesting to watch than a philosophy lecture. And the power dynamic of such sites — thousands of people shouting down a lecturer — makes professors beholden to the sometimes not-so-intellectually pure interests of their students: which they already are, of course; but the online world was a chance to do things differently.