An appendix does little
Weak methodology trips up a new study about higher educationís impact on the racial attitudes of students
A study released Friday by a team of three researchers, "Racial Attitude Change during the College Years," examined how going to college impacted student attitudes about race. Starting in 2006, the Wabash National Study of Liberal Arts Education (WNS) asked a variety of questions in a survey of college students. The WNS asked these questions when students entered college, as well as at the end of their first and fourth years. Then, two researchers from the University of Chicago and one from the University of Iowa took, perhaps out of context, a select number of answers the WNS had gathered to furnish their own study.
Interested in issues of race at colleges, this team analyzed the one WNS question it thought suited its purposes: "How important to you personally is helping to promote racial understanding?" Students could choose from among four answers: not important; somewhat important; very important; and essential.
The researchers found students' average interest in "promoting racial understanding" declined during their years in college. The study interpreted this trend as a contradiction of the common assumption that students generally become more sensitive to racial issues during college. The report closed by saying: "Colleges that can take steps that promote environments conducive for cross-race friendship and discussion may have the greatest impact on students' racial attitudes."
Unfortunately, this study's results suffered from self-selection. The research group focused on the relevant results to the question it had selected. It would have been helpful to randomize these responses, a common statistical move which these researchers admittedly skipped. Selecting from a large, random dataset provides a broader array of responses, as opposed to this study, which drew from fewer than a dozen universities and six liberal arts colleges while omitting community colleges.
In this case, the report's message seems hardly fair given its choice of methods. An answer sheet with four options disallows nuance, for example. Moreover, the researchers purposely chose a question which was concrete and not abstract. This overlooks how some racially conscious students may just be uncomfortable taking action. The study also fails to address the alternative interpretation that older students may be less inclined to take action to improve racial understanding because they feel their campuses are sufficiently tolerant. Of course, students in the racial majority may overlook certain problems affecting minority groups and therefore erroneously conclude their campuses are tolerant. The data, however, suggests that students of all races, not just majority groups, become less interested in promoting racial understanding in college.
Statistically speaking, the researchers take adequate steps to ensure the accuracy of their figures. But even the most robust appendix will not cancel out poor methods.