AT THE risk of being accused of piling on, the Ombudsman is compelled to comment on the brouhaha concerning the opinion piece by Brett Ferrell and Sam Ross on sexual assault statistics. As the journalistic failings of the piece have already been addressed, the opportunity to discuss how not to use social science in journalism remains.
Social sciences play an ever-increasing role in American society, with a growing influence in many spheres, including public policy and law. Corporations and institutions also look to social science in deciding policy and in planning for the future.
However, this reliance on social science carries with it some danger. Many Americans are largely "illiterate" regarding matters of science and statistics. When statistics are used out of context, and served up as a polemic point or a sound bite, it is easy to mislead a public which cannot evaluate a study's statistical findings or design. However, when one understands basic scientific methods and basic statistical principles, then it is difficult to deceive that person with the science alone.
Obviously, every journalist need not be trained as a scientist, even when utilizing social science findings for their articles, or covering a science "beat." At a minimum, journalists, who have an ethical duty to generally understand the topic about which they are writing, should be competent consumers of scientific information.
To begin, most substantive journals of science are peer-reviewed journals. The peer-review process provides a minimum level of trustworthiness in the published articles. The reviewers can criticize the paper for flaws in design, errors in statistical analysis, conclusions not supported by the evidence, or other problems with scientific method that indicate inaccurate science.
The peer-review process helps weed out poor-quality studies and analyses, often including those who seek to promote political or social agendas cloaked in scientific argument. So the first question to ask regarding a study to be used is whether it appeared in a peer-reviewed journal of science. If it didn't, then the journalist should ask herself why. Is there a problem with the study? Is it bad science?
Second, competent consumers of science look at the credentials of the study's principal investigator, or proponent. Is she a scientist trained in the field of inquiry? Does she work and publish regularly in peer-reviewed journals? Does she work for a scientific institution or university, or instead for a "policy institute" or "think tank," or other "quasi-academic" institution that often peddles sub-par science in order to advance a non-scientific agenda? This is not a determinative inquiry, but credential checking serves as a useful shorthand to get a sense of potential agendas, conflicts, or other elements which may impact the credibility of the study.
Finally, look to the state of the field. Most journalists are used to searching a computerized database. With scientific matters, journalists should conduct a thorough literature search for studies addressing the same, or similar, issues as the study that they propose to use.
A journalist conducts a full literature search because science is a dialogue, an ongoing debate. How often have we seen a story based on a single study that heralds some scientific breakthrough which will change lives, but about which we never hear again?
Scientific knowledge is rarely best weighed in the light of a single study. Good science can be replicated, with similar results when the study is repeated. If 50 studies say one thing, and one study says the opposite, then the state of knowledge in the field is generally held to be in line with the bulk of the evidence. This is particularly true with the social sciences, where most of the research is descriptive and correlative, and thus less controllable in terms of confounding variables.
Once a consumer of science has done these three simple things, then they are prepared to form a reasonable opinion regarding a study's merits. Otherwise, such articles reflect an irresponsible use of science, without regard for the ethical duties of thoroughness and regard for the truth that journalists should have, even with opinion pieces.
Admittedly, editors of a paper share some of the responsibility for the misuse of science by the journalists they edit. This may be particularly true on the editorial and opinion pages, where the authors of a piece may not be professional journalists, or may even be intentionally using the veneer of science to give their opinion respectability it would otherwise lack. The tendency of editors is to err on the side of inclusion, so as to avoid the merest suggestion of censorship. Just as the journalists who write the piece have a duty to be good consumers of scientific information, so do the editors. At a minimum, the editor can take the time to ensure that the authors of the piece have done their scientific "homework."
When a newspaper runs an article that misuses science, whether because of ignorance or some ideological agenda, it reflects poorly on the paper as a whole, which in the long run loses precious credibility. The controversy and response generated by articles such as these protect readers from being misled. While I am not encouraging this paper to stifle any viewpoint, it benefits everyone for a paper to ensure that when a piece contains a substantial amount of writing regarding science that someone has considered issues of scientific validity.
(Brent Garland can be reached at ombud@cavalierdaily.com.)
Read More
PARTING SHOT: No man is an island
By Ava MacBlane | May 16, 2024Retirement, as you can tell, has been treating me well. When people ask me now if I miss it, I laugh and tell them some things I definitely don't miss. But I don't always get to tell them about the things I really do miss.
PARTING SHOT: You never know unless you try
By Lexi Baker | May 16, 2024My time at the University was shaped by the people who showed me grace and kindness and by those who had confidence in me, even when I struggled to have it in myself.
PARTING SHOT: A love letter to Cav Daily and Connection
By Ava Proehl | May 15, 2024My time here taught me to look for the genuine, human connection in every moment.
