Weight-of-Evidence Reporting: What Is It? Why Use It?
Journalists ‘find out where the bulk of evidence and expert thought lies on the truth continuum and then communicate that to audiences.’
When it comes to the news media’s coverage of contested science, global warming stories are the favorite whipping boys of everyone from academics to pundits. Commonly, complaints take aim at such journalistic practices as objectivity and balance and conclude, as did a 2004 news media research report by Max Boykoff and Jules Boykoff, published in Global Environmental Change, that “the continuous juggling act journalists engage in often mitigates against meaningful, accurate and urgent coverage of the issue of global warming.”
They are right. But they and others excoriate long-standing behaviors of journalists that arose to help reporters manage some pretty intractable problems. At this juncture, I urge a modicum of respect for those norms—objectivity and balance—but I am also willing to critique their employment in coverage of controversial science issues. And in light of those criticisms, I would propose an alternative: weight-of-evidence reporting.
Normative behaviors do not survive haphazardly within occupations. Rather, those behaviors that confer value on their practitioners will be sanctioned and vigorously defended. Objectivity and balance are two such norms in the journalism world. Other scholars, such as University of California-San Diego Professor Michael Schudson, have explored the history of these norms. I simply want to argue that one important reason for their establishment is that, although journalism exists in principle to help individuals make reasoned decisions about the world around them, journalists are rarely in a position to determine what’s true. Objectivity and balance have evolved over time to serve as surrogates for truth claims.
Why can’t journalists be responsible for reporting what is true? For one thing, most journalists have neither the background nor the time to develop enough expertise about a particular topic or issue to make validity judgments possible. Science writers, for example, are defined as specialists among journalists, yet most cover a wide variety of topics, from nanotechnology to stem cells. There’s solid evidence that years in the saddle is a good predictor of one’s knowledge base as a journalist—science writers who have been covering the beat for a couple of decades know a great deal about many things—but even experienced journalists cannot grasp the factual intricacies of all they cover.
And even if a journalist were an expert at something, readers will react badly to an effort to declare one position on an issue “more true” than another. In our American culture, journalists are assigned a transmitter role, for better or worse, and going outside the role is often recognized by readers as a violation of expectations.
Objectivity and Balance
If a reporter cannot determine what’s true, what is she to do? The “objectivity norm” responds that, if you cannot tell what’s true, then at least capture truth claims accurately. Objective journalism effectively reproduces the views of its sources.
The benefit of such a norm within a contested arena is that it absolves the reporter of having to ferret out truth and sets an accuracy standard in its place. Validity is replaced by a measure of the goodness of fit between the source’s message and the reporter’s story. If the reporter has faithfully captured the meaning and intent of the source, she has done good work.
The “balance norm,” on the other hand, declares that if you cannot tell what’s true, then be sure to include all possible truth claims in the story. Again, the reporter need not determine who’s telling the truth (and who is not). By including a variety of viewpoints, the reporter instead declares that “the truth is in here somewhere.” Lobbing a variety of viewpoints into the public domain sits well with a society that values the “marketplace of ideas,” so once again the reporter has done good work.
These norms deserve to be valued. Determining truth is a hazardous, messy business even for experts, and we should not expect journalists to accomplish that feat. Validity claims confront the occupation with an almost intractable dilemma, and journalism has done a reasonably savvy job of evolving coping strategies to manage the problem.
Why Change Practices?
While journalists have developed reasonable surrogates for validity claims, these normative practices may mislead audiences. Extensive research on audience reactions to media messages suggests that individuals believe what they read and hear. While surveys of public perceptions of the press indicate growing skepticism of journalistic performance these days, it is still the case that news media coverage of a topic legitimizes it in the public eye. Issues covered by the media are considered to be more important than those not so well covered.
This legitimizing effect is at work even for specialists who encounter media coverage of issues in their own fields. One fascinating study some years ago examined the topic by dividing research papers published in The New England Journal of Medicine into those that got picked up and turned into stories by The New York Times and a set matched on all other variables but that did not garner coverage. A search of the science citation literature subsequently found that those research papers covered by the Times received almost 75 percent more citations in the peer-reviewed literature than did their matched counterparts. Media visibility made this research more important—and, presumably, true—even among other scientists.
What this means, then, is that a journalist can work to meet the high standards of accuracy set by the objectivity norm but might still mislead readers into thinking that a source’s position on an issue is important and potentially true. Adding points of view to satisfy the balance norm can mislead in other ways. As most journalists know, balance typically gets put into operation as the presentation of two contrasting points of view, a strategy that can place a deceptively simple interpretation of an issue before the public.
Equally problematic is the meaning given by audiences to balanced stories. Remember that the journalist is trying to communicate to his readers/viewers that “the truth is in here somewhere.” Communication scholars who have fed balanced stories to readers and then captured their reactions find that audiences interpret such stories in a different and more ominous way—as telling them that “no one knows what’s true.”
Presenting an Alternative
I suggest another strategy that would permit journalists to retain their emphasis on objectivity and balance but still share with their audiences a sense of where “truth” might lie, at least at that moment. I call this strategy “weight-of-evidence” reporting. It calls on journalists not to determine what’s true but, instead, to find out where the bulk of evidence and expert thought lies on the truth continuum and then communicate that to audiences. Reporters are still responsible for capturing points of view accurately (objectivity) and for sharing with audiences the existence of more than one contrasting point of view (balance). But added to that mix would be information about which point of view has captured the hearts and minds of the majority of experts, information about where they think the truth lies at that moment.
For example, before sitting down to write this essay I came across a story in the news section of the October 21, 2005 journal Science titled “Confronting the Bogeyman of the Climate System.” Its author, Richard Kerr, reports on climate experts’ evaluations of the possibility that warming temperatures could melt too much ice at the poles, which in turn could shut down the exchange of warm and cold waters in the Atlantic Ocean. Bringing the oceanic “conveyer belt” to a halt could drastically chill parts of the globe, the European continent among them; this phenomenon was the central climate actor in the recent movie, “The Day After Tomorrow.”
The catastrophic potential of a failed oceanic conveyer belt has been the subject of many media stories and deservedly so. Experts who met at the Aspen Global Change Institute last summer to discuss “abrupt climate change,” however, have concluded that the best available data indicate that increasing greenhouse gases might lead to a slowdown but not to a collapse of the conveyer belt. That conclusion was the focal point of Kerr’s article.
Kerr’s piece is what I would call a weight-of-evidence story. It shares with readers views of scientists on both sides of the issue—some who think a failed Atlantic Ocean conveyer belt needs to remain a major scientific and public concern and those who think that it is less likely than other possibly catastrophic outcomes—but then makes it clear to readers that the bulk of experts who know the science fall into the latter category. What’s true might change as time goes on, of course, but stories such as this can go a long way toward helping us, as recipients of news, make sense of the world.
This is a service that news audiences deserve and one that journalists can deliver without compromising the long-standing norms of their business.
Sharon Dunwoody is the Evjue-Bascom Professor of Journalism and Mass Communication at the University of Wisconsin-Madison. She teaches courses on science and environmental journalism, and as a scholar she studies public understanding of science topics, including the newsgathering behaviors of science and environmental journalists, the nature of messages, and their effect on audiences.