Coming to Terms With Uncertainty

Influenza viruses change all the time. No one knows when and where a severe pandemic strain will emerge. Most people, including journalists, have trouble dealing with high levels of uncertainty and change. “Further research is needed” does not make for good stories (nor does it make for effective public health messages).

On this page, Harvard epidemiologist Marc Lipsitch, a voice of research and reason in the preparedness debates of the past decade, explains how multiple sources can help balance uncertainty, how scientists, health officials and journalists all use information very differently in an outbreak, and how mathematical models try to reduce uncertainty.

On this page...
  Understanding conflicting and uncertain information »
  Three different uses of information in an outbreak »
  What is news in a pandemic? »
  Reporting on mathematical modeling results »
  Specific aspects of the science of controlling pandemic influenza »

Marc Lipsitch, Professor of Epidemiology, Harvard School of Public Health:

Understanding conflicting and uncertain information

It’s possible to report on infectious disease outbreaks without being a graduate-trained scientist.
There’s tremendous uncertainty about a lot of the science of influenza viruses and there’s therefore a lot of disagreement among scientists, mainly on the topics about which we’re uncertain. For example, there is legitimate uncertainty about which changes in the influenza virus are most likely to change its adaptation to humans or the severity of disease it causes; about why influenza is usually seasonal (but less so in pandemics); and about why drug resistant viruses become common in some cases and not in others.
So there’s a reason to have multiple sources, good and really knowledgeable sources, to make some sense of where consensus lies.
Many journals (for example, the British Medical Journal and PLoS Medicine) have summaries of their articles that lay out what was known, what the new study adds, and what the implications are. These are the first questions that you should want to know the answers to when a new study or release of data comes out. Many new studies and data releases add little to our current knowledge. These should be threshold questions for deciding whether a piece of information is newsworthy.
Another perspective on the previous point: few good scientists write paper after paper on unrelated observations; rather, they understand why and how their observations are relevant to a larger picture. In my small experience with journalism, that offers a pretty good description of the best journalists as well.
There is a lot of information coming out, and the last thing you want to do is contribute to confusion, panic or complacency. One of those three is hard to avoid in any given case; in good news, bad news or mixed news, there can be grounds respectively for complacency, panic, or confusion. Once again this emphasizes the importance of contextualizing information.
The other aspect is prioritization. With so much news and information and limited space to talk about flu, it’s more important to talk perhaps about the mortuary directors once in a while than to write about each press release. Prioritizing news stories will help to make space for the important ones.

Three different uses of information in an outbreak

Public health authorities should and do manage information for several different reasons—for scientific completeness, for reasons of scientific caution, and for reasons of political or economic caution.
When we learn something about a new disease, especially something as publicly important as a novel pandemic flu strain, there are at least three different very distinct uses of that information.
1. The information is important for scientists to advance the science.
2. It’s important for public health workers, who also base their work on the science, to make a response, and this may often be done well before it is possible to achieve the scientific certainty that scientists strive for.
3. Finally, there is the work of public health communication.
The facts and the scientific basis for each of these is the same, and the uncertainty is the same. But scientists have the luxury and also the duty of reserving judgment and not making strong statements until they know, or think they know, what’s going on. Communicators and responders have to act whether or not they can be certain. They use the same data, but they use it in a different way.
Early responders, particularly those like WHO and CDC, must have a different standard of evidence, a sort of guilty until proven innocent standard. Action has to be taken before all the evidence is in. The reasons are obvious: There are delays in getting samples to labs, delays in getting results, delays in getting enough data/samples to be certain, and delays in performing analyses on those data. Waiting until those delays pass loses time. Beyond that, once the tests are done, there is still some uncertainty.
Therefore, if first responders want to have some hope of containing an outbreak, they will have to respond to false alarms. Unfortunately, perhaps, this same group of people that is supposed to be responding intentionally to false alarms also has the duty of trying to maintain public understanding and public calm. (Read more detailed thoughts on this issue).

What is news in a pandemic?

The start of pandemic response plans can trigger all sorts of damage, economic and otherwise.
WHO, although less than in the past, is still restricted in its ability to report health information from a country, sometimes formally restricted, and also dependent on the goodwill and cooperation of ministries of health and host governments. So announcements to the public must be cautious and must emphasize the possibility that things are better than they seem, given at the same time that actions must reflect the possibility that they’re worse than they seem.
Understanding those dynamics helps us to interpret some of the actions and statements.
In reporting on press releases from drug companies and health authorities, the real question to ask is how the latest finding changes our understanding about an outbreak or about the situation in a given country or the global situation. Are the claims emphasizing the best or the worst case? How uncertain are we? Or, in other words, how strongly does the evidence support our best guess, because to a greater or lesser degree, all science in an outbreak is framed in uncertainty.

Reporting on mathematical modeling results

When we’re planning, we have to extrapolate our knowledge of how a particular measure, such as an antiviral drug, worked in the past. What is the basis on which we can expect it to work equally well or better or worse in the next case?
What mathematical models to date have done is to model how transmission of the virus might be blocked, examining how changes in contact patterns or changes in the course of the infection—if antivirals could make such changes—would alter the epidemic. These models try to reduce the uncertainty in this area.
For mathematical modeling results, a reporter should be able to get someone to explain the results on the back of an envelope or in a 20-minute phone conversation once they’ve done their big computer simulation. Underneath it all it might be complicated, but the phenomena are not counterintuitive. Everything modeled is something in our experience, and the models are simply a way of quantifying it and putting it together. As Robert May, the former UK Chief Scientist, President of the Royal Society, and salty-tongued Australian, has often said, “If the [unprintable] guy can’t explain it to you, he doesn’t understand it.”

Specific aspects of the science of controlling pandemic influenza

The time frame on which events occur is enormously important. With almost every response measure, such as a drug or a vaccine or a mask, it is going to be in limited supply. How will production and timing correspond to the timing of the epidemic? One failure of communication (and reporting) in the H1N1 pandemic was the long-standing suggestion in rich, Northern Hemisphere countries that the vaccine would be available and plentiful. That is probably true, but it will be plentiful only after much of the pandemic is past.
With most response measures, you need to understand whether the measure depends on knowing who’s infectious and whether that’s practical for a given infection. In SARS, it turned out to be practical, but no one knew at the time, how much transmission was happening from asymptomatic or pre-symptomatic people.

Editor’s note: The article above was updated by Marc Lipsitch in October 2009 from his original talk at the December 2006 Nieman Conference “The Next Big Health Crisis—And How to Cover It,” made possible with the generous support of the Dart Foundation.

2 Comments on Coming to Terms With Uncertainty
Pradip Dey says:
April 29, 2010 at 1:01am
First of all, thanks to Neiman Foundation for this nice piece. I would like to add few points to ponder. In my opinion, one of the most important aspects is response time to counter a rumour on pandemic which will create a brand name for any Institution. Another important aspect for pandemic reporting is not to have an activism approach (other consideration than reporting the truth) on the part of reporter. More often than not, this activism is observed in press.
greg tuke says:
April 28, 2010 at 7:44pm
this is helpful information, particularly if one is to be a reporter, but also in understanding how to evaluate various kinds of reporting. seems like a key in this is to overcommunicate, so folks dont feel like info is being withheld.
Submit a Comment
Enter the words above: * Enter the numbers you hear: *
Switch to audio Switch to image
Thank you for your comment. It will be published after it is approved by an editor.