Evaluating Media Reports
Most of us get our information about topics related to science or health through media outlets. Sometimes it can seem like new studies are being reported every day and at times they directly conflict with other reports. When evaluating a media report, whether on the Internet, in print or on TV or radio, check for the following:
- Is the organization providing the information reliable?
- Is the report based on a scientific study or a personal anecdote (an individual story)?
- If the report is about a scientific study, evaluate the information provided about the study using the criteria outlined in “Evaluating scientific studies” below. A good media report will not only include information about where the study was published, but also information about the way the study was done and the size of the study.
- Who is interviewed or quoted? Is the person a scientific or medical expert? How much information is provided about the person being interviewed?
- Are the people being interviewed sharing anecdotes or are they talking about the data or research? While it is sometimes good to hear from people who are personally affected by the topic, it is important to distinguish between an opinion based on someone’s experiences or biases, and a scientific evaluation of the strengths and limitations of the study. If you have ever read a scientific paper, you might recall that the paper not only included the findings of the study, but also its limitations. As a result, a study author or scientist being quoted in a media report will typically be quite specific in what they are willing to state and will typically reference the data, whereas someone voicing an opinion, especially someone with a personal bias, tends to speak broadly, sometimes delivering more assumptions, general statements, or emotion than facts.
For example, speaking about the same study, a scientist might say, “This study shows that drinking 100 cups of coffee every day for 10 years leads to a two-fold increase in the risk of developing stomach cancer.” Whereas, a person from the anti-coffee drinker’s club might say “This study confirms that drinking coffee causes cancer.” - Journalists often talk about presenting a balanced story. However, a few caveats are important to remember here:
- An expert and a parent might be included so as to present a balanced story, but if one is motivated by data and the other a personal experience, this is not balance. It is scientifically based versus emotionally based information.
- Consider the size and expertise of the group supporting each side of a story.
- Which position is supported by scientific bodies or other researchers in the field?
- The goal of a journalist is to appease a large audience. One of the tools that allows for ratings or skyrocketing readership in the media is controversy. Painting an accurate picture may be secondary to the goal of “getting a reaction.”