To judge the quality of the media coverage of health information, research mostly focuses on ten criteria: adequately discussion of costs, quantification of benefits, adequately explanation and quantification of potential harms, comparison of the new idea with existing alternatives, independence of sources and discussion of potential conflicts of interests, avoidance of disease mongering, review of methodology or the quality of the evidence, discussion of the true novelty and availability of the idea, approach or product as well as giving information that go beyond a news release (Schwitzer, 2008, 2014; Smith et al., 2005). Other quality dimensions applied in content analyses of health news coverage are diversity, completeness, relevance, understandability and objectiveness (Reineck, 2014; Reineck & Hölig, 2013).
These criteria are increasingly relevant as people use online health information more frequently and in addition to the information from their physician for medical decision making (Wang, Xiu, & Shahzad, 2019). Thus, analyzing the quality of health content in the media coverage becomes even more relevant. As Schwitzer (2017) points out, there is a variety of quality problems due to hurried, incomplete, poorly researched news.
To measure quality, the content of health news coverage can be compared to content of the original research paper (e.g., Ashorkhani et al., 2012) or the quality of media content is continuously judged by journalist, medical experts or independent organizations such as HealthNewsReview with respect to different criteria (e.g., Schwitzer, 2008; Selvaraj et al., 2014).
Field of application/theoretical foundation:
Online health information, medical decision making, journalism studies
References/combination with other methods:
Focus group discussions with journalists, editors-in-chief and news gatekeepers (Ashorkhani et al., 2012), focus group discussions with consumers of health information (Marshall & Williams, 2006)
Example studies:
Anhäuser & Wormer (2012); Schwitzer (2008); Wormer (2014); Reineck & Hölig (2013); Reineck (2014)
Information on Reineck & Hölig, 2013
Authors: Dennis Reineck, Sascha Hölig
Research question: Which factors contribute to the quality of health journalism?
Object of analysis: Sample of all health-related articles in four German newspapers: Süddeutsche Zeitung (n = 167), Die Welt (n = 426), Frankfurter Rundschau (n = 219) and die tageszeitung (n = 84)
Time frame of analysis: March, 1, 2010 to February, 28, 2011
Info about variables
Variables: Variables defining five dimensions of quality for health-related newspaper articles, deduction of a quality index: coding of 0 to 100 points for each indicator of the different variables, deduction of a quality index for each article based on these points
Level of analysis: news article
Quality dimension
Variable
Indicator(s)
Diversity (rH= 0.78)
Quantitative diversity
Length of the article
Source diversity
Number of sources
Opinion diversity
Discussion of contrary opinions
Completeness (rH= 0.86)
Journalistic completeness and scientific completeness, risks
For diseases: information about prevention, symptoms and remedies
Scientific completeness
For research studies: information about method, sample and results
Risks
For treatment options: addressing of risks and side effects
Relevance (rH= 0.85)
Source credibility
Sources with the highest reputation
Usefulness
Take-home-messages, references to additional information
Newsworthiness
News factors (e.g., topicality)
Understandability (rH= 0.86)
Simplicity
Simplicity vs. complexity of language
Structure
Well-structured vs. inadequately structured presentation
Conciseness
Concise vs. circuitous presentation
Storytelling
Storytelling vs. matter-of-fact presentation
Objectiveness (rH= 0.95)
Emotionalization
Emotional language
Dramatization
Dramatization of information
References
Anhäuser, M., & Wormer, H. (2012). A question of quality: Criteria for the evaluation of science and medical reporting and testing their applicability. PCST 2012 Book of Papers: Quality, Honesty and Beauty in Science and Technology Communication. http://www.medien-doktor.de/medizin/wp-content/uploads/sites/3/downloads/2014/04/Paper-Florenz.pdf
Ashorkhani, M., Gholami, J., Maleki, K., Nedjat, S., Mortazavi, J., & Majdzadeh, R. (2012). Quality of health news disseminated in the print media in developing countries: A case study in Iran. BMC Public Health, 12, 627. https://doi.org/10.1186/1471-2458-12-627
Marshall, L. A., & Williams, D. (2006). Health information: does quality count for the consumer? Journal of Librarianship and Information Science, 38(3), 141–156. https://doi.org/10.1177/0961000606066575
Reineck, D. (2014). Placebo oder Aufklärung mit Wirkpotenzial? Eine Diagnose der Qualität der Gesundheitsberichterstattung in überregionalen Tageszeitungen. In V. Lilienthal (Ed.), Qualität im Gesundheitsjournalismus: Perspektiven aus Wissenschaft und Praxis (Vol. 325, pp. 39–60). Springer VS. https://doi.org/10.1007/978-3-658-02427-7_3
Reineck, D., & Hölig, S. (2013). Patient Gesundheitsjournalismus: Eine inhaltsanalytische Untersuchung der Qualität in überregionalen Tageszeitungen. In C. Rossmann & M. R. Hastall (Eds.), Medien + Gesundheit: Band 6. Medien und Gesundheitskommunikation: Befunde, Entwicklungen, Herausforderungen (1st ed., pp. 19–31). Nomos.
Schwitzer, G. (2008). How do US journalists cover treatments, tests, products, and procedures? An evaluation of 500 stories. PLoS Medicine, 5(5), e95.
Schwitzer, G. (2014). A guide to reading health care news stories. JAMA Internal Medicine, 174(7), 1183–1186. https://doi.org/10.1001/jamainternmed.2014.1359
Schwitzer, G. (2017). Pollution of health news. BMJ (Clinical Research Ed.), 356, j1262. https://doi.org/10.1136/bmj.j1262
Selvaraj, S., Borkar, D. S., & Prasad, V. (2014). Media coverage of medical journals: Do the best articles make the news? PloS One, 9(1), e85355. https://doi.org/10.1371/journal.pone.0085355
Smith, D. E., Wilson, A. J., & Henry, D. A. (2005). Monitoring the quality of medical news reporting: Early experience with media doctor. The Medical Journal of Australia, 183(4), 190–193.