Abstract
Ongoing monitoring and process evaluation of public health initiatives can be challenging, particularly with complex initiatives involving multiple partners and systems and environmental change goals. Limited resources for monitoring and evaluation contribute to these challenges. Many initiatives and programs generate multiple reporting documents, however, such as planning documents, site visit or trip reports, and summative evaluation reports. Often these documents are underutilized, although they contain valuable information that can be mined. If (ideally) thoughtfully designed and then systematically reviewed using quantitative and qualitative content analysis methods (e.g. Neuendorff 2019, White and Marsh 2006), such documents can serve as valuable and cost- and time-efficient sources of data for monitoring and course correction. The presenters have experience both teaching, designing, and utilizing documents and content analysis methods for supporting effective planning, monitoring, and process evaluation. We will briefly present both quantitative and qualitative content analysis methods that can be applied both to documents tailored for such methods and pre-existing documents. The use of simpler tools for managing and analyzing data such as Excel as well as more complex computer assisted data analysis software (CAQDAS) packages will be reviewed. We will give examples utilizing experiences applying these methods to local public health department work as well as global health engagement projects. Following presentation giving an overview of content analysis methods and developing tools for applying them to documents via identified criteria, participants will be dividing into small breakout groups, and supported in brainstorming discussions about how they can use these methods with their own programs and initiatives.
Following this workshop, participants should be able to: 1) discuss the differences between quantitative and qualitative approaches to content analysis of documents, for monitoring purposes; 2) understand the process of identifying criteria or categories tied to the goals and objectives of initiatives or programs, even if goals and objectives change over time, and utilizing these criteria to develop tools for systematic review and content analysis of available documents; and 3) leave with initial ideas about how to apply these tools in their own work and resources for doing so.
Key messages
Program or project generated documents such as plans, site visit and existing reports are underutilized for monitoring and process evaluation but can serve as valuable and accessible data sources. Systematic review including quantitative and qualitative content analysis methods can be used with project documents to support valid and rigorous data extraction for monitoring purposes.
Qualitative process evaluation from a complex systems perspective: A systematic review and framework for public health evaluators
