ABSTRACT
Service Members and military beneficiaries face complex and ill-structured challenges, including suicide, sexual violence, increasing health care costs, and the evolving coronavirus pandemic. Military and other government practitioners must identify effective programs, policies, and initiatives to preserve the health and ensure the readiness of our Force. Both research and program evaluation are critical to identify interventions best positioned to prevent disease, protect the public’s health, and promote health and well-being within our ranks to retain a medically ready force and reduce the global burden of disease. While military and medical leaders are typically well versed in research and understand the role of research in evidence-informed decisions, they may be less aware of program evaluation. Program evaluation is the systematic application of scientific methods to assess the design, implementation, improvement, or outcomes of a program, policy, or initiative. Although program evaluators commonly utilize scientific or research methods to answer evaluation questions, evaluation ultimately differs from research in its intent. Several recently published federal and Department of Defense policies specifically reference program evaluation, emphasizing its importance to the military and government as a whole. The Army is uniquely positioned to conduct medical and public health evaluation activities and there are several Army organizations and entities that routinely perform this work. For example, the United States Army Public Health Center (APHC) is among recognized military experts in public health assessment and program evaluation. Given the breadth of our work, the APHC understands the challenges to conducting evaluation studies in the Army and we have thoughtfully examined the conditions common to successful evaluation studies. In this commentary, we share our lessons learned to assist military colleagues, potential partners, and others in successfully evaluating the programs, policies, and initiatives necessary to keep our Service Members and beneficiaries healthy and ready. There are several challenges to executing evaluation studies in the Army that may be relevant across all Services. These include but are not limited to frequent Army leadership transitions, urgency to report study results, lack of program documentation and adequate planning for evaluation, expectation management to ensure stakeholders are well-informed about the evaluation process, and a disorganized data landscape. These challenges may hinder the successful execution of evaluation studies, or prevent them from being attempted in the first place, depriving Army leaders of quality, actionable information to make evidence-informed decisions. Despite the aforementioned challenges, we have identified a number of best practices to overcome these challenges and conduct successful evaluation studies. These facilitators of successful evaluations can be summarized as: collaboration with engaged stakeholders who understand the value of evaluation, evaluation studies aligned with larger strategic priorities, agile methodology, thoughtful evaluation planning, and effective communication with stakeholders. We wholeheartedly recommend and encourage program evaluation at every opportunity, and we anticipate the call for evaluation and evidence-informed decisions to continually increase. Our hope is that others – to include partners and stakeholders within and external to the military – will be able to leverage and apply this information, especially the identified best practices, in their evaluation efforts to ensure success.
Program, complexity, and system when evaluating sustainable development