Module 4: Build Effective, Sustainable Programs With Measurable Results
In order for global health programs to be effective, sustainable, and continually improving, it is essential to have an evidence-based feedback system in place to evaluate progress. The following are features of a valid evaluation system:
Global health programs cannot claim progress unless there are metrics by which results can be measured and tracked. Valid metrics must conform to the following scientific and ethical standards:(1)
- Relevance: Metrics must accurately measure what is important for community health. Specific relevant metrics will change depending on the goals of the particular global health program, but should generally include health outcomes, health services, and resource inputs.
- Comprehensiveness: Metrics must measure various aspects of community health in order to comprehensively assess progress. Using a single metric can be deceptive. For example, increased access to healthcare is a positive indicator of progress, but if the quality of healthcare decreases, overall community health will not necessarily improve.
- Comparability: Metrics must be comparable over time.
- Independence: Those who measure the data obtained from these metrics must remain impartial to ensure that the data is accurate and unbiased.
Data Collection and Analysis
Census data and civil registries are often incomplete in developing countries. For this reason, the Health Metrics Network of the World Health Organization suggests that “population surveys are the single most important source of population health information.”(2) Individual health data and medical outcomes should be recorded and tracked through patient medical charts. It is crucial to obtain a baseline measurement before a global health program is implemented in order to establish a reference point to determine how much progress was made. There must also be a commitment to transparency in data collection so that data are credible and reproducible.
“Data by themselves do not always tell a straightforward story; meaning is acquired when they are analyzed and interpreted. Data should be synthesized, analyzed and interpreted within the overall context of the health systems functioning and of health intervention delivery.”(3)
Data must be analyzed in order to assess impact. It is important to track absolute progress, and also to determine whether the resource inputs are responsible for health outcomes, or if confounding, unmeasured variables contributed to the results. Data must be presented in a way that is complete, objective, and comprehensible.
Analyzed data should be used to evaluate global health programs across the following four dimensions:
Effectiveness: The fundamental purpose of all global health organizations is to create a lasting impact on community health. The primary reason we measure results of global health programs is to assess whether our efforts are making a difference.
Efficiency: An evaluation of efficiency seeks to answer the question, With these resources, could we accomplish more? This should provide insight into which efforts are highly productive and which investments are wasteful.
Ethics: Global health organizations must hold themselves to the highest standards of medical and professional ethics. Questions to consider include: Are decisions being made in the patients’ best interests? Are local doctors being treated with respect by foreign volunteers? Are patients being treated with dignity? Are there any factors that compromise patients’ quality of medical care? Are providers practicing beyond their qualifications? Are health measures being implemented in a culturally sensitive way?(4)
Financial Responsibility: Resources should be maximally allocated to funding the infrastructure and education that directly lead to effective healthcare. In addition to measuring health impact, global health organizations must evaluate their spending habits to minimize waste.
Plan of Action
An evaluation system is ultimately geared toward improving the implementation of global health programs. If evaluations detect that harm is being done, resources are being wasted, or that there are inefficiencies in the system, action must be taken to address these problems. Conversely, if evidence-based evaluations indicate success, then more resources should be allocated to the relevant programs to further their success.
(1) Adapted from “Who We Are: Vision/Goals/Principles.” Institute for Health Metrics and Evaluation, University of Washington. Accessed on 01 October 2008. <http://www.healthmetricsandevaluation.org/who/vision.html>
(2) "Framework and Standards for Country Health Information Systems (Second Edition).” June 2008. Health Metrics Network, World Health Organization. Accessed on 01 October 2008. <http://www.who.int/healthmetrics/documents/hmn_framework200803.pdf>
(3) “Informal Consultation on Synthesis, Analysis and Use of Country Evidence (SAUCE): Using Health Information for Country-Level Health Policy and Planning.” June 2005. Health Metrics Network, World Health Organization. Accessed on 01 October 2008. <http://www.who.int/healthmetrics/documents/hmn_tool_sauceworkshop_.pdf>