Lesson 7
HMIS MONITORING AND EVALUATION
HMIS MONITORING AND EVALUATION
Monitoring is the systematic collection, analysis, and use of information from programs for three basic purposes:
(1) Learning from the experiences acquired (learning function); (2) Accounting internally and externally for the
resources used; and (3) the results obtained (monitoring function) and taking decisions (steering function).
Evaluation is assessing an ongoing or completed program or policy as systematically and as objectively as
possible. The object is to be able to make statements about their relevance, effectiveness, efficiency, impact, and
sustainability.
M & E Purpose
The primary aim is to have a strong M & E and review system in place for the national health strategic plan that
comprises all major disease programs and health systems.
M & e Plan
The national M & E plan and system should address all components of the framework and lay the foundation for
regular reviews during the implementation of the national plan.
According to the National Health Mission (2014), strategies for operationalizing the framework should:
• be primarily country-focused but also offer the basis for global monitoring;
• address M & E needs for multiple users and purposes, including monitoring program inputs, processes
and results, tracking health systems performance and evaluation;
• facilitate the identification of indicators and data sources, provide tools and guidance for data analysis,
and show how the data can be communicated and used for decision-making
• bring together the monitoring and evaluation work in disease-specific programs with cross-cutting efforts
such as tracking human resources, logistics and procurement, and health service delivery.
M&E FRAMEWORK
Monitoring and evaluation (M&E) is a core component of current efforts to scale up for better health. Global
partners and countries have developed a general framework for M&E of health system strengthening (HSS).
The framework builds upon principles derived from the Paris declaration on aid harmonization and
effectiveness and the IHP+, putting country health strategies, and the related M & E processes such as annual
health sector reviews, at the center.
FIGURE 7.1. WHO Framework for Monitoring and Evaluation of Health Systems Reform/ Strengthening
FIGURE 7.2. Common framework for monitoring performance and
evaluating progress in the scale-up for better health
Key Performance Area Key Indicator
Reproductive Health 1. Family planning acceptance rate
2. Antenatal care coverage
3. Proportion of deliveries attended by skilled health personnel
4. Proportion of deliveries attended by HEWs
Immunization 5. DPR-3 (Pentavalent-3) coverage (>1 children)
6. Measles immunization coverage (>1 children)
Disease prevention and 7. Malaria case fatality rate amongst patients under 5 years of
control age
8. New malaria cases per 1000 population
9. New pneumonia case amongst under 5 children per 1000
population of < 6 years
10. TB case detection rate
11. TB cure rate
12. Clients receiving VCT services
13. PMTCT treatment completion rate
14. PLWHA currently on ART
Resource Utilization 15. Trace drug availability (in stock)
16. OPD attendance per capita
17. Inpatient admission rate
18. Average length of stay (inpatient)
Table 7.1 Key Performance Areas and Indicators
INDICATOR DATA SOURCE PURPOSE
Inputs
Vaccine stockouts in a single month HMIS, Service statistics Monitor vaccine stock outs
(YES/NO, by type of vaccine)
Vaccine stockouts in a single month HMIS, Service statistics Monitor contraceptive stock outs
(YES/NO, by type of vaccine)
Number of services providers trained in Training Records Monitor reach of EPI/FP integration training as
provision of EPI/FP integrated services an input for effective integrated service
delivery
Table 7.2. Key Performance Areas and Indicators: Inputs
INDICATOR DATA SOURCE PURPOSE
Outputs
Number of service delivery points offering Service statistics and Supervision Coverage of integrated service
integrated FP and immunization services delivery
Number of days per month when both Service statistics and Supervision (Observation + Availability of co-located FP/
immunization and family planning services are interviews) immunization services
offered at the same site
Number/percent of women attending routine child Supplemental tracking column that can be added Quality/continuity of
immunization services who received information to existing immunization register [Monitored for implementation
on family planning from a vaccinator demonstration/ pilot program only] of integrated service delivery
Number/percent of women (with children <12 Supplemental tracking column added to FP ledger Quality/continuity of
months) going for family planning who receive [Monitored for demonstration/ pilot program only] implementation
information on immunization from of integrated service delivery
the family planning provider
Number/percent of women attending routine child Supplemental tracking column added to Acceptance of FP referrals
immunization services who accept a referral to immunization ledger [Monitored for provided
family planning services demonstration/pilot program only] by the vaccinator
Number/percent of women attending routine Comparison of supplemental tacking column added Follow through on FP referrals
immunization services who follow through on a FP to immunization ledger, and supplemental tracking provided by the vaccinator
referral from a vaccinator column added to FP ledger [Monitored for
demonstration/ pilot program only]
Number/percent of women attending family Comparison of supplemental tacking column added Follow through on
planning services who follow through on referral to FP ledger, and supplemental tracking column immunization
to immunization services from a family planning added to immunization ledger [Monitored for referrals provided by the family
provider demonstration/ pilot program only] planning provider
Table 7.2. Key Performance Areas and Indicators: Outputs
INDICATOR DATA SOURCE PURPOSE
Outcomes
Number of children receiving DTP1, Immunization ledger/HMIS, and Use of immunization services, dropout
DTP3, measles1, and DTP 1–3 dropout population-based survey data
Immunization coverage for DTP1, DTP3, HMIS and population-based survey Percentage of children <12 months in
measles1, and DTP 1–3 dropout data a given population who have received
DTP1, DTP3
Number of new family planning Family planning ledger/HMIS Uptake of family planning services
acceptors by method type and
demographic/age group
Contraceptive prevalence rate Population survey data Contraceptive use within a given
population
Total financial cost of inputs required to Program data/ Special costing studies Cost of inputs required for integration.
integrate FP and immunization services This may be helpful in planning for
(per facility, per client exposed, per new decisions related to sustainability and
FP acceptor) scale-up of integrated services
Impact
Maternal, infant, and child mortality Studies on maternal and infant Measure improvement in health
rates mortality status.
Table 7.2. Key Performance Areas and Indicators: Outcomes and Impact
INDICATOR ADDITIONAL PREFERRED SOURCE ALTERNATIVE
DIMENSION
Inputs and Processes
Health Financing
1 Total health expenditure as % of DGP NHA, PER
2 Total health expenditure per capita NHA, PER
3 % general government expenditure on health NHA, PER
Health Workforce
4 Doctors per 10,000 population Multiple
5 Nurse/midwives per 10,000 population Multiple
6 Graduates of health training institutions per Administrative records
10,000 population
Infrastructure and IT
7 Hospital beds per 10,000 population Administrative records
8 Doctors using electronic health records Facility assessment Clinic data
Procurement and Supplies
9 Tracer medicines availability Facility assessment
10 Median drug price ratio for tracer drugs Facility assessment
Table 7.2. Key Performance Areas and Indicators: Inputs and Processes
HEALTH SYSTEMS MONITORING INDICATORS AND DATA SOURCES
INDICATOR ADDITIONAL DIMENSION PREFERRED ALTERNATIVE
SOURCE
Outputs
Service Readiness and Access
1 Index of service readiness (combines Subnational; by specific intervention Facility
availability of infrastructure, human (IMCI, MCH, ART infrastructure, assessment
resources, medicines and equipment, human etc)
training)
2 Service accessibility (distance to facility) Subnational Survey Facility
assessment
3 General practitioner utilization rate Outpatient department utilization rate Clinic data Survey
Service Quality and Safety
4 TB treatment success rate (DOTS) Clinical data
5 30-day hospital case fatality rate AMI Hospital records
and stroke
6 Waiting time to elective surgeries: PTCA, hip replacement Hospital records
cataract
7 Surgical wound infections (% of all Hospital records
surgical operations)
8 Cancer treatment delay (time between Hospital records
first GP visit and first treatment, for
breast and colon cancer)
INDICATOR ADDITIONAL DIMENSION PREFERRED ALTERNATIVE
SOURCE
Outcomes
Coverage of Interventions
1 Antenatal care coverage (4+) Antenatal care coverage (1) Survey Clinic data
2 Skilled birth attendance Survey Clinic data
3 DPT3 Immunization coverage HiB, Hep3, measles, OPV Survey Clinic data
4 ART coverage Clinic data
5 Contraceptive prevalence rate Survey Clinic data
6 TB smear+ case detection rate Survey Notifications +
Model
7 ARI in under-fives taken to health facility Received antibiotics Survey
8 Diarrhoea in under-fives receiving ORT With continued feeding Survey
9 ITN coverage among children Pregnant women Survey
10 Cervical cancer screening (20–64 years) Breast cancer screening Survey Clinic data
coverage (50–69 years) coverage
Table 7.3. Health System Indicators and Data Sources: Outcomes:
Coverage of Intentions
INDICATOR ADDITIONAL DIMENSION PREFERRED ALTERNATIVE
SOURCE
Outcomes
Risk Factors and Behaviours
1 Tobacco use (adults) Youth (13–15), pregnant Survey
women
2 Access to safe water Urban rural Survey
3 Access to improved sanitation Urban rural Survey
4 Low birth weight newborns Clinical Data Survey
5 Breastfeeding exclusive for 6 months Initiation first hour Survey
6 Obesity in adults (over 15 years) Overweight Survey
7 Children under 5 anthropometry - Underweight, wasting, Survey
stunting overweight
8 Condom use at last higher risk sex, 15–24 years Survey
15–49
9 Particulate Matter (PM10) exposure Environmental
Impact Data
Table 7.3. Health System Indicators and Data Sources:
Outcomes: Outcomes: Risk Factors and Behaviors
INDICATOR ADDITIONAL DIMENSION PREFERRED ALTERNATIVE
SOURCE
Impact
1 Life expectancy at birth Life expectancy at age 65 Death registration
2 Child mortality (under-5) Neonatal, infant, perinatal Death registration Survey
3 Maternal mortality ratio Death registration Survey
4 Mortality by major cause of death 25 major causes of death, Death registration
ICD based
5 TB prevalence in population TB incidence Survey TB clinic data
6 HIV prevalence among adults 15–49 15–24 year olds Survey ANC survey
7 Notifiable diseases (IHR) Disease
surveillance
8 Depression prevalence (last 12 months)
Financial Risk Protection
1 Out of pocket as % of total health Catastrophic expenses NHA, PER
expenditure
2 Insurance coverage (% covered by public Survey
or private health insurance)
Table 7.3. Health System Indicators and Data Sources:
Impact
INDICATOR ADDITIONAL DIMENSION PREFERRED ALTERNATIVE
SOURCE
Impact
1 Life expectancy at birth Life expectancy at age 65 Death registration
2 Child mortality (under-5) Neonatal, infant, perinatal Death registration Survey
3 Maternal mortality ratio Death registration Survey
4 Mortality by major cause of death 25 major causes of death, Death registration
ICD based
5 TB prevalence in population TB incidence Survey TB clinic data
6 HIV prevalence among adults 15–49 15–24 year olds Survey ANC survey
7 Notifiable diseases (IHR) Disease
surveillance
8 Depression prevalence (last 12 months)
Financial Risk Protection
1 Out of pocket as % of total health Catastrophic expenses NHA, PER
expenditure
2 Insurance coverage (% covered by public Survey
or private health insurance)
Table 7.3. Health System Indicators and Data Sources:
Impact
RELATIONSHIP OF HMIS INDICATORS WITH HEALTH PROGRAMS
HMIS indicators have been carefully selected to meet the key information needs of monitoring the
performance of various health programs and services and provide a snapshot of the available health
resources.
The disease data provide the status report on communicable and non-communicable diseases. The
following sections illustrate the relationship of HMIS information and some of the health programs.
The purpose of these illustrations is to provide an in-depth understanding of how HMIS can be used for
monitoring program performance and how it encourages similar in-depth analysis for all health programs and
services such as: Maternal Survival Intervention, Child Mortality and Child Survival Intervention, and STOP TB
Program .
The Maternal Survival Strategy and HMIS Indicators
The Maternal Survival Strategies lay down a framework for achieving the fifth Millennium
Development Goal of reducing maternal mortality.
In order to routinely monitor the progress towards implementation of a highly effective package
of maternal survival interventions, the HMIS is designed to provide albeit some of the core input,
process, and output indicators.
Figure 7.3. Maternal Survival Strategies
Child Mortality and Child survival interventions
The EDHS 2011 estimated under-5 mortalities to be 88 per 1000 LB, that is, a 47% decline from
166/1000 LB in 2000. Diarrhea, pneumonia, measles, malaria, HIV/AIDS, birth asphyxia, preterm
delivery, neonatal tetanus, and neonatal sepsis are the major causes of under-5 deaths in Ethiopia,
with under-nutrition attributing to over one third of these deaths.
FIGURE 7.4. Causes of under-five deaths and mortality rate
HMIS Indicators to monitor STOP TB Program:
• TB patients on DOTS
• Number of new-smear pulmonary TB cases enrolled in the
cohort
• TB Case Detection
The STOP TB Program
• Number of new smear positive pulmonary TB cases
detected
With the vision to have a TB-free world, • Number of new smear negative pulmonary TB cases
the goal of the STOP TB Program (STP) is to detected
dramatically reduce the global burden of TB by • Number of new extra pulmonary TB cases detected
2015, in line with the Millennium Development • HIV – TB Co-infection
Goals and the Stop TB Partnership targets of • Proportion of newly diagnosed TB cases to HIV
the World Health Organization (2006). • HIV+ new TB patients enrolled in DOTS
• TB Treatment outcome
• Treatment completed PTB+
• Cured PTB+, Defaulted PTB+, Deaths PTB+
Figure 7.5. Routine Monitoring data on STOP TB program captured through HMIS
KEY POINTS TO REMEMBER
� The primary aim is to have a strong M & E and review system in place for the national health strategic
plan that comprises all major disease programs and health systems.
� Monitoring and evaluation (M & E) is a core component of current efforts to scale up for better health.
Global partners and countries have developed a general framework for M & E of health system
strengthening (HSS).
� There are different HMIS indicators which can be used for monitoring of key aspects of the health
system performance. These are from among the five broad categories – Reproductive Health,
Immunization, Disease Prevention and Control, Resources Utilization, and Data Quality
� HMIS is a source of routine data that is necessary for monitoring different aspects of various health
programs implemented in the country. HMIS indicators have been carefully selected to meet the key
information needs of monitoring the performance of various health programs and services and provide
a snapshot of the available health resources.
REFERENCES
Campbell, O., & Graham, W. (2006). Strategies for Reducing Maternal Mortality: Getting on with What
Works. The Lancet, 368(9543): 1284–99.
Ethiopia Maternal and Child Health Data. (2012). Countdown to 2015: Maternal, Newborn and Child
Survival. http://www.countdown2015mnch.org/documents/2012Report/2012/2012_Ethiopia
FP/Immunization Integration Working Group. (n.d.). Key Considerations for Monitoring and Evaluating
Family Planning (FP) and Immunization Integration Activities.
https://www.k4health.org/sites/default/files/FP%20Immunization%20Monitoring%20and%20Evaluatio
n%20 Briefer_0.pdf
National Health Mission. (2014). Monitoring and Evaluation. https://nrhm.gujarat.gov.in/monitoring-
evaluation.htm
USAID. (2013). HMIS Information Use Training Manual.
World Health Organization. (2006). Stop TB Strategy. http://www.who.int/tb/strategy/stop_tb_strategy/en/
World Health Organization. (2009). Monitoring and evaluation of health systems strengthening: an
operational framework.
World Health Organization. (2010). M & E of National Health Plans & Strategies.
