Our evaluation will help understand the effectiveness of approaches to strengthen sub-national health systems through supporting district health planning and management.
The District Health System Strengthening Initiative (DHSSi), which is funded under a grant agreement between UNICEF’s Eastern and Southern Africa Regional Office (ESARO) and the Bill and Melinda Gates Foundation, supports sub-national health systems strengthening across 24 pilot districts in Kenya, Malawi, Tanzania, and Uganda through strengthening district planning and management. We are conducting an evaluation of this initiative for UNICEF ESARO. The purpose of the evaluation is to identify lessons that can inform ongoing DHSSi implementation and refine future programming. Being a multi-year programme, the evaluation is also designed to assess long-term sustainability and identify opportunities and challenges for scaling up the approaches more widely.
Efforts to scale up delivery of interventions to tackle high levels of child and maternal mortality in Eastern and Southern Africa have been hindered by weak health systems. One area of weakness relates to health management and governance, particularly at sub-national level. Challenges include dearth of personnel with management experience within sub-national health management teams; gaps in management practice when skills and experience exist; insufficient use of evidence-based planning and inhibiting environments for good management practice.
The overall aim of DHSSi is to improve management practices at the sub-national level, and to promote scale-up of effective approaches in each country. These changes are in turn expected to contribute towards a reduction in health system bottlenecks, and ultimately an improvement in the coverage of high-impact health interventions.
Our evaluation covers Phase I of DHSSi implementation and takes place in three annual rounds. It uses a theory-based approach and draws on aspects of contribution analysis and realist evaluation to answer a set of questions based on the Organisation for Economic Co-operation and Development (OECD) Development Assistance Committee (DAC) criteria.
The data collection methods include:
- Review of district, national and programme documents;
- Interviews at district, national and cross-country levels, and group discussions with district teams;
- Using a Planning and Management Assessment Tool (PAMAT) designed by the evaluation team to score district planning and management practice across seven domains; and
- Analysis of secondary quantitative data sources, including Health Management Information System (HMIS) data on population-level coverage of key maternal and child health interventions, and programme monitoring information on reduction of priority health system bottlenecks selected by districts through their planning processes.
Outcomes and wider impacts
The evaluation will provide robust evidence to help UNICEF, Ministries of Health (MoHs), and partners understand the effectiveness of approaches to support district health planning and management, and ways to strengthen current programme design. Importantly, the evaluation approach will help to assess the effectiveness of this initiative, processes involved, and key aspects of context that affect programme outcomes. As a multi-year evaluation, annual findings will directly support ongoing programme decisions. The findings will also support decisions by UNICEF and others on future implementation and scale up. Over the longer term, the evaluation can inform other interventions aimed at strengthening sub-national planning and management. The evaluation will also contribute to the evidence base on approaches to strengthening district health system management, as this is an area with limited research.
The primary expected users of the evaluation findings are UNICEF, MoHs, and the Bill and Melinda Gates Foundation. Other key audiences include other government partners in each country, and other institutions working on district planning, management and governance in the Eastern and Southern Africa region or more broadly.