Our evaluation helped to understand the effectiveness of approaches to strengthen sub-national health systems through supporting district health planning and management
The District Health System Strengthening Initiative (DHSSi), was funded under a grant agreement between UNICEF’s Eastern and Southern Africa Regional Office (ESARO) and the Bill & Melinda Gates Foundation. It supported sub-national health systems strengthening across 24 pilot districts in Kenya, Malawi, Tanzania, and Uganda through strengthening district planning and management. OPM conducted an evaluation of this initiative for UNICEF ESARO. The purpose of the evaluation was to identify lessons that could inform ongoing DHSSi implementation, refine future programming and determine opportunities and challenges for sustainability and scale up.
Efforts to scale up delivery of interventions to tackle high levels of child and maternal mortality in Eastern and Southern Africa have been hindered by weak health systems. One area of weakness relates to health management and governance, particularly at sub-national level. Challenges include dearth of personnel with management experience within sub-national health management teams; gaps in management practice when skills and experience exist; insufficient use of evidence-based planning and inhibiting environments for good management practice.
The overall aim of DHSSi was to improve management practices at the sub-national level, and to promote scale-up of effective approaches in each country. These changes were in turn expected to contribute towards a reduction in health system bottlenecks, and ultimately an improvement in the coverage of high-impact health interventions.
Our evaluation covered DHSSi implementation over 2019-2022, through three annual evaluation rounds. It used a theory-based approach, and drew on aspects of contribution analysis and realist evaluation to answer a set of questions based on the Organisation for Economic Co-operation and Development (OECD) Development Assistance Committee (DAC) criteria.
The data collection methods included:
- Review of district, national and programme documents, including assessment of district health plans;
- Interviews with key stakeholders at district, national and cross-country levels, and group discussions with district health teams;
- Observation of DHSSi workshops and district planning activities;
- Using a Planning and Management Assessment Tool (PAMAT) designed by the evaluation team to assess district planning and management practice; and
- Analysis of secondary quantitative data sources, including Health Management Information System (HMIS) data on population-level coverage of key maternal and child health interventions, and programme monitoring information on reduction of priority health system bottlenecks selected by districts through their planning processes.
Outcomes and wider impacts
The evaluation provided robust evidence to help UNICEF, Ministries of Health (MoHs), the Bill & Melinda Gates Foundation and other partners understand the effectiveness of approaches to support district health planning and management, and ways to strengthen programme design. This included information on the strengths and weaknesses of different implementation models, key aspects of context that affected programme outcomes, and other areas where support was needed to enable effective subnational management. UNICEF adapted programme approaches each year following evaluation findings, and the evaluation also informed plans for future programming following DHSSi. The findings can also inform other interventions aimed at strengthening sub-national planning and management, and contribute to the wider evidence base on subnational health governance, an area with relatively limited research.