Evaluating the UK Department of International Development's programme
Andrej Kveder Madhav Vaidyanathan Martina Garcia Aisa Patrick Ward Paul Jasper Sarah Keen Sope Otulana UK Department for International Development (DFID) Tafara Ngwaru
This evaluation project will improve understanding around the impact of DFID’s £72 million Maternal, New-born and Child Health (MNCH) Programme in Zimbabwe. Over the past decade, many health indicators in Zimbabwe have worsened as a result of lower public sector investment in health. DFID’s programme seeks to improve the coverage, quality and accessibility of MNCH services, and to increase the provision of anti-retroviral therapy in order to tackle AIDS-related illness and deaths. We are conducting an independent evaluation of the programme based around three workstreams: peer reviewing an evaluation of the programme’s Health Transition Fund, conducting an in-depth impact evaluation of the programme’s demand and accountability component and completing annual reviews for the overall programme.
The team will use primary and secondary research and will review outputs against defined standards and provide quality assurance. For the impact evaluation of the programme’s demand and accountability component, the team will implement a rigorous mixed-methods design incorporating health facility surveys and qualitative stakeholder interviews. The multi-year project will provide robust evidence to support ongoing programme refinements in order to maximise positive health outcomes for the country.
Recent political and socio-economic upheaval in Zimbabwe has led to lower public sector investment in health and reduced provision at all levels of the health system. As a result, health indicators in the country have worsened significantly, slowing the progress made since the country’s independence in 1980.
DFID’s MNCH programme seeks to improve the coverage, quality and accessibility of MNCH services, and to tackle AIDS-related illness and deaths through increased provision of anti-retroviral therapy. The MNCH programme has four key components:
- Contributing to the Health Transition Fund (HTF): a multi-donor fund that aims to revitalise Zimbabwe’s health system by supporting service delivery, human resources, commodities and decentralised planning and funding
- Providing anti-retroviral commodities
- Supporting paediatric ARV treatment
- Promoting demand and accountability in MNCH services by strengthening health centre committees
This project was established to provide an independent evaluation of the programme - including an in-depth assessment of the demand and accountability component - in order to provide evidence that will feed into programme improvements over the longer-term.
As part of the project, the team is using a mixed-methods approach to evaluate the demand and accountability component of the programme. This will include a rigorous impact evaluation based on surveys of over 140 health facilities and a series of qualitative interviews with community members and key MNCH stakeholders.
The team will also peer review the separate evaluation for the Health Transition Fund, the largest component of the MNCH programme. This will involve reviewing outputs against defined standards to assess whether DFID’s key evaluation questions are being answered credibly. Quality assurance will be provided through replication of results and assessments of robustness.
In addition, the team will conduct a series of independent reviews and updates at regular intervals for the MNCH programme as a whole. These will provide a key mechanism for DFID to monitor and manage the programme in an environment of considerable uncertainty, and will ensure accountability for programme delivery and project funds at its completion.
The team will complete the following activities:
- Undertaking a peer review of the HTF component evaluation – assessing outputs against defined standards and completing quality assurance
- Developing and implementing an impact evaluation design for the demand and accountability component
- Conducting a quantitative survey in over 140 health facilities at two points in time
- Conducting qualitative interviews with community members and key MNCH stakeholders
- Completing regular programme reviews and updates and producing a project completion report
By providing robust evidence, this independent evaluation will support the Government of Zimbabwe, DFID and other donors and implementing agencies (including UNICEF, Save the Children, John Snow Inc. and the Elizabeth Glaser Paediatric AIDS Foundation), to better understand the implementation and impact of the MNCH programme in Zimbabwe.
Through the peer review of the HTF evaluation and the annual reviews of the MNCH programme, we will directly contribute to the refinement and implementation of the programme, helping to leverage successful interventions and maximise their wider impact.
The in-depth evaluation of the accountability component will provide evidence on the impact of accountability interventions, which are increasingly seen as one potential solution to some of service delivery problems in the health sector.