Evaluating a national health programme in Bangladesh

UNICEF Bangladesh, in collaboration of the Ministry of Health and Family Welfare (MoHFW), Government of Bangladesh implemented country-wide health programme to address challenges linked to global health priorities and targets. The programme focused on three main areas:

1. health service interventions across the country focused on the district health system, including maternal and neonatal health, adolescent health, and Extended Programme on Immunisation (EPI);
2. health system strengthening focused on community health, Health Management Information System (HMIS), District Evidence-Based Planning and Budgeting (DEPB), and Leadership Development Programme (LDP); and
3. humanitarian support mainly focused on the Chittagong hill tract areas

Programme activities were implemented in partnership with the Government of Bangladesh (GoB) and other national stakeholders. The health programme built on the UNICEF 2012-2016 country programme and focused on the new programme cycle as defined in the 2017-2020 country programme.

Our evaluation focused on the UNICEF programme’s overall progress in achieving its overarching objectives over the first two years of implementation (2017-2018). The evaluation focused on the current country programme (2017-2020) but in some cases examined linkages with the previous programme (2012-2016), and links to cross-sectoral support such as water, sanitation, and hygiene (WASH), and disability. The evaluation has followed the OECD/DAC evaluation criteria of relevance, coherence, effectiveness, efficiency, impact, connectedness, and sustainability.

Challenges

As primary data was not collected before the programme began, there was no control data to allow the evaluation to consider the counterfactual (i.e. what would have happened in the absence of this programme). The focus was on the UNICEF-GoB programme of support, not the entire GoB sector programme. Attribution was difficult to impute, given the joint implementation with GoB and other programmes, so the evaluation primarily focused on UNICEF-GoB contribution to the results. Moreover, resource constraints limited fieldwork to six districts and one city corporation.

Our approach

A mixed-method design was adopted for the evaluation. This design incorporated six components to address 33 evaluation questions outlined in the Terms of Reference. The qualitative methods included a desk review, focus group discussions, and key informant interviews. The quantitative methods included secondary data analysis, observations of health facilities, consultations and community group meetings and an exit survey of end-beneficiaries.

Fieldwork was conducted over two months in six districts, one city corporation and at the national level. Data was analysed by evaluation areas, and quantitative and qualitative data were presented simultaneously in the report. Results were presented to UNICEF and MoHFW officials.

Outcomes

Study findings suggest greater need for focus on the following four areas:

1) Evidence – the way in which evidence is assembled and compared with other programmes and communicated with stakeholders;

2) Indicators – ensuring that indicators, both physical and financial, are appropriately disaggregated across areas of the country and over time to allow sensitive assessment of programme impact;

3) Monitoring the quality of care given so that assessment of impact can also look at the content of services provided to the population; and,

4) Engagement with the private sector – to develop engagement with a rapidly changing health system with a plurality of providers.

The mid-term assessment of the health sector programme suggests many evident strengths that increase the likelihood of programme’s impact on the sector. We have also identified a number of areas for improvement: around generation of evidence, improving linkages particularly with other partners, and ensuring that information is collected and that facilitates improve monitoring and evaluation.

Photo: Unsplash