Evaluating the Bihar Technical Support Programme

Improving reproductive, maternal, newborn, and child health and nutrition in India

To improve service delivery and better reproductive, maternal, newborn, and child health and nutrition (RMNCH+N) outcomes in Bihar, India, CARE have implemented the Bill & Melinda Gates Foundation-funded Bihar Technical Support Programme (BTSP) over four years. This programme provides techno-managerial support to strengthen the Department of Social Welfare and the Department of Health and Family Welfare of the Government of Bihar, and the public health and nutrition systems that they manage. From 2015-2018, we conducted an evaluation of the programme.

Challenges

The evaluation was complex, given the broad nature of the project (meaning no counterfactual is possible) and the multiple scales at which the project is operating (policy, systems, service delivery). The evaluation was focused on the following questions:

  • Effectiveness of systems strengthening for RMNCH+N

Has the BTSP’s techno-managerial support to the state been effective in improving the systems strength, debottlenecking barriers, and improving the performance of the health and nutrition systems? Particularly, did it improve these systems in the areas of supply chains, finance and budget expenditures, service delivery, human resources, and leadership and management? What has worked and why?

  • Improvements in quality of care

Has the BTSP’s techno-managerial support on quality of care including quality improvement and nurse mentoring and been effective in improving quality of maternal, newborn, and child health and family planning services?

  • Sustainability

To what extent are the activities and results achieved by BTSP likely to be sustained after the end of the programme?

  • Delivering on its mandate

Finally, how has the BTSP delivered on its mandate to provide reflexive, responsive techno-managerial support to the Government of Bihar and act as lead RMNCH+N partner?

Our approach

The methodology of the evaluation includes the following:

  • We developed and operationalised a method of measuring the strength of the public health and nutrition ‘systems’ by breaking them down into component parts. These were based on the WHO building blocks model (leadership and management, governance, supply chains, infrastructure, data, human resources, finance) at different levels (state, district, block, outreach).
  • We used complex multi-level modelling to understand the impact of BTSP interventions on the health and nutrition system component parts (and their inter-relationships), as well as diagnostics for what needs to be done to further improve system functioning.
  • We linked changes in system strength to service delivery at facility and outreach levels, both through assessing service availability and readiness and through direct observations of service delivery (focused on delivery care and family planning).
  • We also linked changes to the supply side with population level outcomes through household survey data. These were linked with analysis of complementary interventions at the demand side (e.g. outreach counselling) and governance/accountability (e.g. the use of self help group platforms to deliver behavioural change communication as well as social accountability roles). Together, we were able to develop a holistic model of how changes in desired outcomes occur.

Outcomes

The evaluation will feed into the design of future programmes of support to the Government of Bihar, as well as generate operational learning for the Government as to how it can work through its health system to achieve its 2030 goals.


Image credit: Travel Stock / Shutterstock.com

Area of expertise