Using an experimental framework we have studied the impact of mobile health initiatives.
BBC Media Action is implementing two large scale mobile health (mHealth) initiatives supported by the Government of India: Kilkari, and Mobile Academy. Kilkari is an outbound service that delivers weekly, time-appropriate audio messages about pregnancy, childbirth, and childcare directly to families on their mobile phones, starting from the second trimester of pregnancy until the child is one-year-old. Mobile Academy is a mobile-based training course for the Accredited Social Health Activist (ASHA) community health workers. ASHAs access the course, offering four hours of audio content, by dialling a number from phone. We have evaluated impact of these two initiatives using an experimental framework.
We partnered with Johns Hopkins Bloomberg School of Public Health to provide inputs in the evaluation method, assignment of treatment and control, data collection and data analysis towards assessing the impact of the Kilkari and Mobile Academy programmes for the state of Madhya Pradesh in central India.
During implementation of fieldwork, the listing exercise had to be expanded substantially to reach the desired number of sample, especially as the sample included pregnant women with 12-34 weeks of gestation, above the age of 18 years, could speak and understand Hindi and had access to a mobile phone during daytime with a private telecom service provider connection. We also increased listing team size and extended timeline.
The evaluation design followed an individually randomised controlled trial (RCT) with parallel and unblinded assignment of treatment and control groups. Under this study, we first collected data from around 3800 villages (covering >4,00,000 households) from four districts of rural Madhya Pradesh. A total of 5200 women were selected for the study as part of the RCT evaluation. During baseline, we collected data from 5200 pregnant women and their spouses in 2018. The ASHAs in the sampled villages were also interviewed. As a follow up survey, the same 5200 women, their spouses and ASHAs were revisited (as part of panel survey) in 2019-2020. The listing survey, baseline and endline were conducted using CAPI, which our in-house experts developed. In addition to that, we also collected qualitative data from the target study population. Our researchers collaborated with the JHU and BBC Media Action teams for the analysis focusing on evaluation of Kilkari and Mobile Academy programmes.
Both qualitative and quantitative analytical research work, have been published in peer reviewed journals focusing on the research protocol, usage of machine learning and mobile messages during data collection for data quality assurance, impact of the Kilkari and Mobile Academy programmes and the underlying pathways of the impact.
Some important references:
Are stage-based health information messages effective and good value for money in improving maternal newborn and child health outcomes in India? Protocol for an individually randomised controlled trial