Data revolution: Transforming policy making with social media insights

Access to the internet and social media platforms has transformed how people acquire and engage with information. By easing different barriers, it has facilitated inclusive conversations on various topics amongst a range of stakeholders. Can we harness publicly accessible social media data to provide actionable insights for policymakers? What is the role and what are the limitations of such analysis vis-à-vis traditional development evaluations?

Social media listening (SML) tools refer to the process of collating large volumes of publicly accessible social media data to track conversations and analyse mentions, hashtags and keywords related to a topic or programme. For us in international development, this can promote a better understanding of diverse public sentiments, which informs inclusive policy. SML can simultaneously have nefarious uses which can adversely affect stakeholder interests that need safeguarding. In this blog post, we present a case study on the use of SML tools to inform the inclusive education landscape in Kazakhstan. 

Applicability to international development

In recent years, traditional research methods have been complemented with SML studies to better understand stakeholder perspectives, public sentiment and real-time trends to improve policy responsiveness.  For example, UNICEF, through U-reports, has been engaging with youth around the world to create digital communities and lend them a voice on matters affecting their communities. Similarly, USAID’s Feed the Future initiative has been leveraging social media conversations on agriculture and food security to better understand the evolving challenges, concerns, and perspectives of smallholder farmers. The World Bank’s Listening to Africa (L2A) programme has been complementing large scale, resource intensive household surveys with innovative methods of data collection by integrating these with follow up mobile phone interviews to collect and analyse high frequency data on a range of welfare indicators. Similar programmes are being tested by the United Nations Development Programme (UNDP) to gain insights on language use and tackle gender stereotypes. 

Development organisations across the world are using SML tools to improve their existing monitoring and evaluation capabilities. Unlike traditional survey data, this organically generated SML data provides ample room for discussion and synthesis of different stakeholder perspectives. These are vital in identifying the local contexts, including social norms and behaviour, which are critical in the programme design phase. Moreover, as social media data is generated in real-time at high frequency, these are better suited to monitor the effects of unanticipated shocks, which are becoming more frequent with climate crises, and account for these in the programme rollout.  

Lessons from the inclusive education study in Kazakhstan

Kazakhstan is a young country with 34 percent of its population below the age of 18 years. UNICEF’s estimates reveal that the country has a sizeable and growing population of children with special education needs (SEN). Importantly, social media users account for close to 61 % of the country’s population, which made this an ideal setting for the use of SML tools to understand perspectives on, and challenges in, promoting inclusive education. The broad lessons from this study were as follows: 

• Timing and choice of broadcasting platform matters: Trends and topics in social media discussions vary across the year with different stakeholders preferring different social media mediums for their exchanges. For example, in Kazakhstan, discussions on inclusive education rise with the start of the schooling session and taper off gradually. The general public’s (parents, teachers, and students) engagement was higher on platforms like Facebook, whereas organisations working on inclusive education used Instagram and Telegram. This information may be used to relay key messages to different stakeholders.
• Leveraging anonymity: In-person surveys are sensitive to enumerators’ ability to make respondents feel comfortable to openly speak their mind. The possibility to anonymously post experiences on social media, in contrast, can offer insights on topics that are often difficult to navigate in face-to-face interviews. In our study, we found significant discussions on the lack of child protection and instances of abuse and violence in homes and school against children with SEN. 
• Real time feedback for policy makers: Unlike traditional surveys where transmission of information entails significant lag, analysis of social media data can provide quick feedback and course correction. In the Kazakhstan study, we found positive sentiment on initiatives to support children with autism, prevent disability in newborns, and increase the capacity building of teachers through the introduction of modern technologies. 

Limitations 

As is the case with other research methods, there are methodological limitations of SML tools that we need to be aware of. For representativeness, it is important to study perceptions from a diverse audience to ensure a voice for all stakeholders, irrespective of their geographic locations or socio-economic backgrounds. In this context, social media may give a disproportionately higher voice to those who are economically better off and residing in larger cities with superior internet connectivity, neglecting others whose perceptions and constraints might be completely different. In our study, we found that most of the conversations were from the larger cities of Astana and Almaty.

 Importantly, research shows that SML studies may be biased due to the more extreme views expressed on social media than via in-person surveys. Moreover, increased use of bots and trolls can dissuade people from sharing their views or provide platforms exclusively to users with a certain profile. This highlights the need for expert discussions to validate the relevance of online posts and complement SML studies with traditional methods of research.

Final considerations

Our experience highlights the potential of SML tools in enriching the design of policy programmes. In particular, they provide real-time insights into different stakeholder perspectives and shed light on local factors such as the prevailing socio-cultural norms, which can be particularly useful in the inception phase of the programme design. As SML tools lend a voice to different stakeholders for whom these policies are designed, policymakers and development practitioners need to consider the feasibility of incorporating SML tools into the programme design. However, they simultaneously need to be aware of the methodological limitations of SML tools and why they should complement but not replace traditional research methods.   

Photo by Victoriano Izquierdo on Unsplash.