Assessing and enhancing government data quality from theory to practice

Making the best use of available data

One of the challenges faced in development is that government administrative data can be of mixed quality. It is thus of heightened importance to understand how to interpret the data that are available. Thus, the question is: which aspects of these data can be used and for what purposes? A related question is whether there are ways to strengthen local capacity to ensure quality data.

This working paper proposes a 10-step approach to conducting assessments of the quality of government administrative data across multiple sectors. Its application is illustrated through real-world examples. The exercise ended in collaboratively strengthening the capacity of local stakeholders. One of the participants declared at the end of a training that ‘my Rome started to be built today,’ implying that an important foundation of skills had successfully been laid.

We are not the first to find ways to make use of imperfect administrative data. However, we think that we can add valuable insights from our experience of doing so over a period of years.

Getting to an approach that works in practice

The 10-step approach was developed over several years of working on M&E-related aspects of the Millennium Challenge Corporation-funded support to the Government of the Republic of Namibia. As part of this work, the quality of the government-sourced data that were used for project monitoring was assessed. The assessment was done by means of data quality reviews (DQRs) of multiple sources of data across multiple sectors.

Through an organic process that was subsequently refined, the DQRs went from asking whether the data sources were ‘fit for purpose’ to a more nuanced judgement of establishing ‘fit for what purpose?’ Essentially, the DQRs sought to answer the following three questions:

1. ‘For what purpose do we want to use the data?’
2. ‘What does the process of producing the data tell us about their fitness for the specified purpose (and for other purposes)?’
3. ‘For what purpose(s) are the data fit, and what actionable steps could be taken in this context to improve on the data’s overall quality?’

The 10-step approach to assessing and enhancing data quality

The approach is summarized in the following 10-step framework. It can be applied across a variety of data collection methods and data sources, particularly in contexts where resource, capacity, and/or circumstantial (such as the current global pandemic) constraints challenge the ability to systemically adhere to strict data quality frameworks.

The three phases in which the steps fall correspond with the three overarching questions that the DQRs sought to answer.

Phase I: Preparatory and planning work – ‘For what purpose do we want to use the data?’

• Step 1: Clarify agenda for DQA and establish the purpose for which the data are being used
• Step 2: Clarify definitions
• Step 3: Review documentation on the data’s creation and use
• Step 4 (iterative): Formulate guiding questions

Phase II: Embarking on a ‘voyage of discovery’ – ‘What does the process of producing the data tell us about their fitness for the specified purpose (and for other purposes)?’

• Step 5: Consult data producers to understand the data collection process and the context(s) in which it is carried out
• Step 6: Observe the data collection process in action
• Step 7: Test the data
• Step 8: Cross-check understanding and, if necessary, conduct follow-up consultations

Phase III: Drawing conclusions and developing recommendations – ‘For what purpose(s) are the data fit, and what actionable steps could be taken in this context to improve on the data’s overall quality?’

• Step 9: Draw conclusions and identify actionable recommendations
• Step 10: Begin the process of strengthening local DQA capacity in line with local priorities and drawing on existing capabilities

The promise of the 10-step approach

We found that the 10-step approach to DQRs was able to provide results with an effort proportionate to the task and adapted to the reality of data collection on the ground. At the same time, it aimed to strengthen local capacity in a collaborative fashion. The original version of the approach was subsequently incorporated into Namibia’s national M&E system. We hope that it thus contributes to broader efforts towards country-led development and inclusive M&E.