We were contracted by the Millennium Challenge Account (MCA) Zambia to conduct a data quality audit (DQA), which was focused on reviewing the quality of data collected and reported in the Zambia Compact monitoring and evaluation (M&E) framework for the period 2014–2017 and 2018.

The scope of the work included ensuring that data collected in relation to the M&E indicators is of sufficient quality, following up on the recommendations of the DQR (conducted in 2016) on data collection, collation, and analysis and the implementation of these recommendations, and ensure that common formats are used for data collection, collation, and analysis.

Our approach

A sample of indicators were selected for the DQA covering indicators related to the Lusaka Water and Sanitation Company, the Innovation Grant Programme Manager Cowi, Cowater and the construction supervising engineers. The DQA involved the mapping of the ideal data flow process using a data quality diagnostic tool for evaluating the indicators in the M&E plan and the data contained in the indicator tracking table, across the data cycle from collection, forms for capturing data, recording, verification, analysis, dissemination, security, and archiving.

Using key informant interviews and observations to determine how the data is collected in practice, and the challenges faced in the data collection process, the team were able to understand the flow of data form data collection to reporting in the indicator tracking table. In order to understand the quality of data, an analysis of the datasets was also undertaken focusing on outliers, inconsistencies, labelling, sufficient documentation and metadata.

Following the fieldwork and data analysis, an assessment was made for each indicator against relevant criterion in the form of a confidence rating (high, medium, or low quality, or  unknown quality if evidence was not sufficient to make an assessment).

Outcomes

The post-compact M&E plan being drafted by the MCA-Zambia incorporates the recommendations of the DQA and will chart the path for the GOZ and implementing agencies to continue to invest in good quality data. The project also helped build capacity for some of the implementing agencies, especially smaller firms and NGOs, through making them more aware about the requirements of data quality – such as the need to document all processes, concepts, and definitions.