Data quality review of a Green Prosperity project

The Green Prosperity (GP) project is a high-value project funded by MCA-Indonesia

Project team members

Sustainable development programming at the local level can be catalysed, in Indonesia, through improving land use practices, appropriate managing of natural resources, and expanding renewable energy. The Green Prosperity (GP) project is a high-value project funded by MCA-Indonesia with this aim.

In order to measure the progress and impact of the GP project, MCA-Indonesia have developed a monitoring and evaluation framework that includes an indicator tracking table (ITT). However, these indicators are based on a range of data sources that rely on the systems and data processing of implementing agencies based in a range of geographically dispersed locations, without any certainty that robust data verification and quality assurance regimes are in place.

We provided a full review and recommendations to assess data quality issues and advise on how adequate data management systems should be put in place, including via learning requirements.

Challenges

Many implementing agencies were involved in the GP project, in many different locations, and so it was necessary to plan and prioritise data collection and analysis for the review according to the complexity, significance, and priority of ITT indicators. In addition, different data sources require different treatments; some are complex and others more straightforward.

The work plan was therefore developed by identifying a representative and adequate sample to reflect these considerations. This also had to be dynamic in response to emerging findings, particularly in terms of the complexity and potential quality issues that need to be addressed. The aim was to ensure that the input data in the scoring system from the questionnaires was adequately weighted in favour of giving reliable ratings for individual indicators, and providing higher levels of confidence in the results for high-priority indicators.

Our approach

We adapted the assessment criteria (data quality dimensions) to be used as the basis for a data quality assessment framework (DQAF) and the development of the tools to be used in the DQR. This included increasing the emphasis on assessing the institutional environment and other issues that seem important for ensuring the data are of high quality and fit-for-purpose. The data quality dimensions covered by the DQAF focused on:

  • validity
  • accuracy and precision;
  • adequacy/comprehensiveness;
  • punctuality;
  • reliability;
  • objectivity and integrity; and
  • practicality and usability.

We used a diagnostic approach to map the data flows, identify all the elements of the data quality dimensions mentioned above, and prepare the metadata.

The steps in our data quality assessment are:

  1. review of all documentation, metadata, and reports relating to each statistical series;
  2. inspection of the raw data to check for accuracy, completeness, and consistency;
  3. interviews with key personnel in the data production chain, or the production process;
  4. interviews with key data users; and
  5. where applicable and available, scrutiny of related external data to check for consistency of estimates and coherence of results.

Outcomes

Our finding sets out the DQR’s conclusions at the programme and indicator levels and provides some related recommendations, from a statistical perspective, to inform future technical and system-wide improvements in the use of similar indicators. In addition to recommendations for individual indicators, we provided operational and strategic recommendations, as well as those relating to the indicators for forward tracking, as the project was closing at the time the review was conducted. In addition, our report detailed the significant improvements required to ensure that the indicators meet the international standards for data quality. This will underpin future effective monitoring of policy implementation, financial investments, and the development of effective interventions to achieve the objectives of the GP project or related future development goals.

Areas of expertise