How to measure governance: a new assessment tool

 “Better governance” is frequently hailed as a fundamental prerequisite for sustainable development. SDG Goal 16 and the World Bank’s World Development Report 2017 are prominent examples. Unfortunately, a successful improvement of governance structures (for example government systems, processes, and policies) takes a long time to materialise, and is notoriously difficult to measure - especially within the life of a short to medium-term focused reform project. This is a big problem because it makes it difficult for governments, NGOs and donors to know whether reform efforts are working and are likely to be sustained. It also makes it difficult for aid agencies to justify spending on governance compared to more easily measurable things such as paying for medicine, roads or mosquito nets.

What do we mean by governance?

At its core, governance is about the formal and informal rules and processes that politicians, bureaucrats, business elites and citizens follow when they interact, and design and implement policies. Improving governance therefore entails changes to the structures through which government works. This could be changes in how civil servants are recruited and promoted, or in how government agencies decide where and how to spend money. It also includes things such as the procedures for how businesses or citizens are involved in a ministry’s decision making processes (such as rules around public consultation) or the ways in which education or health services are delivered.

Why governance is difficult to measure

The challenge for those of us working to improve governance systems, is that it can be really difficult to know whether we’re having an impact or if we’re on track to do so. Are interventions helping to develop capacity and new systems? Are norms and established practices changing? Will these results be sustained and in future lead to improvements in how a ministry and its staff actually perform? Will these results produce tangible benefits for citizens in the forms of improved services and wellbeing? For more on the challenges of measuring governance have a look at contributions from ODI, the World Bank and Robert Rotberg who helped develop the Index of African Governance (IIAG).

Over the years, donors, academics, and NGOs, have designed a plethora of indices intended to measure governance and allow for comparisons between countries. Examples include the Worldwide Governance Indicators, Varieties of Democracy Dataset, Rule of Law Index, Quality of Government Dataset, Social Institutions and Gender Index, IIAG, and many others. While helpful, these datasets focus on an aggregate national level and are often only updated annually. Most of the time, they are also unable to tell us whether specific interventions are likely to lead to sustained and longer-term improvements in governance.

The solution: a new assessment tool

To address this challenge, we have developed a governance results assessment tool. The tool can be used by reform champions, donors and monitoring and evaluation personnel to help measure whether an intervention (or set of interventions) are likely to produce sustained improvements in governance structures and whether these help improve how government works and how services are delivered to citizens.

We developed the tool under the Action on Climate Today (ACT) programme (see here for a collection of our learning products). ACT was a large five-year programme funded by the British Department for International Development. It worked with governments in Afghanistan, Bangladesh, India, Nepal, and Pakistan to help improve climate change resilience by helping governments to incorporate climate change considerations into policy, planning, and investment environments.

The programme worked on a range of issues that involved changes in government systems and processes in relation to areas such as flood forecasting, agriculture and climate finance. Under these areas, the programme helped develop new systems and processes, organisational units, and decision-making tools.

The governance assessment tool helped us monitor and verify whether these results (outputs) were being produced and whether they were actually used or likely to be used (outcomes) to improve climate change resilience (impact). To do this, the tool covered a number of different questions to be answered by our programme staff overseeing a given intervention as well as government counterparts.

These questions concerned issues that either indicated sustainable change or influenced the likelihood that such change would materialise, including:

• whether an output (such as a new budgeting tool) had been formally adopted by relevant government departments;
• the degree of senior-level commitment to implementation (i.e. use of the output) captured on a scale from 1-5;
• whether a new output (such as a new flood forecasting tool) was being used by government departments and/or the likelihood of it (continuing) being used in the future;
• whether the relevant government department had the capacity or external support needed to implement the output effectively;
• whether adequate financial resources were (and would continue to be) in place for the output to be implemented effectively;
• what results government reform champions expected to see after an output had been effectively implemented and when these were likely to materialise; and
• In order to capture the transformational potential of the interventions, we also looked at whether outputs were being replicated in other sectors or regions, or integrated into wider policies, laws or regulations in the country.

To measure these attributes, and to back up evaluative judgements on questions such as the degree of commitment, ACT programme staff drew on several types of evidence, including formal documents (such as Government Gazettes, letters and meeting minutes), project documents and interviews with government counterparts. These various pieces of evidence were compiled and referenced in the tool, and the finished product was reviewed, interrogated and approved by ACT’s governance advisors.

Implications

The governance results assessment tool allowed us to measure improvements in governance that are usually very hard to measure. It allowed us to capture results with much greater rigour and certainty, and to gauge whether the programme’s intended improvements in climate change governance were actually materialising and/or whether they were likely to appear and be sustained after the life of the programme.

This was useful not only as a way to measure results, but also as a method for spotting risks and factors that could compromise sustainability of results. For instance, in Afghanistan the tool helped our team identify and distinguish capacity gaps relating to natural resource management at the provincial and district levels from those in the central government in Kabul. Combined, these insights allowed us to design and deliver support that was as useful as possible for government reformers in South Asia that work on the front lines of climate change adaptation.