Evaluating and learning from the Mitigation Action Facility

The Mitigation Action Facility funds ambitious climate change mitigation projects to implement Nationally Determined Contributions (NDCs) and long-term strategies (LTS) that are central to meet global international agreements. The multi-donor Facility primarily focuses on three priority sectors – energy, transport and industry – and provides grant-based funding to innovative projects that can catalyse sector-wide transformational change.

We are providing long-term support to the Mitigation Action Facility for evaluation and learning services for the national projects it is funding.  This includes developing the theoretical framework for project level ‘Evaluation and Learning Exercises’ (ELE) and using this for 17 mid-term and 20 final ELEs. In addition, we are producing a series of Learning Studies to carry out a type of meta-analysis across the project level learning insights to inform the overall strategic direction of the Facility.

Challenges

The Mitigation Action Facility is set-up to enable transformative national action to avoid further negative impacts from climate change. The projects are expected to catalyse change in systems and behaviours through disruptive climate actions. Measuring transformational change and then evaluating the relative performance of national projects is not straightforward. We therefore developed a ‘transformational change framework’ to guide the ELEs. This rests around a set of project results that interact and reinforce each other to enable transformational change, including:  

• Promoted a demonstration effect, such as proving the viability and benefits of a particular mitigation ‘solution’ such as an approach, technology or practice (e.g. low-carbon coffee production) through pilots.  

• Causing a catalytic effect, to amplify the impact of the mitigation solution by replication and scaling-up to other locations and sectors and  

• Enabling systemic change that unblocks barriers and creates new incentives to promote wider uptake of the mitigation solution (e.g. through market or policy reform).  

These project results are ultimately expected to contribute to additional, large-scale and sustained GHG savings. This might not be achieved within the lifetime of the project, but there should be clear signals that this will happen in the near future.

Expertise

The Evaluation and Learning Exercises (ELEs) are based on a theoretical framework that involves a document review, participatory workshops and stakeholder interviews to collect evidence about projects’ results and lessons. These elements are then analysed using a theory-based approach centred on the use of contribution analysis and reinforced by elements of process tracing. The ELEs seek to address the following questions: 

• Has the project achieved its result? 

• Has the project triggered transformational change?  

• What can be learnt from the project?  

Impact

Some examples of the projects that we have evaluated include:  

• Indonesia Sustainable Urban Transport Project (Final ELE) 

• Colombia Transport-Oriented Development Project (Mid-term ELE of Financial Component and Final ELE of Technical Component)  

• Peru Sustainable Urban Transport Project (Final ELE) 

• Colombia Domestic Refrigeration Project (Mid-term ELE) 

• Mexico New Housing Project (Final ELE) 

• Thai Rice Project (Mid-term ELE) 

• Thai Refrigeration and Air Conditioning Project (Final ELE)   

• Chile Self-Supply Renewable Energy Project (Final ELE) 

• China Integrated Waste Management Project (Mid-term ELE)   

• Costa Rica Low-Carbon Coffee Project (Final ELE)  

We use the experience of carrying out these individual project level ELEs to draw out overarching findings across the portfolio. The First Learning Study considered the findings of the first nine ELEs conducted to explore the question: What are the different pathways projects pursued towards transformational change and how successful have they been in contributing to such change? The findings were discussed by the Facility donors and secretariat and used to refine the transformational change framework and the Facility’s wider Knowledge Creation Strategy. Future learning studies will pursue additional lines of enquiry and will include Qualitative Contribution Analysis as the sample size of projects increases.