Monitoring, Evaluation and Learning for the Asia Regional Resilience to a Changing Climate programme

The Asia Regional Resilience to a Changing Climate (ARRCC) programme aimed to increase the resilience of vulnerable groups and of economic growth to current and future climate and environmental impacts in Asia. This was expected to be achieved through improved forecast capabilities and the use of weather and climate information in decision-making; piloting innovative technology to get this information to vulnerable communities; and strengthened regional approaches to establish warning systems for transboundary climate hazards.

It resulted in several new and enhanced systems for providing climate and weather information to governments and communities at risk from climate impacts, in particular impact-based forecasting, seasonal forecasting, agricultural advisories and an early warning system for wheat diseases. As a result of the implementation of these and other climate services 2.3 million people have better access to accurate and relevant information on risks and coping strategies.

Challenges

ARRCC took a portfolio, multi-partner, adaptive approach to experimenting with new approaches to climate services across the region. The programme was delivered by the UK Meteorological Office and the World Bank, together with several international, regional and local technical partners. The partners piloted new approaches with over 30 interventions focused on different aspects of climate services in different locations.

We were the third-party MEL partner for ARRCC and had to design and implement a robust approach to monitoring and evaluating across the diverse set of interventions. We also had to adapt to and accommodate the delivery partners own internal systems for monitoring and reporting and provide guidance and capacity support as required. The central challenge was therefore to balance the reporting and accountability requirements of FCDO with the adaptive and flexible approach of the programme.

Our approach

We developed and used a theory-based methodology to monitor and measure systemic and institutional processes and changes and assess the programme’s contribution towards the expected impact. We facilitated a series of participatory workshops and discussions with the partners to build consensus around a programme level Theory of Change and MEL framework. We developed indicators and measures that could be applied to very different contexts and types of interventions.

We worked closely with both the UK Met Office and the World Bank to understand and align our approach with their own internal monitoring and reporting standards. For example, a detailed ‘Indicator Framework’ was developed and continuously improved, to provide detailed guidance and definition and sample tools to support their data collection and reporting.

We monitored progress across the programme through a monitoring dashboard that recorded milestones and detailed towards the logframe indicators. This was consolidated and described in annual reports and a full Programme Completion Report, as well as dedicated reporting against the International Climate Fund (ICF) Indicators. We carried out a comprehensive mid-term review of the programme to also evaluate progress and the relevance of the Theory of Change.

The most innovative aspect of our work related to documenting and sharing learning from the programme, both internally and externally. The learning component intended to improve the effectiveness and efficiency of the programme, as well as enhance its impact by informing and influencing the work of others. At the beginning of the programme we worked with partners to define three learning questions, related to what the programme is learning about facilitating regional cooperation, the political economy of climate services, and influencing pathways to support change.

Outcomes

We used a structured process of capturing learning from the partners and FCDO, which uses the findings from the monitoring and evaluation work but investigates why results happened in certain locations (and not others). This included ‘learning logs’ in partners’ reports and annual learning workshops. We investigated certain topics in greater depth through individual interviews with staff and external experts. This was documented in internal ‘Impact Stories’ and ‘Case Studies’ on four topics: The role of regional forums; Models for strengthening capacity; Supporting innovation; and the Political economy of climate services.

The various internal learning outputs were consolidated in a final learning paper ‘Strengthening climate services in South Asia: Learning from the ARRCC programme’. This documents how and why the programme has enabled change, and also where challenges remain.  The learning is organised into the factors that have enabled progress in strengthening climate services, and those that have constrained progress. These are a mix of external factors, such as the growing demand by sectoral line ministries for better information on climate risks, and internal factors, such as the programme’s partnership model and innovative approaches to capacity building. The paper concludes with a series of high-level recommendations, primarily to other programmes and partners that share a similar objective of strengthening climate services.

Some examples of the impact of the programme were also documented in a Stories of Change document. This illustrates just a few of the innovative new approaches to climate services supported from ARRCC and range from localised pilots to national forecasting systems to regional platforms. They aim to make the case for more attention and investment in empowering individuals and governments with the information they need to make informed decisions about how to respond to climate and disaster risks.

[button-link text="Stories of Change" link="https://sway.office.com/s/p660Mh2qkB79rXGS/embed"]