Measuring and Monitoring Adaptive Learning: A Landscape Review

The monitoring and evaluation of adaptive learning is an emergent field. Although there is a growing body of literature on adaptive programming more generally, there is a limited knowledge base on the monitoring and evaluation of adaptive learning interventions and their impacts. Unlike other implementation strategies or program management approaches, there are no standard metrics or monitoring and evaluation frameworks to track the integration, implementation, and effectiveness of adaptive learning in health programming. As a result, USAID has released a landscape review on measuring and monitoring adaptive learning.

The principle of adaptive learning is reflected through the first element of Results-Based Protection, Continuous Context-Specific Protection Analysis which is enabled by an organization’s culture systems and resources. Additionally measuring and monitoring adaptive learning as carried out by MOMENTUM includes an outcome-oriented approach.

Before we go further into the review, it is important to understand what adaptive learning is as it is an emerging field.

As defined by the MOMENTUM Adaptive Learning Guide, adaptive learning is the intentional adoption of strategies and actions to facilitate critical reflection and analysis of data, information, and knowledge—on a continuous basis and from a wide range of sources—to inform decisions that optimize program implementation and effectiveness in expected, unexpected, and changing circumstances.

In short, adaptive learning allows project teams to better understand the context that they are working in, ongoing changes to the context, how that context interacts with or affects project implementation, and how to respond effectively.

Now that we have a collective understanding of adaptive learning, let’s take a look at what the recent review process entailed.

The landscape review includes six key resources that support and can help strengthen efforts to implement an adaptive learning environment. These resources focus on essential capacities, processes, structures, team composition and roles, management, and Monitoring, Evaluation, and Learning (MEL) procedures, and critical skillsets. They resources include:

1. EPIS (Exploration, Preparation, Implementation, Sustainment) Framework;
2. USAID’s Collaborating, Learning and Adaptation (CLA) Framework;
3. MOMENTUM Adaptive Learning Guide;
4. Abt Associates Implementing Adaptive Management guide;
5. Making Adaptive Rigor Work: Principles and Practices for Strengthening Monitoring, Evaluation and Learning for Adaptive Management (MEL4AM);
6. and Rapid Evaluation Action and Learning (REAL) Framework and Toolkit.

The first four resources focus on a comprehensive implementation strategy or program cycle. They consider a range of program capacities and processes that must be introduced or strengthened at different levels when building an adaptive learning approach. The last two resources focus specifically on processes, standards, and mindsets relevant to data collection and data use (e.g., use of TOCs, experimentation cycles, the choice of metrics, data quality assurance, and data visualization), and the need for strengthening skills in data interpretation and steps to facilitate collective decision-making around program modification.

In measuring adaptive learning, MOMENTUM uses outcome-oriented tools. For example, it uses outcome mapping to plan and monitor approach to capture progress towards outcomes which often applied in a participatory manner. Outcome mapping in relation to measuring adaptive learning is often used throughout the program starting from the design phase and can be combined with other approaches or methods to support evaluation. Illustrative MEL questions include what types of outcomes are observed over time across different stakeholder groups? Or how are key partners and/ or stakeholders responding to program and changing their behavior, activities, or relationships?

By iteratively testing what works and what does not, and prioritizing learning and feedback loops, adaptive learning as an outcome-oriented approach, and its uptake amongst senior leadership in the humanitarian sector allows for continuous context-specific analysis and learning in pursuit of protection outcomes.