“It is challenging!” This is the narrative that is widespread when it comes to measuring protection outcomes. Why is it so challenging? Is it because of the sensitive nature of violence people face that makes it difficult to gather information? Or is it because humanitarian actors are accustomed to short-term, output-focused activities and methods which paint outcome measurement as challenging? There is no right or wrong response: the truth is a mixture of both. Nevertheless, new approaches, such as Results Journals, are being developed to help ease the process of measuring protection outcomes. To better understand how Results Journals can support organizations in their efforts to measure protection outcomes, this case example explores CIVIC’s use of them in northeastern Nigeria.
In 2016, CIVIC began operating in Maiduguri, Borno State, Nigeria, in collaboration with a peacebuilding and conflict management consortium. Upon commencing its programming, CIVIC and its partners encountered a significant issue: how could they capture and track changes in behavior—the intermediate results of their programs— that would help them understand whether they were on track to achieve their desired outcomes? How could they know if they were making progress towards protection outcomes, knowing that failure to capture them would mean not knowing if the project was effective and ultimately, successful. In response to this challenge, CIVIC started using Results Journals. Along with the consortium, it refined the tool for the project in Nigeria.
Results Journals, also known as Outcome Journals, draw on outcome mapping methodologies—which guide an organization to specify its target population, the intended change, and how it plans to facilitate that change. Outcome mapping helps organizations undertake a continuous learning process that helps set out a pathway to bring about change within the community. Results Journals, an outcome-oriented tool, generally capture qualitative information that measures that change. Rather than relying on project indicators and logframes, the tool is designed to be community- friendly.[/simple_tooltip]Outcome Journals can be relatively simple, like a chart or spreadsheet in Excel, or a more detailed narrative when the given context is extremely unpredictable. The aim is to capture behavior change within the environment. For protection, that means looking at change within the threat component of the risk equation and measuring whether the vulnerability vis-à-vis that threat or the capacity of populations to overcome the threat has changed.
WHAT IS OUTCOME MAPPING?
Outcome mapping is a method for planning, monitoring, and evaluating projects and programs that aim to achieve lasting social and behavioral change. It was originally designed by the International Development Research Centre in Canada, with the first guidebook being published in 2001. Since then, the method has been developed and used across a wide range of development and program contexts, and has been adapted and built upon by many of the organizations using it. An online learning community has been established to help program managers learn about outcome mapping, and includes a range of useful resources for anyone seeking to learn more. The community is available at https://www.outcomemapping.ca/start-here
Outcome journals are one of the standard monitoring and evaluation tools deployed by outcome mapping approaches to measure complex change within a population, community, or institution. An outcome journal is a tool for collecting data about behavior change over time. What makes it a journal is the use of a community-based record of changes over time. What makes it an outcome journal is the focus on behavior changes within the community itself; rather than recording progress in delivering a program or set
Outcome Journals focus on how a program facilitates change rather than on how it causes change, and looks to assess contribution rather than attribution.EARL, S.; CARDEN F. and SMUTYLO, T. (2001) ‘Outcome Mapping, Building Learning and Reflection into Development Programs’, Ottawa: International Development Research Centre, p.78
To better understand how CIVIC uses Results Journals to achieve protection outcomes, we must look at how CIVIC approaches humanitarian protection work and, more specifically, its intended outcomes.
CIVIC aims to protect civilians in armed conflict. In particular, CIVIC directly engages with armed actors, affected communities and policy makers to develop strategies to reduce civilian harm. CIVIC has employed Results Journals in their fieldwork in Borno State, Nigeria.Its frontline staff uses Results Journals to track changes in the environment monthly, looking specifically at the behavior of Nigerian state security forces and allied ANSAs, such as the CTJF and CIVIC’s contribution to a specific protection result. The information collected in these journals allows CIVIC to adapt its targeted action plans by measuring immediate results, a key step in achieving a meaningful reduction in risk. If the journals indicate that an action plan is not successfully changing behavior and achieving results, CIVIC re-evaluates its strategies and develops a new action plan—establishing an iterative feedback loop.
Moreover, Results Journals help CIVIC engage with local actors, a cornerstone of their approach to humanitarian work.Thus, CIVIC’s use of Results Journals has fostered a culture of learning and established an effective method of communicating with affected populations—two hallmarks of RBP’s Outcome-Oriented Methods.CIVIC Community Engagement staff must talk to community members to acquire the information tracked in Results Journals, and these frequent interactions allow CIVIC to build strong relationships on the ground.
You can’t be in the office and […]monitor results.BULUS MUNGOPARK CIVIC’s Community Engagement Manager in Maiduguri
Apart from its immediate use of tracking intermediate outcomes, CIVIC uses the journals to collect information in a way that helps maintain project continuity and organizational memory in instances of staff turnover.CIVIC’s H.Q. staff uses the outcomes gathered to inform their organization-wide theories of change.Additionally, the H.Q. staff uses Results Journals across different country contexts to identify trends and facilitate cross-project learning. Such identification of trends and patterns helps CIVIC undertake continuous protection analysis.
Although it might look complicated, to start using a Results Journal a few key requirements needed include:
- Access to Excel and basic Excel skills,
- Dedicated frontline field staff able to access tracked information via sound protection analysis,
- Comprehensive training for both program and MEAL staff collecting information and inputting into the journals, and
- Willingness of monitoring & evaluation and program staff to review the spreadsheets to foster learning.
Yes, it might be difficult to traditionally measure protection outcomes. Results Journals, however, as evidenced by CIVIC, present humanitarians with the opportunity to effectively engage with the affected population to understand and prioritize addressing the risks that they are experiencing. Humanitarians can build off of their important work and explore how to use Results Journaling as a steppingstone to reinforce and sustain effective engagement with communities.
Above all, humanitarians everywhere should look to Results Journals or other similar outcome mapping tools to improve how they monitor changes in the threat an affected population encounters, their vulnerability to this threat, and their capacity to overcome it.
RBP QUESTIONS TO CONSIDER
-How might humanitarians use Results Journals (or a similar tool) to improve how they monitor changes in the threat, vulnerability to specific threats, and capacities of communities to overcome threat?
-How could affected populations engage more prominently in collecting information and contributing to the monthly entries in the Results Journals to monitor desired change?
-How could this tool be adapted to capture other intermediate changes or results? For example, changes in perception of the risk by community members, or changes in community members’ self-protection behavior over time?
-How could this tool be used in conjunction with other methods to triangulate key information?
-In what ways could the Results Journals be lightened to lessen the administrative requirement to regularly update entries to monitor change?
This case example is based on an interview with Bulus Mungopark, CIVIC’s Community Engagement Manager in Maiduguri, and Abacha Kachalla, Project Manager in Maiduguri, in December 2019. Lee Sutton, CIVIC’s Senior Advisor for Monitoring, Evaluation, and Learning, based in Washington D.C., also provided input.