On May 4, the EBA launched the report “Confronting the Contradiction” – written by three researchers in Norway, Hilde Reinertsen, Kristian Bjørkdahl and Desmond McNeill. The report was launched during a seminar at Medelhavsmuséet. Through studies of evaluations of Swedish and Norwegian aid, they observe that there is a persistent trade-off between accountability and learning in aid evaluation. The result, they argue, is a prioritization of the former at the expense of the latter – or put simply: learning is crowded out by accountability.
Their main recommendation arising from this is that everyone involved in doing and discussing aid evaluation need to start talking more openly about this trade-off and adjust their expectations of both aid interventions and evaluations. For example, those that commission evaluations need to be clearer about whether they are seeking to enhance learning or assign accountability when they draw up the Terms of Reference. Policy makers need to be clear about what dimensions they are seeking to promote in order to build, assign roles and more generally guide the workings of the evaluation system.
The authors identify four choices that need to be made:
- Does the evaluation process need an evaluation report, and if so, what kind?
- Does the evaluation process benefit from an external evaluation team?
- Should the evaluation report include recommendations?
- Should accountability systems be given the current high priority by donors, even when they come at the expense of learning?
In a panel discussion following the launch of the report, the EBA had invited two heads of evaluation of bilateral development cooperation: Per øivind Bastøe (Norad) and Joakim Molander (Sida), as well as Cecilia Magnusson Ljungman (consultant at NIRAS) and Evert Vedung (professor emeritus at the University of Uppsala) to discuss the report further.
The discussion showed a great divergence of views. While Evert Vedung for example took the position that accountability and learning are indeed incompatible, Per øivind Bastøe maintained that they are not. Joakim Molander was rather of the opinion that it’s impossible to separate the two because learning and accountability is taking place simultaneously at different levels of the evaluation system. Cecilia Magnusson Ljungman also commented along those lines, saying that the one does not exclude the other and that they sometimes even reinforce each other.
The panel was in agreement that the evaluation system as a whole is constantly learning. They also noted that there indeed has been a movement in the last five years or so in the direction that the authors suggest it needs to move. One even suggested we were already there. However, there is certainly scope for continued learning in this field and by implication greater awareness around the choices that are implicitly, or indeed explicitly, made in the process of formulating evaluations. After all, depending on whether emphasis is on learning or accountability evaluations are, both in terms of scope and reporting mechanisms, likely to come out quite different. Clarifying the intentions behind a particular evaluation may also facilitate its uptake.
Through the launch of this report and the ensuing discussion we hope to encourage further reflection and greater awareness about the practices around aid evaluation. We will certainly bring it to our internal deliberations about what we are seeking to achieve through EBA evaluations and look forward to continued discussion around this issue elsewhere! We encourage you to join the discussion at Biståndsdebatten.se