Abstract
Development funders have an important role to play in supporting the pursuit of sustainable impact by the implementers of development programs, but this requires that they systematically integrate the objective of scaling, i.e., achieving sustainable impact at scale into their mission and their operational practices. Evaluation and evaluation guidelines are potentially important instruments for mainstreaming scaling in funder organizations. This paper provides an indicative framework for how evaluation guidelines could address scaling. It then reviews the OECD-DAC EvalNet evaluation guidelines, the OECD-DAC Peer Review methodology and the MOPAN assessment methodology, as well as publicly available evaluation guidelines for 18 large bilateral and multilateral official funder agencies, to determine to what extent and how they incorporate an explicit focus on scaling.
For the OECD-DAC and MOPAN guidelines the paper finds that, while scaling is not entirely absent, the guidelines do not treat scaling effectively, let alone provide helpful guidance to evaluators on how to assess their agency’s approach to scaling and how to evaluate their performance on scaling for specific projects or programs or overall. Since many of the evaluation guidelines of the individual funder agencies are based on the OECD-DAC EvalNet guidelines, this represents a missed opportunity to influence and support the evaluation units of official funder organizations and, indirectly, to strengthen the incentives for funder agencies to mainstream scaling,
The paper finds that for ten funder agencies evaluation policies or guidelines do not address scaling; for another four, there is some, but only very limited coverage; while for a final four agencies scaling is a central part of the evaluation policies or guidelines, with varying degrees of guidance provided. This last set of cases demonstrates that the inclusion of scaling in evaluation guidelines is possible without a fundamental departure from existing evaluation approaches.
The paper then reviews 17 evaluations and assessments for four agencies: four OECD-DAC peer reviews, three MOPAN assessments, six IFAD program and project evaluations and four World Bank country program and project assessments. As would be expected, the IFAD evaluations have the most extensive consideration of scaling, while World Bank and OECD-DAC peer reviews pay very little, if any, attention to scaling. MOPAN assessments somewhat surprisingly do consider scaling to some extent, even though the MOPAN methodology provides little guidance on scaling to assessment teams.
The paper concludes with a summary of findings and recommendations for how scaling could be more systematically and effectively addressed in evaluations.
