A. Introduction
Increasingly, governments and international development actors recognize the importance and challenges of achieving impact at scale, but there remains a glaring gap between stated intentions and actual practices. A movement is emerging to professionalize discussions about scale and scaling and to place these issues in the center of the development conversation; and the Global Community of Practice on Scaling Development Outcomes (CoP) has been actively engaged at the crossroads of this movement. Active since 2015, the CoP’s 2000+ members – representing more than 400 official donors, foundations, governments, academic institutions, think tanks, NGOs, private companies and social enterprises – are drawn from the global north and the global south. The CoP’s mission is to use its multi-sectoral composition, diversity, independence and convening capacity to provide direct support to its members and to professionalize the practice of scaling in members’ organizations and more broadly. This document is based on a series of six virtual events that took place from April to November 2021, organized by the CoP’s Monitoring and Evaluation Working Group (MEWG), one of the CoP’s nine working groups1 . The webinars, and the current working paper, emerged from a recognition that recent attention by donor agencies, foundations and NGOs to the need for scaling improved practices by institutionalizing those changes in government agencies have not been accompanied by comparable attention to the role of M&E, and the role of “evidence” more generally, in facilitating that institutionalization. The goal of this document is to present the rich and critical insights emerging during the 2021 webinar series. The first draft was developed through careful review and culling from the presentation recordings which are listed in the annex on page 14 by a consultant on behalf of the CoP. The series presenters spoke from their practical experience supporting efforts to institutionalize within government health and education interventions that were incubated in NGOs. They represent, we believe, a useful point of departure for the MEWG, the CoP, and the larger development community to engage a range of issues not fully addressed in the initial 6 webinars or in this paper. Given the critical importance to the development field of better understanding what makes institutionalization succeed generally, the CoP has as a goal to continue and expand its deep dive on this topic
CLICK HERE TO READ MORE
B. Overview of Institutionalization and the Scale-Up Community of Practice
When the Monitoring and Evaluation Working Group (MEWG) was established at the launch of the Global Community of Practice (CoP) in 2015, its first order of business was to disaggregate and map the relationship between Monitoring and Evaluation, on the one hand, and Scaling, on the other. After almost two years of discussion, the Working Group settled on a three-tier framework (Figure 1) which is has used to guide it subsequent deliberations. Although work remains to be done in fully adapting Tier 1 to the implications of scaling, the MEWG concluded that there is relative consensus and considerable good practice in the first tier, Proof-ofConcept, on articulating a prototype intervention and pilot testing to demonstrate its impact2 . The group concluded that, as a community, we have less understanding and agreement on good practice in the next scaling tier, Scalability Assessment, intended to support refinement, streamlining, and testing for robustness; and it devoted considerable effort during 2020 to augmenting scalability tools and frameworks3 . The third tier, Change Management, focuses on the information needed to guide and track the widespread integration of the new intervention. It represents the least developed area of Monitoring, Evaluation and Learning (MEL) within the development community and includes issues such as monitoring the scale-up process, tracking institutionalization, ensuring fidelity, and minimizing unit cost at scale. As an effort to deepen our knowledge and practice regarding this tier, the MEWG focused particular attention in 2021 on the role of MEL where the preferred scaling strategy is “institutionalization” within government of interventions or practices that were initially developed and tested outside of government.
From April to November 2021, the M&E Working Group convened a series of webinars to learn from select partners about the role MEL played in institutionalizing their education and health approaches into public sector programming in low-and-middle-income countries. The webinars included presentations by NGOs that initially played the role of originating organization, leading or co-leading the pioneering of particular interventions (Tier 1) and subsequently also played roles as intermediary and boundary organizations (Tiers 2 and 3)4 supporting institutionalization of those interventions within government. To round out institutionalization perspectives, presenters also included government ministries that aim to institutionalize these interventions and donors that support these efforts.
The webinars underlined the fact that institutionalization is a complex, context-specific process with technical, financial, and sociopolitical dimensions. As one webinar panelist, Angela Gichaga, CEO of Finance Alliance for Health, said, institutionalization is “a process not an event; a continuum, not a binary outcome; subjective and not always quantifiable.”
Two lines of thinking cut across the presentations. First, they reflect a shift in the broader development community to ‘do development differently.’5 Originators of new interventions are increasingly moving beyond direct service provision and taking on the role of long-term partners to government, working collaboratively to support ministries to adapt, integrate, scale, and sustain interventions through government systems and programs many of which were pioneered outside of government. In the scaling parlance, these organizations are playing the role of both originating organizations and intermediary/boundary organizations. The case examples highlighted in this webinar series– including from Educate!, Last Mile Health, Living Goods, and Young Love – reflect this change. This change in perspective is echoed in the MEL approaches discussed below. Second, there is growing recognition of the complexity and the non-linear processes and extended timeframes that characterize institutionalization into national systems. This acknowledgment of complexity and extended timeframes is also mirrored in the metrics and MEL systems needed to inform, track, and support these changes.
To provide a common point of departure and a foundation for the discussion, presenters at all six webinars –representing intermediary/boundary organizations, governments, and donors – were asked to link their presentations to the MSI Institutional Tracker6 and reflect on the fit between that framework and their respective approaches to monitoring the process of institutionalization.
The MSI Institutionalization Tracker is a maturity model that tracks progress in a total of 11 institutional building blocks spread across five domains. Figure 3 below provides illustrative stages of institutionalization for two of these domains and a total of four building blocks. The four levels shown for each building block represent dimensions of institutionalization that can be tracked as institutionalization proceeds and that provide a way to chart progress and outstanding challenges. Each of the originating organizations participating in the webinar series used some analogous rubric to track and guide progress towards institutionalization.
There was general agreement that the indicators and metrics included in the MSI Institutionalization Tracker and similar instruments can and should be enriched by viewing systems as having both “hard” and “soft” characteristics and by related efforts to measure soft system characteristics. Hard system perspectives of institutionalization focus on more tangible elements such as funding re-allocation and policy revision, with relevant objective indicators tracking achievement. Institutionalization efforts also often integrate soft system perspectives that explore less tangible elements of motivations, incentives, power, and voice of different actors in the systems and the broader political enabling environment. Relevant indicators of achievement for these soft system elements are often defined more subjectively7 . This paper summarizes discussions from the webinars focusing on: how originating and intermediary organizations support institutionalization processes in real-life contexts; and how M&E data can help in tracking and guiding the institutionalization processes. The closing section looks at M&E issues raised during discussions that merit more attention from the Scale-up Community of Practice.
C. How Originating and Intermediary Organizations Support Institutionalization
1. Collaborative engagement and systems-support roles are crucial to support the institutionalization process.
All the speakers – whether from intermediary/boundary organizations, donors, or government – indicated that early and frequent collaboration with government was crucial to advance institutionalization from intervention co-development to planning for and implementing scale up strategies. Each also emphasized the central role MEL plays in this process and how the respective MEL roles of originating organizations and government ministries shift as institutionalization proceeds. The webinars illustrated numerous variations on collaboration. Many of the NGOs which initially focused on direct implementation have now embraced their roles as intermediary/boundary partners to transfer the model to government systems through a support function. Each of these intermediary organizations is working at multiple levels, with staffing both at delivery sites and within central ministries. Their collaborations are not just technical but include building coalitions that facilitate access and trusted relationships with government and donors, civil society, and researchers. At the same time, most of these NGOs continue to implement the intervention directly, albeit sometimes at a smaller scale, to maintain on-the-ground understanding and push further “R&D” and refinement of the intervention models. Presenters emphasized that these new and evolving roles mean shifting their own mindset and MEL systems — and those of donors — to highlight working collaboratively with government ministries to support institutionalization. It requires them to think and walk in the government’s shoes. Practical advice offered by these NGOs includes:
• Be participatory and insider-focused, involving government in co-planning, co-design, and evaluation of interventions. A scaling mindset shifts the scaling focus from that of an originating organization to one focused on government aims.
• Take the time to co-visualize institutionalization aims with government partners and users and co-analyze and define the change in services, funding streams, and other support sectors to improve health and education outcomes. Ideally, these collaborations begin during proof-of-concept research8.
• Consider co-locating within government units MEL and program advisors with expert knowledge of the intervention being institutionalized. Co-location, which often happens at the central or district level, can maximize an intermediary organization’s insider interactions and understanding of issues from a government perspective, facilitate knowledge sharing, and offer insider opportunities to leverage actors and resources for institutionalization.
Download the report to read more!
