Scaling Up Community of Practice Workshop, October 27-28, 2016

Day One: Technical Workshop – Cross-cutting issues in scaling up development interventions

Opening and Welcome by Larry Cooley (MSI) and Gina Lagomarsino (R4D)

Larry Cooley, President Emeritus of Management Systems International (MSI), welcomed the 39 participants from 24 organizations. (See Table) He summarized the history of the Scaling Up Community of Practice (CoP) as a joint undertaking of MSI and the Results for Development Institute (R4D). He explained its purpose and activities to date (including the Quarterly Newsletter, four working groups, and annual CoP workshops). He also welcomed Gina Lagomarsino, who recently took over from David de Ferranti as President of R4D.

Gina Lagomarsino joined Larry in welcoming the participants and noted the importance of scaling up for the work of R4D. She explained that the purpose of the workshop is to explore crosscutting lessons of scaling approaches and experiences on the first day, and to review progress of the CoP since its launch in February 2015 and the way forward on the second day.

Warm-up: Practical lessons from recent scaling experience

Moderator: Gina Lagomarsino

Panel: Tressa Johnson (ELMA), Barbra Gerhager (GIZ), Takao Toda (JICA), Aviva Kutnick (USAID)

The panel presented lessons from four cases of successful scaling up: early childhood development in South Africa (ELMA), Kenya water sector (GIZ), education in Niger and Cambodia (JICA), and commercialization of orange sweet potato in Malawi (USAID). Common lessons were:

• It’s important to have a clear vision of scale at the outset;
• Local ownership, local leaders and local partners are important from early on (“co-creation”);
• Solid evidence of successful impact from effective monitoring and evaluation can be a major driver of scaling up;
• Don’t only push from the supply side, but also pay attention to the demand side (early child development in South Africa);
• Sustainable/scalable financial solutions are essential, but not unambiguous:

» Short-term funding undermines sustained scaling, while longer-term engagement (e.g., with a pro-poor trust fund in the Kenya water sector) helps; and

» Access to funding in supporting uptake of an innovation is critical, including co-investment with a private partner, esp. a larger firm, able to bear the start-up risk (USAID’s orange sweet potato program);

» But too much money can also undermine sustainability (“minimize the use of money, maximize the use of wisdom” – JICA’s education programs); » Transparency is needed to prevent corruption.

Technical Session: The value and use of analytical frameworks in approaching scaling up – a conversation

Moderator: Ruth Simmons (ExpandNet), with Eileen McGivney (Brookings), Larry Cooley (MSI), Johannes Linn (R4D, Brookings)

The moderator and panelists presented three alternative frameworks for the analysis, planning and management of scaling up pathways (ExpandNet, MSI, Brookings/IFAD), which have been tested in various different country and institutional settings. Eileen McGiveney reported on a practical application of key elements of these frameworks in a major empirical study of scaling up in education (Brookings). Key lessons drawn by the panel and in the ensuing discussion were these:

• Use an analytical framework that guides the thinking, allows for a systematic testing of assumptions, including through scalability assessments; it helps starting with an existing framework, although the specifics of the framework need to be creatively tailored to the sector, business line, and institutional characteristics; key elements of the frameworks presented in the session are common and complementary (see Cooley and Linn, http://www.msiworldwide.com/project/new-scaling-up-paper-describeslessons-learned/);
• Keep the desired end-result in mind when designing and implementing a scaling pathway (“vision of scale”);
• Apply a systematic, strategic approach to the planning and management of a scaling pathway; scalability assessments can play an important role in assessing the viability of the scaling process and the priority factors that need to be addressed;
• Understand the ecosystem of actors and institutions within which the scaling process takes place and endeavor to create a supporting enabling environment (“field building”, “creating the drivers and spaces for scaling up”);
• Watch the institutional dimension of the scaling pathway, esp. what intermediary organization(s) is (are) needed to facilitate the scaling process; aid institutions can serve as important intermediary organizations, but need to hand-off in good time and in a well-prepared manner;
• Focus on the creation of alliances, partnerships, stakeholder coalitions, by driving for consensus, and ownership; and
• Address the common problem of the “valley of death” experienced by innovators when scaling up, by assuring sufficient funding is given to innovations in the early-to-middle stage of scaling (i.e., after proof of concept and before the scaling process has become self-sustained);
• Ensure effective monitoring and evaluation of impact and of key enabling conditions along the pathway (“RCTs help, but are not sufficient”); this may well lead to an adaptation of the innovation or of key enabling factors in the scaling pathway (“iterative process of innovation-learning-scaling”); and,
• Demystify the scaling up language and keep the frameworks and bureaucratic requirements simple, so as not to overburden operational teams in funder and implementing agencies alike.

Technical Session: Accessing scalability – a participatory exercise

Facilitator: Larry Cooley (MSI)

This session featured a participatory exercise to explore issues and approaches regarding how to assess scalability of innovations and interventions. Larry introduced the session by reminding participants of the three functions of scalability assessments:

• For donors: assess how feasible is it to take X, Y or Z to scale
• For planners: design pilot projects and prototypes in ways that test and improve prospects for scaling (i.e., design with scale in mind)
• For practitioners: illuminate blind spots

And he noted that scalability checklists, such as the one contained in the most recent MSI handbook on scaling (see: http://www.msiworldwide.com/wp-content/uploads/ScalingUp_3rdEdition.pdf), serve to identify what’s working for you (i.e., is it simplifying), and what’s working against you (i.e., is it making the task more difficult), so that you can more effectively select what you can and should take to scale and what you can drop without ruining the integrity and core of the innovation. (“Piloting is the process of addition, scaling is the process of subtraction.”)

Small-group discussions among participants around these issues were animated and informative. Key points addressed included:

• Sustainability and scalability must be treated as “twins” – you cannot address one without the other;
• Scalability assessment tools are best used to facilitate a discussion around assumptions and solutions to scaling challenges, and for ranking alternatives; they are less helpful for go/no-go decisions.
• While donors can and should use these tools, ultimately it’s important that innovators and implementers to use them in designing and managing scaling pathways.
• Scaling is subject to uncertainty and risks; scalability assessment tools can help in appraising the risks.
• The systematic development of an explicit scaling strategy (pathway), assisted by scalability assessments, makes it possible to connect high-level, long-term goals (e.g., the SDGs) with today’s and tomorrow’s project interventions.
• A key feature of scalability is local ownership and alignment with the local ecosystem.
• “Seeing is believing!” Credible evidence of success along the pathway will enhance the scalability of an intervention.

Technical Session: Mainstreaming scaling up into operational practice in development institutions – a conversation

Moderator: Johannes Linn (MSI, Brookings)

Panel: Maria Elena Mangiafico (IFAD), Mulu Checkol

(Save the Children), Simon Winter (Technoserv), Alastair

McKecknie (ODI)

Johannes introduced the session by stressing the importance of mainstreaming the scaling up practice in development institutions; currently, the scaling up idea is often still pursued by people and units are the fringes of these institutions. Panelists spoke about ongoing mainstreaming efforts in the institutions they work or have worked with. Key lessons included these:

• Mainstreaming scaling involved an institutional change effort that is in effect a scaling process;
• It takes time and persistence, it requires a vision, a systematic approach, an assessment of enablers (champions, leadership from the top, staff incentives, partnerships, etc.) and barriers (bureaucratic inertia, staff overload with “new fads and buzz words”, the cost of training and staff development, etc.) and appropriate action to address them, etc.;
• Operational business models and instruments, as well as internal policies and management processes need to be reviewed and aligned with the scaling agenda;
• M&E practices must be adjusted to include an explicit consideration of the scaling objective, and the mainstreaming process itself must be monitored and evaluated at regular interval;
• At the same time, bureaucratic processes need to be kept as simple as possible, with a focus on changing the mindsets of managers of managers and staff, allowing more risk taking, recognizing that lack of success in projects is an opportunity to learn, rather than a failure, and remembering that scaling is about more and better development impact, not necessarily about more money or bigger organizational footprint.

Day Two: Organizational meetings of Working Groups and of the Scaling Up Community of Practice

Education Working Group

The Education Working Group, led by Caitlin Moss (R4D), began with introductions by each session attendee and a brief review of their current work on scaling up and/or education. These were followed by three presentations: Laura Ghiron (ExpandNet) presented on a collaboration between The Education Partnership (TEP) Centre and ExpandNet with a focus on girls’ secondary education in Nigeria. Eileen McGivney (Brookings) presented an update on the Millions Learning project and its role out of phase two. The third and final presentation by Collette Chabbott (GWU, International Education Program) discussed scaling innovations at the local level and the role of organizations and stakeholders in the scaling up process. The working group session concluded with a discussion of the group’s current activities and its role moving forward. To date, the group has focused on collecting and disseminating information on scaling up in education and building member relationships. Additional activities suggested by the group include mapping current scaling initiatives in education, and identifying research gaps related to scaling in education. R4D is the current secretariat of the Education working group, and proposed that the role become a rotating position amongst willing members. The group will also establish an Executive Committee to help set the agenda for future activities.

Agriculture and Rural Development Working Group

The Agriculture and Rural Development (ARD) Working Group, led by Maria Elena Mangiafico, began with introductions by attendees and a review of the work each member organization is conducting within the scaling up and/or agriculture sectors. Common themes in the introductions: research and innovation, rural community development and local capacity building, and cross collaboration between scaling up and agricultural programming at USAID. The group discussed opportunities for collaborating on similar initiatives within their different organizations. Specifically, identifying countries where CGIAR initiatives were already underway and where platforms for citizen level change were already in place. The idea is not to start from scratch, but to use common resources and contacts to plug into an existing platform. Working group members agreed that a next step would be identifying countries and programs where the working group members feel they can work together. The session’s attendees also agreed that it would be valuable to create a shared term of references and resources. A proposal was made to create an executive committee for the ARD working group with rotating leadership in order to share the management of the group and its resources.

Monitoring and Evaluation Working Group

The Monitoring and Evaluation (M&E) Working Group discussion, led by Larry Cooley, acknowledged that M&E in pilot projects too often limits its attention to proof of concept and accountability for project level results.  A framework was introduced that included four “tiers” of information needed for successful scaling, namely: 1) proof of concept; 2) information to guide refinement and streamlining; 3) scalability assessment; and 4) information needed for successful change management. In the discussion that followed, Working Group members most commonly identified tier 3, scalability assessment, and tier 4, change management, as areas where improved evidence protocols and systematic investigations could add greatest value. Group members shared experience and methodologies for systematically informing the conditions under which interventions can be transferred from one context to another and cost-effectively scaled.  Other themes touched on during the discussion included the importance of articulating a “second theory of change” linking pilot projects to broader change strategies; and the need for additional casebased documentation of the process by which interventions are successfully scaled.  Finally, the group identified several actions they would find helpful in support of their own M&E efforts related to scaling and formed dyads to collaborate, act as sounding boards, inform and support one another’s work on M&E for scaling up moving forward.

Fragile States Working Group

The Fragile States working group, led by Jonathan Papoulidis, began with an introduction around the primary characteristics of fragile states, which include weak governance, broken social contracts, low resilience to stressors and shocks, and irregular financing. During the session three questions were posed to the group around scaling in fragile states, including: 1) how do we better understand complex risk environments? 2) how do we work to scale development outcomes that also work at fragility? and 3) how do we coordinate financing in these states? The discussion that followed highlighted the need to identify what should be scaled in fragile contexts, the integral component of building local capacities for scale, and the need for flexibility and adaptability in methodological approaches to scaling within volatile contexts. Finally, the group discussed how best to take the Fragile States Working Group forward. Prevalent ideas included the need to share its work with others given the context rather than sector focus of the group, substantiating the process of scaling through sharing case studies and stories, as well as the working group itself acting as a support organization for members who are becoming change agents for scaling within their own institutions.

Scaling Up Community of Practice

Members of the Scaling Up Community of Practice (CoP) met under the leadership of Larry Cooley. They reviewed the CoP organization, modalities and progress since its launch in February 2015. The following are the key takeaways from this meeting:

Members provided strong endorsement of the value of the CoP. They recognize that a model that relies entirely on the voluntary contributions of all its members, including the pro bono provision of all organizational tasks, is subject to limitations regarding the intensity and continuity of effort. However, they feel that the current approach (including annual workshops, quarterly meetings of working groups, quarterly CoP newsletters) has many strengths and benefits, and hence is worth preserving. The meeting explored options for further strengthening the work of the CoP (including expansion of membership, the development of a curated Web site, the establishment of a formal secretariat, mobilizing dedicated funding) and agreed that these will be kept under review. The meeting welcomed the tentative offer of the JICA representative to fund participation of developing country experts at the next annual workshop. Participants confirmed that, as a general rule, the CoP invites two members for each participating organization. They also welcomed the partnership between the Scaling CoP and “Community of Practice on Systematic Approaches to Scale-up on Family Planning/Reproductive Health Best Practices” run by E2A and ExpandNet. MSI and R4D indicated their readiness to continue taking a joint lead in organizing the work of the CoP. Participants expressed their appreciation of the efforts by MSI and R4D in leading the CoP and organizing the workshop, and offered their special thanks to Kelly Phillips and Molly McFarland for their efficient organizational support and note keeping.

Publications and Blogs

We list here selected publications that have come across our desks and that we think you might be interested in. They deal with scaling up successful development innovations across a wide range of topics. The brief summaries are drawn from the abstracts or summaries in the original documents.

General

Dan McLure, Ian Gray, “Scaling Assessment Map: An Evolving Tool Supporting Innovation Scale Up.” ThoughtWorks Blog, 14

November 2016

https://www.thoughtworks.com/insights/blog/scaling-assessment-map-evolving-tool-supporting-innovation-scale

The Scaling Assessment Map was developed by Ian Gray and Dan McClure to support the evaluation and planning for pilot innovation programs embarking on the scale up journey. This particular design for a scaling assessment is based on their prior work exploring the diverse challenges an innovation team must meet when working to create a sustainable ecosystem around their idea. The map is an evolving design that is in the process of being used and refined with innovation teams and mentors.

Chris Blattman, “Why ‘what works?’ is the wrong question: Evaluating ideas not programs.” Blog, 2016 http://chrisblattman.com/2016/07/19/14411/

The blog argues that Instead of asking “does the program work?”, one should ask “How does the world work?” What is wanted is a reasonably accurate model of the world: why people or communities or institutions behave the way they do, and how they will respond to an incentive, or a constraint relieved. Randomized trials, designed right, can help move us to better model. Even this theory-driven, “test the big assumptions” approach to randomized trials is not enough. What is needed are many other ingredients, including better theories and understanding of context and cases. Another powerful ingredient, however, is bigger randomized trials: multiple trials, in multiple places, testing some of the same key ideas and assumptions. This strikes is the only possible path from randomized trials to general knowledge.

Gareth Davies, “Getting to scale: Lessons in reaching scale in private sector development programmes”, Adam Smith

International, June 2016 https://beamexchange.org/resources/785/

This paper looks at the experience of getting to scale in Private Sector Development (PSD) programs. In particular, it looks at the successes and failures of programs in scaling pro-poor ‘innovations’. The paper focuses on taking the initial ‘innovation’ to scale: section A summarizes a variety of different strategies and tactics for getting to scale; in section B, ten lessons in getting to scale are presented, drawing on a variety of programs and contexts; and section C concludes by discussing the implications for donors and practitioners. The Annex presents a new tool to track progress in getting to scale.

Education

R4D, “Journeys to Scale: Accompanying the Finalists of the Innovations in Education Initiative.” Report, 2016.

http://www.resultsfordevelopment.org/sites/resultsfordevelopment.org/files/resources/Journeys%20to%20Scale%20-%20 Executive%20Summary.pdf

This joint report by R4D’s Center for Education Innovations (CEI) and UNICEF shares the story of five promising innovations as they attempt to scale and ultimately improve educational outcomes.  Journeys to Scale accompanies programs from Brazil, Ethiopia, Ghana, Peru, and Sudan, as they strive to increase their impact. Drawing from the challenges faced and strategies employed to overcome such hurdles, it lays out clear recommendations for implementers, donors, policymakers and researchers who want to support innovation.

J-Pal, “Teaching at the right level.” Blog, 2016

https://www.povertyactionlab.org/scale-ups/teaching-right-level?utm_source=newsletter&utm_campaign=october&utm_ medium=email

“Teaching at the right level” (TaRL) is a pedagogical approach that involves evaluating children using a simple assessment tool and then grouping them according to learning level rather than age or grade. Each group is taught starting from its current competency level, and level-appropriate learning activities and materials are used. Throughout the entire process, teachers assess their pupils’ progress through ongoing, simple measurement of their ability to read, write, and comprehend and do basic arithmetic. Research by J-PAL affiliates over the past 15 years has shown that this method of reorienting teaching to the level of the student, rather than the rigid expectations of a curriculum, consistently improves learning outcomes. A recent paper summarizes the iterative approach of design, rigorous evaluation, and redesign that developed the TaRL models now being scaled in developing country government schools.

Alice Jowett, Caroline Dyer, “Scaling-up successfully: Pathways to replication for educational NGOs”, International Journal of

Educational Development, Volume 32, Issue 6, November 2012, Pages 733–742 http://www.sciencedirect.com/science/article/pii/S0738059311001702

Non-government organisations (NGOs) are big players in international development, critical to the achievement of the Millennium Development Goals (MDGs) and constantly under pressure to ‘achieve more’. Scaling-up their initiatives successfully and sustainably can be an efficient and cost effective way for NGOs to increase their impact across a range of developmental outcomes, but relatively little attention has been paid in the education sector to how best this may be done and debates appear to have stalled. One approach to scaling-up is replication, on which this paper focuses. This paper presents four known paths to replication alongside new illustrative diagrams to explore their advantages and disadvantages, and highlights the need for a fifth approach that does not appear in current literature. ‘Network replication’ is proposed as new pathway that draws on known strengths of networks and offers a learning-oriented approach to scaling-up. While inevitably not without its potential problems, Network replication is a scaling-up option that deserves consideration.

McKinsey & Company, “Scaling up systematic change for learning in the US.” Blog, 2016 http://www.mckinsey.com/industries/social-sector/our-insights/how-to-scale-personalized-learning

The US public education system is struggling to meet the needs of its students. A system-wide approach to personalized learning may help. The approach consists of identifying a vision, establishing a process of piloting and learning, and creating a supportive environment for innovation.

Health

USAID, “Idea to Impact: A Guide to Introduction and Scale of Global Health Innovations.” Report, 2015 https://www.usaid.gov/sites/default/files/documents/1864/Idea-to-Impact_Jan-2015-508_0.pdf

This Guide, along with its companion Toolkit and Workbook, identifies priority activities at each stage of the development continuum to help innovators and practitioners think through, plan, and execute the many activities needed to ensure successful launch and scale from bench to bedside. The Guide provides context through examples, and offers a growing set of tools that can help users put these activities into practice.

USAID, “Pathways to Scale: A guide for early-stage global health innovators on business models and partnership approaches to scale-up.” Report, 2016 https://www.usaid.gov/sites/default/files/documents/1864/Pathways-to-Scale-Guide_20161013_online-508.pdf

“Pathways to Scale” aims to help early-stage innovators develop business models and partnership approaches that align with the development of their products, and envision potential pathways to bring products to scale. It introduces the most commonly found models for scaling up global health innovations, and features case studies that highlight and explain pathways taken by innovations that have begun to scale-up. It also offers a toolkit with exercises, structured questions, key considerations, and curated resources that innovators can use to identify the most suitable scaling model(s) to forge their path.

Agriculture

USAID. “Workshop Report: Bureau for food Security’s Agricultural Innovations Workshop”, Report, September 22, 2016 https://agrilinks.org/sites/default/files/resource/files/BFS%20Scaling%20Workshop%20Report%20DEC.pdf

This report provides an overview of a workshop hosted by USAID’s Bureau for Food Security on July 13, 2016, in Washington, DC. The purpose of this workshop was to share and validate preliminary evidence and recommendations from a series of case studies looking at the successful scaling of pro-poor agricultural innovations through commercial pathways. The workshop sought feedback from a small group of experts on the preliminary conclusions and insights generated by the case studies on what facilitates successful scaling up and how to improve donor scaling practices.  Following a brief overview of the five case studies and the workshop participants, this report provides a descriptive narrative of the discussions during each of the workshop sessions. The report presents key ideas and overarching themes from the workshop that may be of interest to readers engaged in the scaling of agricultural innovations in developing countries.

Water and Sanitation

GIZ, “Closing last mile for Millions: Sharing the Experience on Scaling up Access to Safe Drinking Water and Adequate

Sanitation to the Urban Poor.” Report, 2016 https://www.giz.de/fachexpertise/downloads/giz-2015-en-closing-the-last-mile-for-millions.pdf

This paper reports on the experience gained in Kenya and Zambia in providing support for a significant increase in access to water and sanitation by a more focused support to extending formal services into low-income areas. Key lessons learnt from Kenya and Zambia include: Scaling up needs to be embedded in national and local sector institutions and supported by pro-poor sector policy and strategies to reach the urban poor in a sustainable manner. Rigorous monitoring beyond completion of investments ensures adherence to minimum service standards and long-term effective contributions to access targets. The process also allows for incremental improvements to service levels in line with available finance as well as customer demand and expectations. By ensuring cost efficiency and long-term serviceability of the infrastructure, demand orientation and environmental health, a comprehensive scaling up approach successfully links infrastructure to sustainable sector development in general. Joint action on the part of financial and technical cooperation is critical to combine the funding of last mile investments with the conceptual and capacity development that is necessary to scale up services.

Social Enterprises and Inclusive Businesses

GIZ, “Grow – Scale – Impact: How to help inclusive businesses achieve scale.” Report, 2016. http://www.gsen.global/wp-content/uploads/giz2016-0310en-inclusive-business-scale.pdf

The objective of this handbook is to provide bilateral private sector development programs and other practitioners with a user-oriented, simple and practical handbook on how to support the scaling up and replication of inclusive businesses, based on best-in-class know-how, tools and methods.

The Practitioner Hub for Inclusive Business, “Scaling & Replicating Inclusive Business Models.” Blog, November 2016.

http://www.inclusivebusinesshub.org/wp-content/uploads/2016/04/Scaling-Inclusive-Business-Practitioner-HubNovember-2016.pdf

Reaching scale is a difficult proposition for any business, but even more so for those negotiating the unique challenges of working at the Base of the Pyramid. This blog focuses on what exactly does scaling an inclusive business entail? It explores key ingredients of a pathway to scale, debates and new ideas on replication, and looks at what small companies, large companies and ecosystem actors can do.

Evaluation

J-PAL, “Court Mediation Program Being Scaled Up to Improve Access to Justice in Mexico.” Blog, 2016 https://www.povertyactionlab.org/news/court-mediation-program-being-scaled-improve-access-justice-mexico

The Mexico City Labor Court is collaborating with J-PAL affiliated professors, through support from J-PAL’s Government Partnership Initiative (GPI), to scale up results from an evaluation of a court mediation program designed to improve citizens’ access to justice. This randomized evaluation, funded by an earlier grant from GPI and conducted by J-PAL affiliate Enrique Seira (Instituto Tecnológico Autónomo de México) along with Joyce Sadka (Instituto Tecnológico Autónomo de México) and Christopher Woodruff (University of Oxford), found evidence suggesting that informing people of the outcomes of court cases similar to their own increased use of mediation and settlement, leading to faster results and, over time, reductions in the court’s backlog. A new GPI grant enables the court to leverage technical assistance from Professor Seira, his co-researchers, and J-PAL staff in Mexico in the first phase of scale-up, targeting five of the twenty sub-courts.