From Pilot Programs to National Scale

Several countries have recently begun large-scale educational interventions to respond to low learning outcomes. The evidence used to justify national or large-scale programs typically has emanated from pilot initiatives that increasingly have provided rigorous evidence of impact on learning outcomes. For example, several recent meta-analyses showed a dramatic increase in results from pilot programs focused on improving learning outcomes while also using rigorous research designs (Conn 2017; McEwan 2015; Murnane and Willett 2011).

Although an increased dependence on causal evidence to justify large-scale implementation strengthens the research base for large programs, the decisions to take these programs to scale have largely been undertaken without a body of literature that examines the barriers to successful large-scale educational implementation or whether the external validity assumptions for these comparisons hold (Bates and Glennerster 2017). This failure to develop robust scale-up literature and practice has allowed the field of educational development to focus heavily on proof of concept, with randomized controlled trial (RCT) studies estimating the program impact of small- or medium-scale interventions in several contexts. We found a handful of recent studies and research that made explicit a model by which evidence from pilot programs should interact with scale-up designs. However, this evidence is too recent to influence the design decisions made by governments. Instead, based on the guidance of educational researchers, governments often depend on pilot evidence as sufficient for scale-up decisions, without undertaking a thorough investigation of the aspects of system capacity that will be most critical to producing similar impact at scale.