Scaling up successfully: Lessons from Kenya’s Tusome national literacy program

Our intention in this analysis was to apply a framework to determine whether key elements of education system institutional capacity were contributing to the successful large-scale implementation of a particular set of instructional changes. Kenya’s Tusome program provided a relevant example because it represents a fully national-scale implementation of a successful pilot program aimed at improving the teaching and learning of reading in English and Kiswahili. The Crouch and DeStefano (2017) framework, when applied to Kenya, enabled us to recount how changes in three core education system functions were contributing to the ability of the education system in Kenya to ensure instructional improvement. The first three research questions we posed at the beginning of this paper led us to investigate whether Tusome had reinforced system capacity to

• set and communicate expectations;

• monitor and hold schools accountable for meeting expectations; and

• provide basic inputs, intervene to support students and schools that were struggling, and hold the system accountable for delivering those inputs and supports.

The analyses presented in this paper allowed us to answer RQ1, and we found that expectations had successfully been created for the acquisition of specific literacy competencies in the first years of primary school. Expected outcomes for students had been codified in the form of benchmarks for oral reading fluency and comprehension in English and Kiswahili that students should achieve by the end of Classes 1 and 2 (Piper et al. 2016b). In addition, the learning materials used in Tusome were structured around a sequence of lessons designed to develop those competencies (Mourshed et al. 2010). The evidence suggests that expectations were created for how teachers would follow that sequence of lessons by using the teachers’ guides for their classes (Piper et al. 2017b). Teacher training and the materials provided to teachers and students communicated those instructional and curricular expectations. However, we found little evidence of whether and how these expectations were communicated to teachers outside of Tusome-specific activities, although anecdotal evidence gathered by researchers and Tusome staff suggested that it did happen intermittently.

With respect to the second research question on whether and how the Tusome reform was able to monitor and hold schools accountable, we found positive results. The regular visits by CSOs reinforced teachers’ adherence to the sequence of lessons in the teachers’ guides and generated data on student learning progress. It is becoming increasingly clear for all involved that progress in delivering lessons and progress in student performance are important initial indicators for the system. Somewhat surprisingly, we found less evidence that the quality of the CSO feedback made a difference in the implementation of Tusome in classrooms (Zuilkowski and Piper 2017). We did find examples where low aggregated fluency scores from CSO visits were used to create specific interventions for particular teachers or schools. Enabling CSO visits to include structured feedback to teachers relative to expected instructional practice and having those visits generate student fluency data are two examples of how the Ministry of Education’s institutional capacity was being developed to monitor and hold schools accountable—a positive indication of progress toward the second of the core functions undertaken in Tusome.

Interestingly, school- and student-level accountability under Tusome is not typically expressed through sanctions imposed for not meeting expectations, for either individual teachers or CSOs. We hypothesize that the accountability is being expressed in terms of a change in organizational culture rather than a more stringent punitive accountability model. Elmore (1996), reflecting on why incentives alone do not lead to dramatic uptake of instructional change in education systems, reasoned that unless the prevailing social norms under which teachers operate are also changed, the incentives will not induce the desired behaviors in ways that economists might predict. Having CSOs (and their head teachers at their schools) visit and conduct structured observations of teachers’ classroom instruction on a dramatically more frequent and regular basis represents a profound change in the prevailing norms under which teachers and education officials typically work in sub-Saharan Africa. For years, if not decades, schools and teachers have operated in virtual isolation from the education system—not receiving classroom support (Zuilkowski and Piper 2017) or system- or county-level oversight and, therefore, accepting the prevailing norm that they are not actually expected to perform their job up to any reasonable standard (such as, most fundamentally, showing up regularly to teach). Early signs in Kenya suggest that the systems for regular monitoring introduced by Tusome are leading to this type of normative shift, even without hard negative consequences (or, conversely, remunerative rewards) being tied to the outcomes of the CSO monitoring data.

Our analysis of RQ3 showed that providing support to schools and teachers that is more responsive to their expressed needs is a higher-order capacity, which the education system in Kenya has only begun to manifest. On the positive side, we found that training content was taking into account what coaching visits to schools revealed about teacher take-up. However, support resources—whether in the form of more attention from CSOs or other parts of the system or in the form of compensatory allocations that would enable schools to add staff, materials, or supplementary instructional opportunities—were not being directed based on what data were revealing about school, teacher, and student performance. Such an approach to the provision of support would require the education system to accept that the pursuit of equitable outcomes demands inequitable allocation of support resources. This approach would also oblige the system to have the data management capacities to consistently review the findings of the data (which appeared to be the case in many counties in Kenya) and to also apply resources to respond to those findings. The second aspect of this balancing act remains elusive in Kenya, but the potential is at least there.

The changes in system management that Tusome has been able to support could be perceived as limited in that they focus on adherence to what Fullan (2016) may characterize as an overly prescriptive pedagogical approach. Indeed, the basic inputs and instructional changes (i.e., use of structured teachers’ guides) supported at scale under Tusome fall short of the “deep change” exemplified by the dispersed and collective engagement in the construction of innovation at the local level in programs such as Escuela Nueva or the Learning Communities Project discussed in the special issue of the Journal of Educational Change (Fullan 2016). However, one can consider that in a large education system, such as in Kenya, shifting the culture to expect the provision of basic inputs, the use of available class time to assure the delivery of basic instruction, and the provision of regular follow-up and monitoring could be an important first step in achieving deeper, more collective and, thus, more sustainable improvement. The authors’ experiences suggest that focusing on specific instructional strategies is what enables the cultural shift to take place. Continuing to monitor and learn how a large system such as Kenya’s moves from being able to deliver a structured and somewhat “prescriptive” model to delivering one that is “deeper” would contribute to further understanding how bureaucracies can come to emulate more dispersed, collective, and networked systems [such as argued for by Pritchett (2013) and described as a “starfish”].

Our fourth research question investigated the combined ability of the Kenyan system to provide these three core functions and its impact on learning outcomes. The results showed that Tusome’s impact was large and meaningful. The gains in Tusome after 1 year exceeded both the gains made in other neighboring countries over several years and those of the preceding pilot programs, particularly the PRIMR RCT research used to design Tusome (Freudenberger and Davis 2017). The effect sizes, which were greater than 0.7 SD for both English and Kiswahili at the national level, were remarkable in terms of both their magnitude and their equitable distribution. Tusome impacts on the ultimate outcome of reading comprehension were somewhat smaller than in the other tasks, and somewhat lower in English than Kiswahili. This suggests that careful consideration should be given to the expansion of oral language and vocabulary skills in a language that is a second language—or even more frequently, a third or fourth language—for most children in Kenya (Piper et al. 2016d).

The external evaluation results showed that the effects were relatively stable across the four lowest wealth quartiles and that the causal gains were similar in the low-cost private schools (i.e., APBET institutions) and public schools that were part of the sample (Freudenberger and Davis 2017). Note, however, that the PRIMR RCT research had treatment and control groups, whereas the Tusome external evaluation results presented in Freudenberger and Davis (2017) simply compare year-on-year gains without a counterfactual. Even with this caveat, the gains were remarkably large.

Compared to the impacts identified in a study of 12 separate RCTs in Uganda looking at the impact of literacy improvements over a period of either 1, 2, or 3 years, the impacts of Tusome in Kenya were larger than those for any of the languages in Uganda (Brunette et al. 2018). They also were larger than those achieved in the national program in Rwanda (Education Development Center, EDC 2017) and under similarly designed pilot programs in South Africa (Fleisch et al. 2016; Taylor and von Fintel 2016). The fact that the gains in Tusome were achieved at the national level is striking.

The external evaluation did not involve the qualitative research necessary to produce a detailed set of explanatory characteristics for these meaningful impacts, but it did show that Tusome’s provision of student books, teachers’ guides, teacher professional development, and classroom support was similarly high across the country. It also showed that the teachers perceived the program as successful, and the data from classroom observations revealed a meaningful percentage of teachers progressing through the sequence of lessons at a pace consistent with almost daily adherence to the program in both languages. The combination of the institutionalized capacity mentioned above with materials and training (which benefited from several rounds of revisions because of the PRIMR experimental pilots implemented since 2011) appears to have positioned Kenya to see large effects at scale. Although the external evaluation results did not include the data needed to test this proposition, national-level ownership, from the Presidency to the Ministry to the county-level leadership, appeared to be important. Additional research should examine these conditions in more depth to reveal how this ownership and leadership were experienced in local settings in Kenya and, thus, to understand the relationship between that local level decision-making and the assurance of the core functions. In addition, more implementation analyses are needed in this sector. Indeed, the Tusome example shows that understanding and focusing on teacher behavior change, the dissemination of changes in the normative environment for teachers and administrators, and increased accountability at the local level are essential elements of successful large-scale implementation (Datnow and Park 2009).