Scaling up successfully: Lessons from Kenya’s Tusome national literacy program

Background

In 2012, Kenya began implementing the PRIMR Initiative, which was the precursor to the Tusome intervention (2014–2019). PRIMR encompassed two separate research programs (one funded by USAID and the other funded by DFID Kenya), organized into a set of RCTs with various treatment groups to determine the most cost-effective means to improve early literacy and numeracy. The PRIMR model required that the actual training and classroom support be done by existing government officers and that research be undertaken to understand whether and how these officers would be able to accommodate the PRIMR activities in their daily work. This is an important consideration that many pilot programs do not take into account (Gove et al. 2017). Local government officers’ attention and time are sparse, as they face many competing demands. Many small programs that have had statistically significant effects have been unable to be scaled given inattention to this inherent weakness. The PRIMR design showed 0.2–1.0 standard deviation (SD) effects on literacy and 0.1–0.6 SD effects on numeracy (Piper et al. 2016c, 2018). The literacy results showed somewhat larger effect sizes for PRIMR’s impacts on reading skills than on comprehension outcomes, but PRIMR did achieve consistent improvements in reading comprehension in both languages. Furthermore, PRIMR’s findings indicated that coaches did improve the literacy program and that 15:1 was a more cost-effective school-to-coach ratio than 10:1 (Piper and Zuilkowski 2015); that learning impacts were possible after only 1 year (Piper et al. 2014); that the impact of PRIMR was sufficient to reduce the poverty gap (Piper et al. 2015a); that performance on reading assessments administered in mother tongues could also be improved, even without mother tongue instruction (Piper et al. 2016f); that the most cost-effective information and communication technology intervention was tablets for coaches (rather than for students or teachers; Piper et al. 2016a, e); and that a package of teachers’ guides and learner books was more cost-effective than programs that offered only training without these materials (Piper et al. 2018). To the credit of the Kenyan Ministry of Education, Tusome was designed according to the research evidence collected from PRIMR.

The Tusome program design was structured to implement the most cost-effective elements that PRIMR showed would be able to work through government systems. The learning materials (student textbooks and teachers’ guides) in Tusome were adapted and refined from PRIMR’s, the coaching model was the same, and the teacher training model was similar (although only 6 days per year were affordable at the national scale, compared with 10 days of teacher professional development under PRIMR). However, while PRIMR was able to provide consistent ongoing support to Curriculum Support Officers (CSOs)Footnote 3 implementing the program, the sheer size of Tusome precluded this arrangement. Therefore, whether Tusome would realize higher levels of take-up and implementation than found in other large-scale education projects was unclear. We examined Tusome’s scale-up implementation outcomes to determine whether the core functions of the education system enabled pedagogical change to be implemented consistently enough to produce widespread improvements in learning (Crouch and DeStefano 2017). The analysis presented below compares the impact of Tusome to that of the pilot program that preceded it and describes the conditions within Tusome that supported or hindered achieving results at scale.

It should be noted that Tusome is focused on a particular set of program elements that were determined at the national level and are being adhered to at the local level. Some might argue that such an approach is inadequate to describe the complex processes that occur in policy implementation (Datnow and Park 2009). For Tusome, given that the research used to design the program provided significant evidence that a successfully implemented version of the previously piloted program would be effective, this criticism might be less relevant. Better understanding how the Tusome initiative was received by local actors and actualized by individual educators far from the national-level policy maker is essential to fully understanding the impact of Tusome at scale.

The core functions framework

Pilot programs often achieve success on a small scale by intervening directly to change instruction in a given set of teachers or schools. System-level change requires identifying the mechanisms that will ensure that the majority of schools and teachers adopt the new strategy. Crouch and DeStefano (2017) argued that an education system’s set of core functions determine whether classroom practice can be improved at scale. These functions are essential to increase the likelihood of all schools succeeding in any new education reform effort. Their key contributions were the notions that the institutional capacity to implement varies in scaled programs and that the contours and content of that institutional capacity are “what makes an education system a ‘system’ and not just a collection of schools” (Crouch and DeStefano 2017, p. 8). These core functions are as follows:

• Setting expectations for the outcomes of education Decision-making at all levels should be driven by clearly articulated (and communicated) expectations for what students should learn and be able to do, for what teachers should do on a day-to-day basis, and for what others in the system should do to support teachers and students to achieve those expectations.

• Monitoring and holding schools accountable for meeting those expectations At the local or decentralized level, the education system should monitor whether expectations are being met. For example, are the basic inputs of learning materials and administrative requirements being provided to schools in a timely manner, and are schools and teachers using those inputs in ways that promote increased learning? Finally, are students demonstrating the expected levels of skill development?

• Intervening to support the students and schools that are struggling to meet expectations and holding the system accountable for delivering that support If schools and teachers are going to be evaluated and held accountable according to their ability to meet the expectations, the system must ensure that schools, teachers, and students have the basic inputs they need. Furthermore, the system needs to be able to intervene in response to any weaknesses that monitoring the schools reveals. Schools that are succeeding should be recognized, and those that are struggling should be given additional help in a timely and supportive way. Rather than administering the status quo of inequitable learning performance, this core function requires that systems recognize the need for and be able to deliver differentiated responses based on school and student performance.

The focus on education system capacity to assure these three core functions constitutes a deliberately limited framework. The setting of goals, provision of inputs, and monitoring of progress rely on a technical-rational perspective of policy development and implementation. Using this rationalistic framework allows us to examine ways in which the more complex features of education change are addressed in the case of Tusome in Kenya. Attempting to shift the normative culture of the education bureaucracy, and thereby changing the nature of the relationships among teachers, school directors, and their supervisors, presents an inherently complex web of challenges. Using a framework focused on a specific set of technical implementation challenges may provide some insight into what enables the more complex dimensions of system change to be addressed. However, it must be acknowledged that this perspective does not address all the socio-political and other contextual factors that should be taken into account to improve national education systems.

Research questions

Given the limited evidence regarding whether and how large-scale interventions are implemented through existing government education systems, we applied the Crouch and DeStefano (2017) core functions framework to answer the following research questions.

• RQ1: Has Tusome reinforced system capacity to set expectations for the outcomes of education?

• RQ2: Has Tusome reinforced system capacity to monitor and hold schools accountable for meeting these expectations?

• RQ3: Has Tusome reinforced system capacity to provide the inputs necessary to implement the reading program at scale, to intervene to support students and schools that are struggling to meet expectations, and to enforce accountability for delivering these types of support?

• RQ4: Given the ability of the system to fulfill its core functions, what has been the impact of Tusome on learning outcomes?