World Development Report 2018: Learning to Realize Education’s Promise

Kenya’s government discovered just how difficult it is to turn successful small-scale interventions into systemwide improvements in learning. In the late 2000s, even though access to primary schooling was high, many children failed to acquire even basic skills. The government argued that large classes, with their overburdened teachers, lay behind these disappointing results. But a constrained education budget meant that hiring more civil service teachers to address these problems was not an option. Instead, in 2009 the government hired 18,000 temporary contract teachers. The new program shared many of the same features of an earlier pilot experiment by a nongovernmental organization that provided government schools with contract teachers. The pilot reduced class sizes, leading to improved learning outcomes for students taught by the new contract teachers. Moreover, these gains were achieved at a cost well below the cost of the alternative of hiring more civil service teachers.

But unlike the pilot intervention, the government program failed to deliver any improvements in learning. A combination of union resistance and lack of Ministry of Education capacity to manage contract teachers underpinned the program’s lack of impact. The Kenyan teachers’ union successfully challenged the program in the courts, arguing that hiring teachers on a contract basis violated constitutional rights to equal pay for equal work. The ruling led to guarantees from the government to gradually absorb all contract teachers into the civil service and provide them with the same employment protections. These developments significantly changed the employment prospects of contract teachers. In particular, they weakened the link between performance and the chances of contract renewal—the main channel through which the original trial had improved student learning. At the same time, the ministry also struggled to implement the program. Governmentemployed contract teachers were paid on average three months late, hurting student learning.

This example illustrates a more common finding that working at scale is not the same as “scaling up.” Similar difficulties in changing teacher employment conditions in government schools have occurred in many other countries, despite evidence from pilot programs showing their potential to improve learning. These examples show that implementing interventions at scale can also induce responses from other actors or parts of an education system that can alter the potential impacts on learning.

In many countries, education systems suffer from two related weaknesses. First, systems are not well aligned with the overall goal of learning; other goals can detract from, and in some cases compete with, efforts to improve learning outcomes. Second, the elements of an education system are often incompatible or incoherent. For example, government funding allocations sometimes fail to provide the resources schools need to improve learning. Even when school funding is available, the rules governing its use often leave little flexibility for schools to use it in ways tailored to the specific needs of students.

Technical and political factors underlie these system weaknesses. Getting all parts of an education system to work together is difficult, and the agencies responsible for designing, implementing, and evaluating education policies often lack the capacity to take on this role. For example, timely information on student learning outcomes is not available in many low-income countries, making it harder to design appropriate interventions and to monitor their effectiveness. The interests of system actors can also contribute to misalignments. For example, calls to devolve control over resources to schools are sometimes resisted because private textbook providers fear losing out on lucrative centralized contracts.

Failure to tackle these technical and political constraints can trap countries in a low-learning, low-accountability, high-inequality equilibrium. When different parts of a system fail to work together, education outcomes will fall far short of what is possible. When actors in the system interact to pursue many goals, the mechanisms that hold them accountable for learning are weakened. And where powerful groups can divert resources to align with their own interests, education systems can exacerbate inequalities. Together, these factors can pull an education system out of alignment with the overall goal of learning (figure 9.1).

Misalignments and incoherence impede learning

Taking a systems approach can help to identify the elements that are incoherent with each other or misaligned with learning (box 9.1). Though every education system faces its own challenges, incoherence and misalignments tend to occur across four elements:

• Learning objectives and responsibilities. Clearly articulated learning goals are often missing. But even when they exist, the roles and responsibilities of different system actors in achieving them are unclear, resulting in limited accountability.

• Information and metrics. Accurate, credible information on learning is often unavailable. This can divert attention from learning and hinder monitoring and evaluation of interventions aimed at improving outcomes. • Finance. Education funding is sometimes inadequate and often allocated in ways inconsistent with a goal of providing equitable opportunities for effective learning. • Incentives. The motivation and incentives of system actors are often only weakly linked to student learning.

Though most education systems recognize learning as a central goal, it often receives less prominence than other objectives. Looking beyond high-level policy documents often reveals the objectives that matter most in the day-to-day affairs of education agencies. Bangladesh has made progress in linking education sector objectives explicitly to government budgets—for example, budget documents link allocations to specific activities aimed at improving education outcomes. However, the government’s key performance indicators mostly deal with access and completion; only 1 of the 12 indicators targets learning. Moreover, that indicator tracks literacy rates in the population over age 15, which is insensitive to changes in school performance over the medium term.

Even where learning is a clear goal, the way education systems are organized sometimes hampers performance. Because tasks are often fragmented across education departments and government agencies, it can be hard to identify who is accountable for outcomes. In Romania, responsibilities for textbook provision were split among four different agencies, yet none of them was solely responsible for ensuring that schools received the right books. Delivering early childhood development services typically requires coordination among several government agencies, including health and education ministries. Managing these many agencies is challenging. In the early 1990s, constant shifts in responsibility for early childhood development in Ghana resulted in inadequate stewardship of these services.

Systems often lack the information needed to support the design and implementation of reform. Education management information systems cover a wide range of indicators on service delivery, but in many countries they do not routinely include data (figure 9.2). Public expenditure reviews and other on learning. India’s District Information System for studies reveal similar patterns across subnational

Education (DISE) is designed to provide report cards administrations and even across schools (spotlight 6).

for districts, but of the 980 data points reported, none

The weak link between spending and learning is a covers student learning. That omission can make it feature of the different environments in which educadifficult for systems to track interventions to improve tion systems operate. Systems with higher corruption learning, for parents to demand better services from or lower bureaucratic quality are less likely to use politicians or directly from schools, and for agencies resources effectively to raise learning.

to design effective policies to improve learning.

These simple correlations also suggest that many education systems are delivering learning outcomes well below what is possible given current levels of

Public spending does not correlate strongly with funding. In India, excess teacher absenteeism in the learning. The link between spending and learning public sector is estimated to cost US$1.5 billion a year.

differs enormously, even among countries at similar

If teacher accountability systems were more strongly levels of economic development. In 2015 Peru spent aligned with learning, teacher attendance would

28 percent less per student than the Dominican improve, allowing the system to achieve higher levels

Republic, but it had Programme for International of learning at the same cost.

Student Assessment (PISA) mathematics scores that

Improvements in learning are unlikely when addiwere more than half a standard deviation higher.

tional resources are allocated like past funding. The

More generally, cross-country correlations between composition of education spending in many counpublic spending and learning levels are weak and tries is suboptimal. Funding for teacher salaries often statistically insignificant after controlling for income absorbs more than 80 percent of education budgets in per capita. Moreover, for any given level of spending low-income countries, leaving little room for spendthere is a wide range of outcomes. Even changes in ing in other areas. Using additional funding to shift public education spending over time sometimes spending patterns to ensure that teachers have the result in unexpected outcomes. For example, Bulgarcomplementary inputs needed—such as textbooks ia’s PISA mathematics scores increased between 2009 and in-service training—would improve alignment and 2015, despite reductions in spending per student and significantly aid learning.

Education system actors face many incentives, but only some of these incentives are aligned with learning. System actors are motivated by a range of factors that affect how they carry out their duties. Professional rewards—the social status afforded to their occupation, the ability to develop new competencies, intrinsic motivation—are all important factors driving behavior. Financial rewards and accountability mechanisms, such as feedback from parents or from managers, can also affect how system actors perform. Though some of these factors that motivate system actors are aligned with learning, some are not. For example, salaries and career progression are often determined largely by a combination of qualifications and experience, despite these characteristics having only a weak relationship with learning. Even where countries have invested in mechanisms to evaluate teacher performance, those mechanisms are often disconnected from decisions on professional development. Edo State in Nigeria conducts annual performance evaluations, but these evaluations do not affect decisions on teacher promotions, nor do they lead to sanctions or rewards for teachers based on their performance.

Ensuring that the parts of an education system work together is as important as ensuring alignment toward learning. Even if a country has prioritized student learning, established reasonable learning metrics, and aligned funding with incentives, it still needs to ensure that system elements are coherent (box 9.2). If a country adopts a new curriculum that places greater emphasis on active learning and creative thinking, the curriculum alone will not change much. Teachers need training so that they use more active learning methods, and they need to care enough to make the change—given that teaching to the new curriculum could be much more demanding than old methods that favor rote learning. Even if teachers are on board with curriculum reform, students and their families could weaken its effects if an unreformed examinations system creates misaligned incentives. In the Republic of Korea, efforts to introduce a more student-centered curriculum—one that encourages greater creativity—have sometimes conflicted with pressure on students to succeed on the all-important university entrance examinations.

The need for coherence between different parts of an education system makes it risky to borrow from other countries. Education policy makers often scrutinize higher-performing systems to identify what they could borrow to improve learning outcomes in their own systems. Indeed, the search for the secret ingredient behind Finland’s record of learning led in the 2000s to a swarm of visiting delegations in what has been dubbed “PISA tourism.” Finland’s system gives its well-educated teachers considerable autonomy, so they are able to tailor their teaching to the needs of their students. But lower-performing systems that simply import Finland’s teacher autonomy into their own contexts—contrary to the advice of Finnish educators who emphasize coherence—are likely to be disappointed. If teachers are poorly prepared, unmotivated, and loosely managed, then giving them greater autonomy will likely compound the problem. South Africa discovered this in the 1990s and 2000s, when it adopted a curriculum approach that set objectives centrally but left implementation up to teachers. The approach failed in many schools, in part because it proved to be a poor fit for the capacity of teachers and the resources they had at their disposal.This example illustrates why coherence between different system elements and the development of home-grown solutions are so important.

Technical complexities make it hard to align education systems with learning

Every day 23 million children—a fifth of the population—attend one of the 47,000 public elementary and high schools in the Philippines. When their parents are included, about two-thirds of Filipinos interact with the school system on a regular basis.

The national government in Manila manages the system through a network of more than 200 division and 2,500 district education offices. These offices oversee over 600,000 public school teachers, or more than 40 percent of the public sector workforce. Even routine tasks involve coordination between many parts of the system. For example, management of public school operational funds relies on student data from the central office. Once schools have their allocations, they issue about 500,000 checks and generate as many spending reports, each detailing individual spending items. The monitoring of these financial flows alone puts a significant strain on the system, even though they account for less than 5 percent of government education spending.

Three characteristics of complex education systems magnify the technical challenges of managing them. First, systems are opaque. Many of the goals pursued by these actors are hard to observe, as are many of the interactions among the actors, whether they take place in the classroom or in the bureaucracy. Second, systems are “sticky”: reforms to improve learning are hard to launch, and they take time to bear fruit. Third, implementing reforms successfully requires capacity that many bureaucracies lack.

Education systems typically have a range of goals, including equipping students with the skills needed for the labor market, advancing social equity, and teaching children the norms, beliefs, and histories of their community. But education systems can have other goals that can hamper efforts to improve learning. For example, politicians sometimes view education systems as a tool for rewarding their supporters with civil service jobs, or for impressing voters with school construction programs that are visible but not strategically planned. These goals can be misaligned with learning, leaving schools with buildings they cannot use and teachers who are not proficient. Where these goals compete with other goals, the result is that the overall education system and its actors are not aligned toward learning.

Managing the system to improve learning is difficult. Promoting learning in the classroom involves significant discretion for teachers, who must use their professional assessment to tailor their teaching to the needs of their students. Teaching also involves regular, repeated interactions between students and teachers over a relatively long period. These characteristics— coupled with a dearth of information and metrics on student outcomes at the school level—make it hard to manage and monitor learning. These challenges may be exacerbated if private schools are a major player, because those schools typically operate outside the direct control of the public system (box 9.3).

Some things are easier to monitor. School building and cash transfer programs are highly visible and easily monitored investments aimed at expanding access. By contrast, investments to raise teacher competence or improve the curriculum are less visible, and monitoring their impact on student learning is more difficult. Such challenges can sometimes prompt education systems to emphasize improvements in access over improvements in quality. Even when systems to monitor student learning are effective, they can linear relationships from cause to effect. The multiple sometimes lead to biases toward better-performing interactions that characterize teaching and learning students, short-term test preparation, or a narrow and the almost continuous feedback that they provide focus on subjects that are explicitly tested.

can result in teachers, parents, and students adapting

The multiplicity of actors and institutions in an their behavior in unpredictable ways. For example, education system makes the outcomes of efforts to the introduction of school grants in Andhra Pradesh, improve learning unpredictable. Learning is a comIndia, and in Zambia failed to improve student learnplex process that is difficult to break down into simple ing in the long term because parents reduced their financial support in anticipation of the increase in government funding. Reducing the financial burden on parents may be a desirable effect of these grants, but it was not their primary intent. More generally, many factors outside the classroom and the school system, including health and economic shocks, can alter the impact of interventions aimed at improving learning. Failure to learn and adjust policies in response to such changes often means that interventions do not work as planned.

Education systems are slow to change. Some of the best-known successes in reforming systems, such as in Chile or Finland, took decades from initiation to fruition. Even at the micro level, such as in schools in the United States that enacted comprehensive school reform, it took 8–14 years for the full effects to be felt. These long time frames present two further challenges to better aligning education systems with learning. First, to improve learning, policies usually have to remain relatively consistent. This is difficult under normal circumstances: changes in government, volatile funding, and shifts in the overall economic context all threaten the sustainability of policies.But staying the course is even more challenging when the reforms fail to show any benefits in the short run. Second, the long lags make program evaluation more difficult, because attributing improvements to specific interventions is especially challenging when their impacts emerge only in the long run.

Opacity and stickiness make technical alignment hard enough to achieve; weaknesses in implementation capacity make the task even more daunting. Successful implementation depends on effective leadership, coordination between education agencies, and implementation teams that are motivated, use resources efficiently, and can troubleshoot in real time—all of which are in short supply in many systems. Moreover, behavioral economics highlights many cognitive pitfalls that policy makers commonly face in complex operating environments. These include difficulty in evaluating policy effectiveness when faced with too many options; loss aversion, or the tendency to feel failures more intensely than successes, which makes policy makers wary of experimentation; biases that lead to selective use of information to reinforce existing views; and relational bias, which makes it harder for officials with elite educational backgrounds to grasp the challenges of mass education.

Education agencies often lack the capabilities needed to deal with these complexities. A recent assessment shows how multitasking and fragmentation within education agencies can blur lines of accountability for learning. In Cyprus, because of the absence of a department for human resources and general administration, pedagogical departments had to manage these responsibilities, diverting time from developing programs and policies. Public expenditure and financial accountability assessments also highlight the low capacity in many developing countries in key areas. For example, only about half of the 72 low- and middle-income countries assessed since 2010 had any system in place to ensure that resources intended for schools, health clinics, and other service delivery units reached the front lines.

Technical challenges and lack of implementation capacity result in misaligned education systems. When countries are unable to overcome these challenges, their education systems deliver levels of learning far below what is possible. But tackling the technical barriers to better learning is only part of the battle. To break out of low-learning equilibriums, countries must also address the political constraints that are often at the heart of these technical misalignments.