World Development Report 2018: Learning to Realize Education’s Promise

“Shock as 60 [Percent] of Tanzania Students

Fail National Exam” (East African, 2013)

In the United States since 2001, information on different schools’ performance on standardized tests has notably increased turnout in local school board elections (Holbein 2016).

Measures of learning motivate action through three channels:

• Participation. Learning outcomes are often far worse than stakeholders realize. In Uganda, nearly three-quarters of parents said they were satisfied with the quality of education—yet only a quarter of grade 4 students could pass a math test based on grade 2 questions. By documenting service delivery shortfalls, learning metrics can motivate parents to hold their schools accountable for learning.

In such contexts, learning metrics can correct information failures, which are especially severe for the poor. This correction can in turn rebalance the relationship between users and providers. This channel operates via the direct or short route of accountability running from parents directly to schools. • Choice. Providing parents with hard evidence about learning outcomes at alternative schools can encourage schools to improve learning by increasing competitive pressures. When parents have objective information about learning outcomes across schools, they can punish poorly performing schools by “voting with their feet.” Public schools care about such outcomes because their resources are often tied to the number of students they enroll. But this channel may also disproportionately penalize schools that serve poor children. • Voice. Learning metrics can facilitate lobbying for reform by providing information on what needs fixing. Lack of reliable metrics, by contrast, undermines accountability for results. This channel operates via the long route of accountability, where learning metrics may help citizens use the political process to hold politicians accountable for learning.

That said, the links from measurement to action are neither automatic nor straightforward. India’s citizen-led assessment, the Annual Status of Education Report (ASER), has documented low proficiency scores since it was introduced in 2004. However, clear or sustained improvements are not yet visible for the country as a whole. At the same time, some Indian states have shown significant improvements in grade 3 reading levels between ASER 2010 and 2016. This shows that it is not just the information but action that matters. For learning to improve, not only do learning assessments need to be available, but also someone needs to act on them. In fact, an evaluation of the impact of citizen-led assessments in Kenya finds that for information on learning to spur action, those who receive the information must understand it, see it as actionable, care about the topic, and believe that their actions will improve outcomes.

Political pressures may limit the extent to which measures of learning spur positive action. Where education quality is low, politicians have an incentive to hide or obscure learning outcomes. They may also try to evade blame for poor performance by setting low standards, trying to limit year-to-year comparability, or restricting access to outcome information. For example, Argentina amended its standardized test so that year-to-year comparisons are not possible, decreased the frequency with which the test is administered, and delayed the publication of results by two years to obscure the poor performance of students. Teachers, too, might resist learning assessments to minimize opportunities for blame. In Chile, teacher training institutions have shown resistance to the national assessment. Assessments are also political because they can affect the flow of resources or prestige in an education system—as in the United States under the No Child Left Behind policy. Underlying politics can make student assessment systems particularly hard to reform (see part IV of this Report).

When does measurement mobilize citizens to demand accountability for learning? Because of limited attention, information is often ignored, especially if it is complex or provides unwelcome news. Therefore, for measurement to spur action, information must be available in an easily digestible way. But this in itself may not be enough. Learning metrics can galvanize communities to hold their schools accountable for learning only when collective action problems are resolved. A participatory approach—where schools and communities have a say in what type of “learning metrics” are generated at the school level—may be likely to work better here. In addition, for citizens to be able to act on information, fear of reprisals must be low. Finally, for citizens to act in behalf of change, they must believe that their own individual actions can make a difference.

Efforts to benchmark country performance through international or regional assessments have in some cases galvanized action because international comparisons make learning politically salient. Release of the Trends in International Mathematics and Science Study (TIMSS) or PISA rankings often triggers intense media interest, inserting learning into political and economic debates. This increase in interest often generates momentum for government action—an effect known as “PISA shock”—thereby unleashing targeted reforms. About half the countries participating in the PISA assessments under the aegis of the Organisation for Economic Co-operation and Development (OECD) have launched reforms because of the results. Learning assessments also spur action by making learning a tangible goal. Whereas the United Nations’ Millennium Development Goals (MDGs), which inspired efforts by governments and donors, focused on enrollment, the current Sustainable Development Goals (SDGs) place greater emphasis on learning. The success of the SDGs will depend on countries’ ability to turn rhetoric into action by tracking learning.

Choose learning metrics based on what the country needs

half of the countries surveyed produce data or participate in any regional or international tests to assess mathematics at the end of lower secondary school. Just under half assess reading. This means that comparable information about learning is missing

When choosing which measures of learning to invest for most children and youth outside of high-income in, policy makers must consider the context. If assesscountries (figure 4.1).

ment systems are nascent, priority should be given to fostering classroom assessment. Once that piece

Will learning metrics narrow the vision for education?

is in place, countries can develop relatively quick, sample-based, low-cost national assessments. When classroom and national assessments are established, much can be gained from participating in regional or

Putting emphasis on measurable learning does not global assessments that enable performance benchmean ignoring other outcomes of education, such as marking. The ultimate goal is to build assessment physical, moral, civic, or artistic development. Indeed, systems in which different parts are aligned but serve focusing on learning—and on the educational quality different needs.

that drives it—is more likely to crowd in these other

Not every student needs to be tested in national desirable outcomes. Conditions that allow children to assessments. Sample-based assessments can accuspend two or three years in school without learning rately measure a system’s performance. These to read a single word or to reach the end of primary assessments still require capable administrators, school without learning two-digit subtraction are but they are much less expensive than census-based not conducive to reaching the higher goals of eduassessments. They can also be administered more cation. An experiment in Andhra Pradesh, India, that often. Schools participating in these assessments do rewarded teachers for gains in measured learning in not have to be identified. This helps lower the stakes, math and language led to improved outcomes not just making the assessments less susceptible to perverse in those subjects, but also in science and social studresponses by teachers or schools.

ies—even though there were no rewards for improveAssessment systems should test students at an age ment in the latter two subjects. A study of ninth when effective remedial action remains possible. Of graders in the United States found that behavioral

121 countries in four regions, a third lack any reportfactors correlate positively with test scores. Another ing data on the reading and mathematics proficiency

U.S. study revealed that teachers who improve test levels of children at the end of primary school. Only scores also improve broad outcomes into adulthood.

Learning assessments of key foundational subjects such as language and mathematics are likely to be good proxies for whether an education system is delivering on its broad promise.

That said, cognitive skills are not the only skills that matter. Socioemotional skills (sometimes called noncognitive skills) such as grit, self-control, self-management, effective communication, and prosocial behavior can be central to not just economic outcomes but life outcomes more broadly. Evidence from high-income countries suggests that such skills strongly affect employment status, work experience, occupational choice, and wages. They also reduce risky behaviors such as crime, violence, or drug use. For example, a study from the United Kingdom found that even after controlling for cognitive skills, socioemotional skills were important for predicting whether individuals stayed in school, obtained a degree, were employed, smoked, or were involved in a crime. An understanding of how to measure these skills, along with how to influence them, is growing rapidly. Like cognitive skills, socioemotional skills develop early in life but are malleable. In fact, socioemotional skills help build cognitive skills and vice versa, with current skill levels dependent on investments made earlier in life (see spotlight 3).

Lower-performing countries probably do not face the same sharp trade-offs faced by high-performing countries on the education frontier. Economists use the concept of the production possibilities frontier to understand how producers—or in this case, countries—make trade-offs between production of different goods (figure 4.2). For example, in recent years many stakeholders in the Republic of Korea have argued that their high-performing education system places too much emphasis on test scores (shown in figure 4.2 as “measured learning”) and not enough on creativity or certain socioemotional skills such as teamwork (“other outputs”). Implicitly, this Korean debate is about whether to try to move up and to the left on the frontier—that is, from A toward B. But in the low-learning trap, represented by “low-performing country C” in the figure, there is so much slack that this OECD-driven debate is not relevant. Country C has an opportunity to improve on both measured learning and other education outputs at the same time.