World Development Report 2018: Learning to Realize Education’s Promise

Why does the learning crisis persist? How can children attend school for years but remain functionally illiterate? Why don’t the people in education systems fix this? One big reason is that, for many, the learning crisis is invisible. Education systems have little systematic information on who is learning and who is not. As a result, it is impossible to generate an impetus for action—let alone a plan.

To tackle the crisis, it is necessary—though not enough—to measure learning. But learning metrics must facilitate action, be adapted to country needs, and consist of a range of tools to meet the needs of the system, including at the classroom level.

The learning crisis is often hidden—but measurement makes it visible

“Almost no low-income countries have standardized (equated over time) national assessment systems to track learning and provide a feedback mechanism to national education policies and programs” (Birdsall, Bruns, and Madan 2016, 2).

Education systems routinely report on enrollment— but not on learning. Because learning is missing from official education management data, it is missing from the agendas of politicians and bureaucrats. This is evident in how politicians often talk about education only in terms of inputs—number of schools, number of teachers, teacher salaries, school grants— but rarely in terms of actual learning. Lack of data on learning means that governments can ignore or obscure the poor quality of education, especially for disadvantaged groups.

Without objective information on learning, parents may be unaware of the poor quality of education. This prevents them from demanding better services from schools and governments. In Kenya, one study found that less than half of the children in grade 4 could pass basic proficiency tests in literacy or numeracy, yet more than two-thirds of adults were broadly satisfied with the government’s performance in education. The realization that learning outcomes are poor may come only when children face poor labor market prospects, but by then it is too late. If parents have no real information on how much (or little) their children are learning, how can they hold schools or governments accountable?

Without clear information on what students do not know, how can schools improve instruction? Teachers may find it hard to judge to what extent students understand what is being taught. This is particularly true in low-income countries, where teachers face large classrooms that mix students of very different abilities. For example, a study from Delhi, India, found that the same grade may contain students whose achievement level spans the equivalent of five to six grades. In such contexts, learning measures provide teachers with timely feedback about which students may need additional support. More broadly, these measures provide school management with information about which areas need attention to improve instruction. If the information is shared with parents or students, it can help them direct their own efforts toward improving learning.

Yet concerted action is often derailed by concerns about the possible pitfalls of learning metrics. These metrics generate much debate on, for example, the outsize impacts of international assessments on local policy, the limited use of national assessments for improving classroom practice, or the potential gaming of high-stakes testing. But measurement of learning is not shorthand for international testing such as the Programme for International Student Assessment (PISA) or for the high-stakes accountability approach implemented through the U.S. No Child Left Behind policy. Instead, the term covers a range of assessments, including formative classroom assessments (box 4.1). Even in this form, measures of learning provide information on only some of the skills students acquire as they develop (see spotlight 3 on multidimensionality of skills). Thus metrics are complements of, not substitutes for, careful, context-specific analysis to determine how to improve learning.

Measures for learning guide action

Testing in Rio de Janeiro, Brazil, happens at two levels. First, every two years all students in grades 5 and 9 take a national test (the Prova Brasil) designed to assess public education. Second, students are tested at the end of each two-month curriculum block. These tests, given by municipal education departments, aim to provide quick feedback to teachers and principals, allowing schools and the broader system to provide more support to struggling students (Elwick and McAleavy 2015).

Identifying learning gaps in the classroom is the first step toward resolving them. In environments of low learning, there is often a gap between the level of students and the level at which classes are being taught. This might be because teachers are unaware of students’ levels. Fostering a culture of classroom-based assessments can address this problem. In Singapore, students are given screening tests at the start of grade 1, which helps teachers identify those who require additional instruction to learn to read.

Learning metrics help highlight where support is most needed. School districts and schools are then better able to target resources to improve service delivery. In Brazil, national assessments have been widely adopted by states and municipalities to strengthen school performance. Learning metrics have also guided big-banner education reforms. In Chile, PISA’s reading framework guided national curriculum reform. Similarly, findings from the Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ) I, 1995–99, underpinned a review of Mauritius’s education master plan. In some cases, learning metrics have been instrumental in making education reform data-driven. In Germany, lower than expected results—especially for students from poorer backgrounds—on the 2000 PISA led to the development of more support for disadvantaged students, especially those from immigrant backgrounds.

For learning metrics to guide action effectively, they need to be used as a range of tools to serve different needs, from classroom practice to system management. Measures of learning come in various forms, with different measures serving different purposes for different actors. These range from simple oral questions posed by a teacher to national assessments that help policy makers prioritize action (box 4.1). In well-functioning systems, these different tools complement one another to form a coherent whole.

Policy makers should rely on a broad range of information instead of any one measure. When a single metric becomes the sole basis for big policy triggers, the corresponding stakes may become dangerously high. A striking example is the U.S. No Child Left Behind policy enacted in 2001. This policy had strong negative repercussions for schools that performed poorly on annual statewide standardized tests. Though the policy led some poorly performing schools to improve, it also generated various undesirable strategic responses by teachers and school administrators. These included reclassifying students as requiring special education, exempting certain students from testing, reallocating resources to students at the margin of passing, and suspending low-scoring students near test dates.Even in the case of PISA, some studies have suggested that the performance in some places—Argentina, Malaysia, Vietnam, Shanghai (China)—could be tied in part to (perhaps inadvertent) “selective samples” that may exclude some poorly performing schools or students.

Education systems also routinely underuse the information generated by learning metrics—making for a lot of measurement that leads to little action. Often, findings are simply not communicated in a timely way to relevant audiences. There may also be credibility issues. If teachers or schools do not feel heard or acknowledged in a national assessment process, they will likely reject its findings. For example, teachers are more likely to resist quantitative forms of evaluation when metrics do not take into account context. This is particularly the case for measures of learning disseminated as rankings, which are susceptible to being taken out of context. In some education systems, such friction is heightened by the use of technology, which raises questions about privacy and transparency. Approaches using technology also involve limited social interaction, which is associated with less impact.

For measurement to guide action, it must be actionable. It also needs to be available to stakeholders. At the design stage, stakeholders have to ask themselves how learning data will be used. In Chile, all students in grades 4 and 8 take the Sistema de Medición de la Calidad de la Educación (SIMCE) each year. After the test identifies the 900 schools scoring in the lowest 10 percent on the tests in their province, these schools receive special resources. The data, then, are clearly linked to action. Many assessment systems measure outcomes too infrequently or too broadly to be of practical use. The most recent publicly available data from the SACMEQ are for 2007. Another constraint is the lag between when data are collected and when they are made available, as well as how data are made available. Many ministries produce only hard copies of summary reports, which make them difficult to use.