World Development Report 2018: Learning to Realize Education’s Promise

Tip 1: Measure gaps. The learning crisis will be truly salient politically only when vulnerable subpopulations, who are disproportionately likely to suffer from learning gaps, are adequately covered by national assessment systems. To ensure that happens, assessments should be deployed in a way that shines a light on all children. Measurement must allow for the disaggregation of data around important dimensions such as socioeconomic status, gender, location, or disability status. In particular, groups at risk for social or economic exclusion may need to be oversampled to ensure adequate representation.

Tip 2: Track progress. The use of uniform methodologies, approaches, and psychometrics across years is crucial for education systems to discern trends in learning over time and changes in learning gaps across tests. Year-on-year comparisons of learning progress should also be ensured for vulnerable subpopulations.

Tip 3: Test students when effective action is still possible. Returns from student assessments will be maximized if they focus on ensuring that students attain basic skills—literacy, numeracy, critical thinking—early in their schooling. Systems should also consider household-based testing, which would allow assessments to cover students not currently in school, making the resulting measurement more useful for universal learning targets. Household testing would also allow more nuanced understanding of all the different influences on a child’s school access and learning outcomes. To that end, standardized learning modules can be included at little additional cost in surveys conducted both nationally (such as income and consumption surveys) and internationally (such as Living Standards Measurement Study surveys or Demographic and Health Surveys).

Tip 4: Balance the stakes. No single measure should be misused or overused. One way to avoid that outcome is to frame learning measures that guide policy as low-stakes diagnostic tools—not as one summary number that determines sanctions and rewards. Again, “learning metrics” should be considered a system of tools, each with its own place and purpose.

Tip 5: Good design is not enough—facilitate action. Learning measures should be used explicitly not just for tracking progress, but also for policy making. One way to ensure that happens is to devote resources (including effort) to the timely distribution of understandable results to key stakeholders. Another factor is an open, collaborative process for instrument design. Student assessments developed with the collaboration of various stakeholders are more likely to be considered valid and relevant at local levels.

Tip 6: Exploit global public goods on learning. Leveraging international assessments can yield high returns. For example, there is considerable advantage to forging common links between international and regional assessments so they can be put on the same scale. This not only increases harmonization between international assessments such as PISA and TIMSS, but also allows ties to national and citizen-led assessments, enabling meaningful global tracking (box 4.2). Researchers have tried to link various assessments after the fact, but these attempts have faced severe technical challenges. Ex ante linking of measurements through common items is likely to prove much more technically sound and cost-effective.

Education systems are unlikely to tackle the learning crisis unless it becomes clearly visible. This is possible only through well-designed measures of learning. To be effective, “learning metrics” must overcome two important challenges: ensuring that information leads to action, and minimizing the potential perverse impacts of measurement. Alarm at the rise of a “testing” culture has dominated recent discourse. But in most low-learning contexts there is too little assessment and, consequently, too little accountability for learning in the system.