OECD Digital Education Outlook 2026: Exploring Effective Uses of Generative AI in Education

The paradox of generative AI in skill acquisition

Despite the promise of generative AI, its integration into education raises fundamental questions that demand critical scrutiny. As the capacity of AI systems to automate cognitive tasks increases, it becomes imperative to interrogate not only what these technologies can accomplish but also what might be lost in the process. This section examines how prevailing assumptions about skill development and assessment are being disrupted and suggests a reorientation of educational priorities for the era of generative AI.

A central contention of this chapter is that educational systems must intentionally foster human capabilities even as they leverage GenAI’s transformative potential. This imperative is not merely pedagogical; it is foundational to the cultivation of human skills that will enable individuals to thrive in rapidly evolving digital environments. According to Yan, Greiff, et al. (2025), it is important to distinguish between two interrelated dimensions in human learning when using GenAI: AI-empowered performance and human skill development (Figure 2.4). The first dimension (vertical in Figure 2.4) is focused on development of human skills – i.e. human learning. This dimension has traditionally been covered in education including educational research in AI in education and the application of GenAI to support human learning are covered in the previous section on "Existing Practices". However, the ubiquitous presence of GenAI changes the context in which learning happens. This is why we also consider the second dimension (horizontal in Figure 2.4), which concerns the extent to which individuals use AI tools, such as large language models, to enhance task execution and produce high-quality outputs. In the remainder of this section, we consider the implications of these two dimensions on human skill development according to evidence emerging from the existing literature.

The intersection of these dimensions defines the horizon toward which education should strive: learners who combine strong independent skills with the effective, reflective use of AI augmentation. However, a growing body of evidence suggests that this aspiration is not easily achieved and that the introduction of generative AI can create a "mirage of false mastery," where high-quality, AI-enabled output conceals underlying weaknesses in human skill – i.e. the undesirable curve in Figure 2.4 where task performance does not correlate with learning.

While generative AI has shown promise in supporting various educational tasks, its effectiveness in fostering long- term skill development remains uncertain. An important study in this space is conducted by Darvishi and colleagues (2024) who investigated the extent to which an AI support tool could extend student ability to provide effective peer feedback. The GenAI tool was designed to support students in generating feedback more effectively, rather than directly improving the content of their responses. In a large-scale randomised controlled trial with approximately 1 600 students, Darvishi et al. (2024) observed that while initial AI-supported gains in peer feedback quality were significant, these gains were not sustained once the tool was withdrawn. Students did not retain the feedback skills that appeared to have been acquired with AI support. Moreover, there was no robust evidence of synergistic development of human and AI-empowered skills; students generally exhibited strength in either AI-assisted performance or independent skill, but rarely both. These findings are echoed by a systematic review and meta-analysis by Vaccaro et. al. (2024) which analysed 106 experimental studies of human-AI collaboration. The meta-analysis found that, on average, human-AI combinations performed worse than the best of either humans alone or AI alone, especially in decision-making tasks. This cautions against the assumption that human-AI synergy will naturally emerge. The risk in many educational contexts is that generative AI either simply augments current abilities or, more problematically in an education content, substitutes for human effort without fostering genuine skill development.

This pattern of substitution is often driven by external pressures. Research by Abbas et al. (2024) revealed that university students were more likely to use ChatGPT when facing a high academic workload and time pressure. Their study, involving nearly 500 students, found that this utility came at a cost; increased use of ChatGPT was correlated with higher levels of procrastination, self-reported memory loss, and ultimately, diminished academic performance. Such findings suggest that students may turn to GenAI not as a partner for learning but as a tool to manage overwhelming demands, leading to unintended negative consequences. Furthermore, this substitution can foster an uncritical over-reliance on AI. In a systematic review, Zhai et al. (2024) investigated how over-reliance on AI dialogue systems affects students’ cognitive abilities. They define over-reliance as the uncritical acceptance of AI-generated recommendations, a tendency that arises when individuals struggle to assess the trustworthiness of the tool. Their findings indicate that this behaviour encourages the use of cognitive shortcuts, favouring fast, efficient answers over slow, effortful reasoning. This preference undermines the development of essential cognitive abilities, including decision-making, analytical reasoning, and critical thinking. It is crucial, therefore, to resist the temptation to conflate AI-augmented performance with authentic competence or deep learning.

Another dimension of concern relates to the effects of the use of GenAI on learning processes. There is accumulating evidence that increased reliance on GenAI tools can suppress students’ engagement in self-monitoring (defined as the ongoing process of checking, regulating, and adjusting one’s understanding and strategies during learning), reflection, and evaluative judgement of one’s learning processes, processes that are fundamental to autonomous learning (Molenaar, 2022). When GenAI is used as a shortcut rather than as a scaffold that promotes learning, students may defer cognitive effort to technology, thereby weakening the very skills that underlie deep learning.

Empirical research has begun to quantify risks of GenAI on reduced human cognition and metacognition. In a study comparing the use of ChatGPT to traditional search engines for a scientific inquiry task, Stadler et al. (2024) found that students using the large language model experienced a significantly lower cognitive load. However, this cognitive ease came at a cost: these students produced lower-quality reasoning and argumentation in their final recommendations compared to the group using the Google search engine. This highlights a critical trade-off, suggesting that while LLMs can reduce the cognitive burden of information gathering, they may not promote the deeper cognitive engagement necessary for high-quality learning. This finding is reinforced by a randomised experimental study by Fan et al. (2024), which compared university students’ writing processes when supported by ChatGPT, a human expert, a writing analytics tool, or no additional support. While the ChatGPT-supported group showed greater improvements in essay scores, these gains did not translate into deeper knowledge acquisition or transfer (as measured by knowledge transfer test on different topics). More importantly, the study found that learners in the AI-supported group demonstrated a marked reliance on the technology and were less likely to engage in metacognitive activities such as self-monitoring and reflection, a phenomenon the authors term metacognitive laziness.

The impact of AI on self-directed learning is further complicated by students’ motivations for using these tools. A year-long longitudinal study by Xie et al. (2024) examined how interaction frequency with chatbots affected learning autonomy. The results were nuanced: for learners seeking virtual companionship, the social presence fostered by the AI had a positive mediating effect on their learning autonomy. Conversely, for learners focused purely on knowledge acquisition, more frequent interaction with the chatbot was negatively correlated with both social presence and learning autonomy. This indicates that the effect of AI interaction is not uniform and that frequent use for instrumental purposes may undermine the development of independent learning habits.

These findings illustrate a crucial distinction: apparent improvements in performance enabled by generative AI may mask deficits in learners’ underlying cognitive and metacognitive processes. However, this does not mean AI cannot play a productive role in learning. When structured intentionally within a collaborative learning environment, AI can act as a powerful scaffold. For instance, An et al. (2025) studied student teachers using a mind-mapping tool integrated with GenAI. The groups using the AI tool not only outperformed the control groups on their collaborative tasks but also demonstrated a more sophisticated knowledge construction process, moving progressively from individual ideas to peer interaction and group synthesis.

Rethinking assessment

As students increasingly use GenAI tools for learning, traditional assessment models that focus solely on final outputs are becoming inadequate. When high-quality products can be produced with minimal engagement in the learning process, assessment risks measuring technological proficiency rather than human skill or understanding. To address this challenge, there is a pressing need to reorient assessment practices towards process-oriented approaches that evaluate not just what students produce, but how they engage with learning to create products. Assessments should aim to capture the processes students use to plan, monitor, and adapt their work, thereby revealing the authenticity and depth of their learning in GenAI-rich environments. Only by prioritising cognitive and metacognitive engagement alongside product quality can educational systems ensure that AI augments, rather than supplants, the development of meaningful human expertise.

One promising way to operationalise this shift is through evidence-centred assessment design (ECD) (Mislevy, 2006). The ECD framework provides a principled model for linking assessment tasks, evidence, and inferences about learners’ knowledge and skills. By moving beyond a narrow focus on final outputs, ECD enables the design of multidimensional assessments that capture both product and process evidence.

An illustrative example of this process-oriented approach comes from recent work in medical education, where clinical reasoning tasks have been redesigned to capture a more holistic view of learning (Tang et al., 2025). Drawing on the ECD framework, this approach moves beyond assessing only the final diagnostic conclusion. Instead, it builds a multidimensional evidence model by collecting three streams of data as students interact with GenAI-powered virtual patients: product evidence (e.g. diagnostic accuracy), process evidence (e.g. conversation logs where students do history taking), and metacognitive evidence (e.g. clickstream data and interaction logs). Analysis of this rich data reveals that integrating all three evidence sources provides a significantly more reliable prediction of learner performance than relying on product-based measures alone. Notably, process data emerged as the strongest standalone predictor, underscoring the value of assessing the “how” of learning, not just the “what.”

Building on this ECD foundation, predictive models are now being paired with Explainable AI (XAI) to make the assessment process not only accurate but pedagogically meaningful. Simply predicting performance with a “black-box” machine learning model is insufficient for supporting learning. To make insights actionable, the XAI layer identifies the key factors influencing a prediction (Khosravi et al., 2022). These technical explanations are then translated by a GenAI system into structured, personalised, and pedagogically relevant feedback for the learner. This hybrid XAI–GenAI approach ensures that feedback is aligned with self-regulated learning principles, helping students understand not only their performance but also the cognitive and metacognitive strategies that shaped it. By grounding feedback in specific evidence from the learning process, this approach extends the ECD model beyond assessment design to feedback delivery, providing transparent, actionable guidance that fosters genuine skill development.

Figure 2.5 demonstrates how this assessment approach is implemented in the FLoRA platform for history-taking skills as part of the development of clinical reasoning in medical education (Tang et al., 2025). Learners first interact with the virtual standard patients Figure 2.5 A, which are also based on GPT. Once the learner completes the interaction with the virtual standard patient and submits their diagnosis (Figure 2.5 B), the system applies the evidence model and generates personalised feedback (Figure 2.5 C)

Methodological rigour

The methodological rigour of research on generative AI in education is critical to produce quality evidence. If we are to make sound, evidence-informed decisions, it is essential to move beyond the commentaries and hype cycle and uphold high standards of empirical inquiry. As also indicated in the previous subsection and Figure 2.4, a central challenge in producing robust empirical evidence about effects of GenAI on human skills is the pervasive conflation of performance with learning (Yan et al., 2025). Performance refers to the observed performance of a task, whereas learning involves an enduring change in knowledge and skills that is demonstrated through retention and transfer (Soderstrom and Bjork, 2015). The distinction is essential; high performance, especially when mediated by a powerful tool, does not imply that learning has occurred.

A second, related but distinct issue is the media/methods fallacy (Clark, 1983). For decades, researchers have cautioned against simplistic “media comparison studies” that attribute learning gains to a technology itself, rather than to the specific instructional methods it enables. Much of the nascent research on generative AI repeats this error, comparing an ill-defined “ChatGPT condition” with a control group and concluding that the technology “works”. Such designs may demonstrate that a particular arrangement (e.g. students working with ChatGPT) can yield different outcomes than another (e.g. students working alone). However, because they attribute effects to the technology as a whole rather than to the specific instructional processes it affords, these studies provide limited insight into the underlying mechanisms. This limits their explanatory power and risks conflating performance support with genuine learning (Weidlich, 2025).

A further methodological weakness, distinct from but often co-occurring with the media/methods fallacy, is the conflation of task performance with learning. For instance, meta-analyses claiming that ChatGPT enhances “academic performance” often measure immediate task achievement, not durable learning, and sometimes not even learning (Deng et al., 2024). While students may produce a better essay or translation with AI assistance, this performance gain may mask a lack of underlying cognitive engagement and learning gains. As discussed earlier, offloading effort to AI can reduce cognitive load but also risks fostering “metacognitive laziness”, thereby undermining the very processes required for deep skill development (Fan et al., 2024; Stadler, Bannert and Sailer, 2024). This problem is amplified by the “fast science” culture, where sensational claims, such as GPT-4 “acing” the MIT curriculum (Zhang et al., 2023) gain traction despite significant methodological flaws, including data contamination and a lack of transparent verification (Chowdhuri and Koplow, 2024). Even if such claims were accurate, they would have limited educational meaning, as GenAI outperforming standardised or benchmark tasks does not come with conceptual understanding and does not imply that the underlying processes of learning and transfer by humans should be abandoned. The danger lies in conflating technical proficiency with educational value, which can distort expectations and policy directions and fuel the kind of policy-practice misalignment that has characterised many AI-in-education debates (Lodge, 2025).

To build a robust evidence base, the field must adopt a more rigorous research agenda. First, researchers must explicitly differentiate learning from performance by incorporating process-oriented assessments, such as delayed retention and knowledge transfer tests, into their designs (Yan et al., 2025). Second, studies must move beyond media comparisons to isolate causal mechanisms, clearly defining the pedagogical function of the AI intervention, much like the decades of theory-driven research on Intelligent Tutoring Systems (Weidlich, 2025). Finally, we must prioritise longitudinal research that tracks the durable effects of AI interaction on students’ knowledge, skills, and dispositions over time.

For policymakers and funding organisations, this highlights a critical need to guide future investment. To build a robust evidence base, funding should prioritise longitudinal studies that track durable skills, demand that interventions clearly specify their pedagogical underpinnings and support the development of process-oriented assessments. Only by investing in research that distinguishes task performance from learning can we ensure that technology serves our ultimate goal: fostering deep and lasting human competence.