OECD Digital Education Outlook 2026: Exploring Effective Uses of Generative AI in Education

The integration of GenAI agents into education has moved beyond theoretical promise to rigorous empirical validation. In recent years, some randomised controlled trials and large-scale field studies have provided valuable insights on the current efficacy of ITS powered by GenAI. In this section, we synthesise emerging evidence across three distinct deployment models: hybrid/human-in-the-Loop (augmenting human tutors), independent tutoring (replacing or supplementing lectures), and classroom integration (supporting real-time classwork), and concludes with a streamlined framework (see Box 3.3) for evaluating the efficacy of such systems.

Emerging empirical evidence

A first scenario for the use of GenAI systems is that the AI does not teach the student directly but acts as a real-time “whisperer” for a human tutor, suggesting pedagogical moves to enhance instruction. The most prominent example is Tutor CoPilot, deployed in a large-scale randomised controlled trial involving 900 human tutors and 1 800 high school students (Wang et al., 2025). The study found that while using the GenAI tutor improved student mastery by 4 percentage points on average, its true power lay in "levelling up" the workforce. Students paired with lower-rated or novice tutors who used the CoPilot saw learning gains of 9 percentage points compared to the control group, effectively closing the gap between novice and expert tutoring. Analysis of chat logs revealed the mechanism: the GenAI system successfully nudged inexperienced tutors away from simply giving answers and toward using expert scaffolding strategies, such as asking guiding questions. This suggests that one of the most effective uses of GenAI is not to replace humans, but to scale expert pedagogy across a variable workforce.

Another scenario involves students interacting directly with an AI pedagogical agent to learn new concepts or accelerate their study, often outside standard classroom hours. At Harvard University, the Harvard Physics Tutor (a custom GPT-4 agent) was tested against a “gold standard” active learning classroom in a randomised crossover trial (Kestin et al., 2025). The results were striking: students using the AI tutor achieved learning gains more than double those of the active learning group (effect size d ≈ 0.73 - 1.3) and, crucially, spent significantly less time to reach that proficiency. This highlights the efficiency of “hyper-personalisation”, where the GenAI addresses specific misconceptions that a classroom teacher cannot individually address for every student simultaneously.

Similarly, in the distance learning context, IU International University deployed Syntea, a GenAI teaching assistant, to over 10 000 students (Möller et al., 2024). The primary metric of success here was “learning velocity”: students using Syntea reduced the average time required to complete a course by 27% while maintaining exam performance. By acting as an always-available Socratic study partner, the GenAI agent removed the “wait time” for feedback, effectively accelerating the learning loop. In low resource settings, the text-based math tutor Rori demonstrated that high-fidelity interfaces are unnecessary for impact. Deployed via WhatsApp to over 1 000 students in Ghana, Rori produced significant math growth (effect size d = 0.37) at a marginal cost of roughly $5 per student, proving that conversational AI can bridge the digital divide even on basic mobile infrastructure (Henkel et al., 2024).

Thirdly, GenAI tutors can be used alongside standard instruction for practice problems. In that case, current evidence points to a high risk of cognitive offloading if “guardrails” are absent. (Cognitive offloading is the act of using external tools or resources to reduce the mental effort required to perform a task or remember information.) A study involving nearly 1 000 high school math students compared a standard “GPT Base” model against a pedagogically engineered GPT Tutor. Students given unrestricted access to the “Base” model performed 48% better during practice but 17% worse on subsequent independent exams, a phenomenon termed the “Crutch Effect”. The students had learned to use the AI to bypass the cognitive struggle necessary for learning. The “GPT Tutor”, intentionally engineered to withhold direct answers and prompt for self-explanation, mitigated this harm but did not yield the artificial performance boost seen in the base group (Bastani et al., 2025).

Other classroom tools like Khanmigo (Khan Academy) have shown mixed quantitative results but strong qualitative benefits. While some trials of Khanmigo showed “no statistically significant difference” in short-term test scores compared to standard web search, students reported a significant reduction in “evaluation apprehension”. They felt safer asking “stupid questions” to the AI than to a teacher (Slijepcevic and Yaylali, 2025).

In summary, given this landscape of heterogenous outcomes – ranging from accelerated mastery to skill degradation – it is clear that efficacy is not inherent to the technology but dependent on implementation. More importantly, rigorous evaluation to the implemented tools should be warranted to distinguish authentic learning gains from deceptive performance boost. This calls for continuous and systematic evaluation of different uses of Gen-AI powered tools for learning. Box 3.3 provides some ideas of measures for evaluating these tools.

Challenges, ethics and practical implications

Deploying GenAI agents to facilitate technology-enhanced tutoring brings specific challenges that must be addressed to ensure these tools are responsible, equitable and educationally effective. While the potential for personalised, adaptive tutoring is vast, the implementation must navigate technical limitations and pedagogical risks.

Reliability and fairness in dialogue

A primary technical challenge in dialogue-based tutoring is the tendency of generative LLMs to “hallucinate”, producing plausible but incorrect information (Zhang et al., 2025). In a Socratic context, where the tutor leads the student through a chain of reasoning, a false premise introduced by the AI can derail the entire learning process. If students internalise these errors, the damage is significant. Studies have already observed students reproducing AI-introduced errors in homework tasks (Pardos and Bhandari, 2024).

To mitigate this, systems increasingly employ Retrieval-Augmented Generation (RAG) to ground AI responses in trusted corpora, such as textbooks (Li et al., 2025). Additionally, fairness remains a critical concern. LLMs can exhibit performance gaps across languages or dialects, potentially disadvantaging non-native speakers (Lai and Ngo, 2023). Furthermore, without careful calibration, an AI tutor might inadvertently favour specific cultural perspectives or arguments, undermining the neutrality required for effective tutoring.

Quantitative metrics (outcome and process)

• Standardised Learning Gains: Use pre-test/post-test designs with validated instruments such as the California Critical Thinking Skills Test (Facione, 1990). Critical: Include delayed post-tests (e.g. one week later) without AI assistance to detect the "Crutch Effect" (performance illusion). • Artefact Quality (Blind Grading): For writing / programming, use blind grading with rubrics (e.g. OECD’s rubric for creativity and critical thinking) to assess if AI-coached drafts show superior critical thinking or creativity compared to control drafts. • Efficiency: Measures “time-to-mastery” – whether AI allows students to reach the same proficiency standard sooner than traditional approaches. • Interaction Granularity:

1. Time-on-Task: Time spent engaging in the learning activities with AI (e.g. longer duration may suggest enjoyment with AI as the learning companion).

2. Dialogue Symmetry: Ratio of student-to-AI words (e.g. low ratio may imply passive consumption while high ratio may indicate active engagement).

3. Lexical Diversity and Syntactic Complexity: Use NLP to track if student responses become more complex over time (a proxy for deeper thinking).

• Think-Aloud Protocols: Engage students to verbalise their thought process while using the AI. This identifies “mechanised convergence” (i.e. blindly accepting AI suggestions). • Evaluation Apprehension: Survey students on their perceptions of using the ITS to support learning (e.g. comfort level asking questions to the AI vs. a human teacher). • Teacher and Stakeholder Feedback: Interviews or focus groups to assess integration friction (e.g. whether the integrated tool disrupts class flow).

• Controlled Comparisons: Introduce Randomised Controlled Trial (or Quasi-Experimental design if RCT is not feasible) to enable comprehensive evaluation of system efficacy. • The “Ablation” Study: For systems designed with a full suite of features, examine the systems’ efficacy by introducing experimental conditions with and without specific features. • Human-in-the-Loop RCT: Randomised access at the instructor level (not just student level) to scrutinise how the introduction of tools alters instructors’ pedagogical practices. • Longitudinal Tracking: Track usage over a period (e.g. a full semester) for the “novelty effect” wearing off; Track learning gains over a period to ensure they sustain.

Equity and safety checks

• Bias Audits: Stress-test the tools with “adversarial inputs” to check for cultural or political bias. • Demographic Disaggregation: Analyse results by language backgrounds to ensure the tool aids learners of diverse competencies, not just the high-achievers.

Learner autonomy and cognitive dependency

Perhaps the most significant pedagogical challenge is maintaining the delicate balance between support and independence. There is a valid concern that over-reliance on AI assistance may reduce mental effort and compromise the depth of inquiry (Stadler, Bannert and Sailer, 2024). If an AI tutor is too directive, or if the student passively accepts the AI’s guidance, the metacognitive benefits of the method – self-evaluation and critical thinking – are lost.

Designers must thus ensure the AI empowers the learner rather than making them a passive recipient. This involves transparency in why a question is asked and explicitly prompting students to verify information, fostering the metacognitive development essential in the era of GenAI (Tankelevitch et al., 2024).

The teacher-in-the-loop

Consistent with the consensus in the field, AI tutors should be viewed as tools to augment, not replace, human educators (Li et al., 2025; Cukurova, 2026). The ethical safeguard for these AI tutors should be a “human-in- the-loop” approach, where teachers retain oversight of the AI’s guidance (Alfredo et al., 2024; Colonna, 2023; OECD-Education International, 2023). Teachers must have the agency to determine when the AI is used – for example, assigning it for preliminary homework discussions so class time can be reserved for deeper analysis (Strauß and Rummel, 2026). This requires distinct professional development to ensure teachers are literate in interpreting AI outputs and intervening when the system’s logic drifts.

Implementation logistics and ethical safeguards

Beyond the specific pedagogical dynamics, the broader deployment of these agents requires strict adherence to operational and ethical standards. As Luckin and Holmes (Luckin, 2016) argue, technological innovation must be paired with ethical guardrails. Key considerations include:

• Data Privacy and Governance: Systems must comply with regulations like European Union’s GDPR or United States’ FERPA. Since AI tutors collect deep behavioural data, strict anonymisation and access controls are required to protect student privacy (Colonna, 2023). • Infrastructure and Equity: Deploying LLMs like GPT-OSS and Qwen3 is computationally expensive. To prevent a digital divide where only wealthy institutions access high-quality GenAI-powered tutoring, strategies must include subsidised access or the use of optimised, lower-cost models. • Transparency and Trust: It is an ethical imperative to transparently label the agent as an AI. Users should be informed of the system’s limitations – specifically its potential to hallucinate – to encourage critical evaluation rather than blind trust.