OECD Digital Education Outlook 2026: Exploring Effective Uses of Generative AI in Education

To ground the discussion in a concrete example, this section introduces the operational Socratic Playground (SPL) prototype and examines how it functions in real educational scenarios. The SPL system serves as a demonstration of generative Socratic tutoring in action (Figure 3.1), focusing initially on the domain of essay writing and critical thinking. This section will describe a typical user experience with SPL, summarise preliminary evaluation data on its effectiveness, and discuss practical challenges encountered during deployment. The lessons learned from SPL’s pilot use – including user feedback and technical issues like latency and hallucinations – highlight the gap that can exist between research promise and practical deployment, offering valuable insights for future improvements.

Use case: Socratic tutoring for essay writing

In the piloting scenario with SPL, learners were asked to compose a short argumentative essay. For example, one prompt might be “Should renewable energy be subsidised by the government?” Instead of grading the essay outright or providing a static set of comments, the SPL agent engages the learner in a multi-turn Socratic dialogue about their essay (Figure 3.2). The session typically begins with the agent greeting the user and asking to see their draft or initial ideas. Suppose the student writes a few sentences stating their position. The agent will analyse this input (via GPT-4 and the underlying prompt structure) and then respond with a thoughtful question – often a “why” or “how” question – aimed at deepening the student’s argument. For instance, if the student asserted “Yes, renewable energy should be subsidised because it’s better for the environment”, the agent might ask, “Why do you think government subsidies are necessary for environmental benefits, as opposed to letting the market handle it?” This kind of why- question scaffold pushes the student to elaborate on their reasoning. The student then responds, perhaps adding that “without subsidies, renewable projects might not attract investment”. The agent continues this process, maybe following up with another prompt like, “Can you think of a specific example or evidence that supports that point?” Through such iterative questioning, the student is led to flesh out their argument with reasoning and evidence, essentially engaging in critical thinking about their own writing.

One notable observation from the SPL demonstration is that students often improve the quality of their reflections and explanations during these dialogues. Preliminary data collected from pilot users (university students in a writing workshop) suggest that after interacting with the Socratic agent, the students’ final essays included more justification for claims and considered counterarguments more frequently than their initial drafts. While this is not a controlled study result, it aligns with the expectation that prompting learners to explain and justify would yield deeper engagement with the material (Nye, Graesser and Hu, 2014). The GenAI agent essentially acts as a catalyst for self-explanation, a well-known mechanism for learning gains (Chi, 1994). Users reported that the agent’s questions made them think more critically: one participant noted “The AI asked me things I hadn’t considered, like how exactly the subsidies work. It was challenging but it made my argument better.” This anecdotal feedback resonates with our goals – the agent is not providing direct answers but improving the learner’s thought process and output.

Personalised scaffolding in action

The SPL system also demonstrates personalisation by adapting to different users’ needs within the essay task. For a learner who is struggling to generate content (Figure 3.3), the agent takes on a more supportive, even slightly leading role. It might break down the task: “Let’s start by outlining two main reasons you support subsidies. What’s one reason?” If the student is totally stuck, the agent can even offer a gentle nudge like, “One reason might be related to climate change – do you want to expand on that or think of another reason?” On the other hand, for a confident learner who writes a strong initial paragraph, the agent switches to a more challenging role – perhaps by introducing a counterpoint: “Some critics argue subsidies distort the market. How would you respond to that counter-argument in your essay?” This not only personalises by difficulty but also by role: with the less confident student, the agent was a coach breaking down the task, whereas with the advanced student, it became a debate partner injecting opposing views. The underlying mechanism enabling this is the continuous profiling of student performance and the flexible prompt template described earlier. Essentially, after each student response, the system classifies how well the student is doing and decides on a strategy (simplify vs. challenge, new question vs. give hint). This can be thought of as following the student’s Zone of Proximal Development – always trying to operate just above the current level. By observing features like the length and substance of the student’s answers, the system adjusts its scaffolding depth. Early user sessions show this adaptive behaviour could lead to deeper engagement: students at earlier stage of mastery benefit from tasks broken into manageable steps, while more advanced students remain engaged through progressively complex problems. Users in a professional development workshop who tried SPL noted that the agent felt “attentive”, an impression due to this adaptive behaviour.

Preliminary outcomes and feedback

Although comprehensive efficacy studies are still to be done (and this paper later suggests a framework for such studies), initial trials of SPL have been encouraging. In a pilot at the Hong Kong Polytechnic University involving about 20 adult learners (teachers and professionals exploring GenAI tools), participants reported high levels of satisfaction with the agent’s usefulness. A post-session survey using a 5-point Likert scale found that the majority agreed (4 or 5) that the AI tutor’s questions helped them think more critically about the topic. Many also indicated they would like to use such a tool regularly for brainstorming or refining their writing. From the system’s perspective, logs showed that the dialogues often went through 8 to 12 turns of Q&A, with the learner contributing increasingly complex answers. Linguistic analysis of the student responses from their first turn to the final turn in each session indicated an increase in lexical diversity and sentence complexity, which can be seen as proxies for richer content (though more rigorous content analysis is needed). These observations align with the idea that Socratic generative tutoring fosters deeper reflection and engagement, resulting in improved outcomes like more thoughtful essays. Furthermore, previous findings suggest that a GPT-4 powered Socratic tutor (like SPL) facilitates more effective tutoring interactions than legacy ITS (Liu et al., 2024), a result qualitatively and empirically reinforced by our hands-on trials.

However, the SPL demonstration also surfaced significant challenges, offering a reality check on the hype surrounding AI tutors. One practical issue was GPT-4’s latency, as anticipated. In some cases, users had to wait 5 to 10 seconds for the AI tutor to respond, particularly as the prompt context increased (for instance, after the student had written a few hundred words of essay, sending all that plus the JSON structure to the model could prolong the response time). While many users were patient and understood this was a prototype, a few found the waiting time disruptive, especially when the conversation was flowing and then had a pause. This underlines a need for further optimisation or perhaps using a faster model for intermediate turns. Another issue observed was occasional hallucinations by the AI tutor. For example, in one session the student mentioned Germany’s renewable energy policy, and the agent responded with a detailed “reminder” about an apparent German law that was not actually real – it had fabricated a plausible-sounding fact. The student was savvy enough to question it (“I’m not sure that law exists”) and the agent then backtracked, but this could mislead less knowledgeable learners. We have since added additional checks when the agent provides factual statements, but it is a stark reminder that even a pedagogically well-intentioned AI can introduce misinformation. Ensuring factual accuracy remains an ongoing battle in generative AI tutoring, reinforcing arguments in the literature about the need for grounding and verification (Li et al., 2025).

User interface and usability refinements

The SPL pilot also highlighted some interface issues. For instance, the initial version did not make it obvious that the user could ask the agent questions at any time; some users thought they could only answer the agent’s questions. This one-sided interaction wasn’t the intention – the system was capable of handling user-initiated questions or clarifications, but the UI cues were not clear. In response, we adjusted the interface to clarify that users can ask the AI tutor for explanations or request a hint. Another minor but interesting observation was that some users interacted with the AI tutor in a formal tone at the beginning of their conversation (for example, “Dear tutor, I have a question…”). Over time they became more conversational as they realised the agent responds like a human would. This acclimatisation suggests that building user trust and familiarity with the agent’s style is part of adoption; any deployment should consider an onboarding or tutorial that lets users get comfortable with talking to the AI. From a design perspective, we found that maintaining a user’s sense of control was vital. When a participant disagreed with the AI’s suggestion, the AI persisted in pressing its point with the intention to be thorough, which left the user feeling frustrated. In subsequent tweaks, we have the agent explicitly acknowledge and respect the user’s viewpoints more (for example, “That’s a valid perspective. Shall we explore it further or do you want to consider alternative angles?”). This preserves the pedagogical goal of reflection but avoids the impression of the AI insisting on its way. Such fine-tuning makes the agent more like a supportive guide than an interrogator, which is important for sustained engagement.

Summary

The Socratic Playground demonstration provides a valuable case study in the real-world implementation of a generative pedagogical agent. It points to several potential benefits that are emerging in discussions of LLM-driven tutors (personalised scaffolding, improved critical thinking in student work, and positive learner reception) (Córdova- Esparza, 2025; Liu et al., 2024; Park and Seo, 2025). At the same time, it unearths the pragmatic issues that arise when moving from controlled development to practical use: latency, occasional AI errors, interface clarity, and the delicate balance of control between student and agent. The experiences from SPL underscore a central theme of this chapter: there remains a gap between the research promise of GenAI in education and the practical deployment of these tools, which can only be closed through iterative refinement, user-centred design, and rigorous evaluation.