OECD Digital Education Outlook 2026: Exploring Effective Uses of Generative AI in Education

Generative AI in higher education can involve helping personalise the guidance students receive, not just from a recommender system, but from human academic advisors. In Lekan and Pardos (2025) an advisor-facing, GPT-driven model was tested whereby first year college students (n = 33) were asked questions about course preferences and career goals, typical of a human advising session. These responses were fed to a GPT model that, instead of giving advice directly to the students, gave major recommendations and justifications to an advisor (n = 25). The study found that academic advisors rated the suggestions of the GPT model favourably and exactly agreed with the model’s major recommendation 33% of the time. In this case, participating advisors were positive on this type of human-AI collaboration, as providing assistance and leaving them as the point of contact to students, rather than supplanting them.

Research collaborations with registrars and admissions offices have begun to explore how advisors can use analytics to better support student course selection. For example, nascent work (Borchers and Pardos, (n.d.)) with undergraduate advising integrates Big Five personality traits, such as conscientiousness and neuroticism, with multi-semester course enrolment data and found that students high in conscientiousness or self-efficacy tend to perform well even under heavy workloads, while those lower on these traits are more likely to struggle. This suggests that advisors could move beyond general heuristics (e.g. “don’t overload”) and instead offer more individualised recommendations based on a student’s likely capacity to manage challenging course schedules.

The same form of advising could also be used in high school to help students choose their higher education study programmes, or later on in their life, to support their choices of lifelong learning options. While generative AI is also explored as means to provide or facilitate other advising and counselling services to college students (e.g. mental health), these applications are not without controversy and have shown growing pains in their current stage of development (Moore et al., 2025).

Assessment represents one of the most resource-intensive components of higher education. Generating high-quality items for standardised tests in some subjects requires significant faculty time, while evaluating and calibrating those items demands large respondent pools and psychometric expertise. Recent advances in generative AI offer institutions the opportunity to refresh assessment practices by accelerating both the production and the evaluation of large-scale item pools. Importantly, these processes often occur in the institutional “backend”-funded, managed, and maintained by campuses or system-level services-in addition to being driven by faculty or students directly. Developing assessment items is also relevant at the school level in countries with national assessments, whether they are developed by public evaluation agencies or by private companies.

LLMs offer novel perspectives on automating the creation of multiple-choice and short-answer items, particularly when anchored in existing curricular material. Studies comparing LLM-generated questions to textbook-sourced questions find comparable psychometric properties. For example, Bhandari et al. (2024) report that ChatGPT-generated algebra items demonstrated difficulty and discrimination parameters statistically indistinguishable from traditional textbook items when evaluated with item response theory. Notably, the LLM-generated items exhibited slightly stronger differentiation between high- and low-ability respondents, suggesting that GenAI can produce assessment content of similar or even superior quality under controlled conditions. This holds particular promise for large lecture courses and general education programmes where instructor time is scarce and item demand is high (e.g. test banks must be regularly refreshed to ensure continued assessment validity). To retain the instructor’s agency over course assessments while decreasing the time it takes them to create an assessment is not just an algorithmic matter but a human-computer interaction design issue. New human-computer interaction research, such as work on the PromptHive tool, provide examples of placing subject matter experts in the driver’s seat of generative AI to integrate their expertise into the workflow of assessment creation (Reza et al., 2025). An instructor, for example, provides her existing assessments as a style reference as well as the new learning objectives she wants additional assessments to cover. PromptHive creates a pool of assessment items covering the learning objectives and allows the instructor and TAs to instruct PromptHive on the types of hints that should be produced to scaffold learning of the related content. The instructional team can then preview the generated hints and assessments on a subset of items or all items. The limitation here is that generative AI still hallucinates in most topic areas. If hallucination rates are not evaluated to be 0% in the topic area, this necessitates the instructional staffto check every problem and hint being produced before it is seen by a student.

Relatedly, generative models can also address known limitations in traditional test banks. One persistent challenge is the overexposure of items, in cases where repeated use narrows the effective variance of assessments and introduces unwanted correlations between items. For example, many high-stakes, summative tests are strictly proctored with test items that are not released to the public; however, over time, test takers may socialise the contents of the test to future test takers, or test prep companies, leading to overexposure of the items if they are not changed frequently. By creating novel assessments and even well-crafted multiple-choice distractors, large language models can diversify item pools, reducing the risk of distributional shifts when the same questions are repeatedly deployed. Yet producing new distractors at scale raises the question of quality control. Poorly written distractors-those that are implausible, misleading, or inadvertently cue the correct answer-can reduce both the fairness and the psychometric value of multiple-choice questions. Here, automated evaluation methods are beginning to complement generative approaches. Moore et al. (2023) provide evidence that such methods can systematically detect flaws in student- and AI-generated multiple-choice questions. The authors evaluated undergraduate students in introductory courses who were prompted to generate multiple- choice questions on recently learned material. Comparing a rule-based system to GPT-4 on 200 student-generated questions across four domains, they found that the rule-based approach identified 91% of item-writing flaws flagged by human annotators, compared to 79% for GPT-4. Many of these flaws involve distractor design-such as implausibility, redundancy, or overlap with the correct answer-precisely the weaknesses that can undermine the benefits of automated generation at scale. Hence, human expertise and quality control will remain important pillars as LLMs are used for content generation at larger scale.

Beyond production, GenAI is emerging as a tool for evaluation. Item calibration – the process of estimating psychometric properties such as difficulty and discrimination – typically requires thousands of student responses. Liu et al. (2025) demonstrate that multi-agent AI models bringing together ensembles of LLMs can serve as “synthetic respondents,” producing response distributions with psychometric properties closely aligned to those of college students. While a single LLM was not measured to exhibit abilities similar enough to the target human population, ensembles of different LLMs expand variance, yielding item parameter estimates highly correlated (> 0.8) with human-calibrated values. Augmentation strategies, such as adding LLM responses to even a small set of human respondent data further improves alignment with exclusively human responses. These findings suggest a new institutional workflow: LLM-based calibration can complement limited student response data, reducing costs and accelerating item validation cycles. While human responses remain essential for final benchmarking, AI-assisted evaluation can substantially shorten development timelines for new assessments.

Beyond opportunities and feasibility, a practical open challenge is instituting policies for the use of generative AI in assessment. Drawing from emerging frameworks proposed by Corbin et al. (2025), open issues include:

• where to set meaningful limits on AI assistance for different outcome types; • what disclosure, attribution, and provenance practices are sufficient (e.g. prompts, drafts, and model/version logs); • how to handle discipline-specific variation without sacrificing consistency; and • how to mitigate workload burdens for staffwhile maintaining validity and fairness.

A challenge in these policies is that AI models and their capabilities will continue to change. Therefore, policies will need to be adaptive rather than static and focused on guiding principles and review mechanisms rather than fixed prohibitions. These considerations are especially important as AI is deployed in assessing high-stakes scenarios for learners, such as for university admissions. For example, von Davier and Burstein (2024) discuss several practices toward human involvement in AI decisions to ensure ethical, accountable, and valid use. These include ongoing human oversight of automated scoring and item generation, systematic review of algorithmic outputs for fairness and bias, engagement of diverse stakeholder groups in test development and validation, and transparent communication of AI roles and limitations to test-takers and institutions. There is also a tension between the continued adoption of rule-based approaches (e.g. college degree audits) and AI evaluation approaches, with hybrid approaches being a fruitful area for future exploration.

AI-powered systems are increasingly being integrated into academic advising and course planning infrastructures, often spearheaded by researchers in collaboration with administrative units. While many of these tools remain student-facing, their design and deployment reflect broader shifts toward data-informed institutional coordination and administrative personalisation.

1. Administrative collaboration and oversight. Course planning platforms such as AskOski at the University of California, Berkeley and ATLAS at the University of Michigan exemplify tools developed in partnership with faculty and administrative stakeholders. These systems aim to guide students through course selection while integrating data from past students' course selection pathways.

2. Faculty and advisor-facing systems. Some platforms are oriented primarily toward professional staffand faculty advisors, supporting them in tailoring study guidance recommendations. For example, ongoing research explores how articulation officers interact with algorithmic guidance to make course articulation decisions (see atain.org).

3. AI in admissions and curricular governance. Beyond advising, machine learning and analytics increasingly support institutional decision-making in admissions and curriculum planning. Administrative priorities should be the starting point to help co-develop or curate algorithmic systems.

4. Towards analytics-informed curricular innovation. Institutions may adopt systems that combine machine learning-driven insights with human advisor expertise to rethink credit hours, assimilate curricular content to new content standards, and enhance curriculum-level analytics, including of course difficulty.

These examples demonstrate how research-administrator collaborations for course advising, transfer, and curricular planning platforms are becoming testbeds for the future of responsible AI integration in institutional workflows. Their success hinges on administrative collaboration, methodological rigour, and thoughtful engagement with human expertise.