Strengths and Shortcomings of Generative AI Tools

General-purpose GenAI tools often provide pertinent and contextualized answers to questions, with the ability to clarify and ask follow-up questions. These capabilities were not possible with earlier (non-generative AI) natural language processing. They are trained on massive data sets that exceed what humans could retrieve manually. Moreover, they are flexible and can be applied to many different subjects. Contrary to most educational AI, general-purpose GenAI tools usually offer free versions, enabling students and teachers to use them even if they are not provided by universities or schools, assuming they have an adequate device and connectivity. Even offline, small language models can run, albeit with lower performance (Isotani, 2026).

A series of well-known shortcomings are also specific to current GenAI systems and inherent to its technology. Because they are based on probabilistic models, they can “hallucinate”, that is, produce a plausible but wrong answer or fabricate details of an output (see OECD, 2024, chapter 2). They usually do not generate consistent results over time. For example, repeating the same task several times will yield (at least slightly) different answers or productions, which is sometimes a problem. This is due to regular system updates and to their probabilistic nature. As they are trained on available datasets, their answers and other productions tend to reproduce the views and perspectives represented in those datasets, which are overwhelmingly based on English-speaking (and Western) cultures. For example, unless prompted otherwise, they will typically use Western names or examples in their production. In addition, despite appearing intelligent, GenAI tools do not “understand” the input they process or the content they generate. As a result, their outputs typically require human supervision and scrutiny, often more than specialized, non-generative AI systems.