International AI Safety Report 2026

Reliability challenges

Key information

• When general-purpose AI systems fail, they can cause harm. Failures include producing false or fabricated information (often described as ‘hallucinations’), writing flawed computer code, and giving misleading medical advice. These failures have the potential to cause physical or psychological harm and expose users and organisations to reputational damage, financial loss, or legal liability.

• Models’ behaviour is often difficult to understand or predict, making it challenging to guarantee reliability. Even the developers of general-purpose AI models can often not meaningfully explain model behaviour, anticipate specific failure modes, or demonstrate that such failures will not occur. Malicious actors can also induce failures by interfering with AI development or giving systems adversarial inputs that evade safeguards.

• AI agents pose heightened reliability risks because they act autonomously and can directly affect other systems or the physical world. Agent failures can cause greater harm because humans have fewer chances to intervene. Multi-agent systems introduce further risks, as errors can propagate and amplify through agent interactions.

• Since the publication of the previous Report (January 2025), AI systems have generally become more reliable and, as a result, have seen greatly increased commercial deployment. Many kinds of failures, such as hallucinations, have generally become less likely, but systems still commonly make mistakes when performing more complex tasks.

• Despite significant research efforts, no combination of methods ensures the high reliability required in critical domains. New training methods and giving AI systems access to tools can make failures less likely, but usually do not eliminate them completely.

General-purpose AI systems fail in ways that have already caused real-world harm, from fabricated legal citations to medical misdiagnoses. While human professionals also make mistakes, AI failures raise distinct concerns because of their novelty, potential scale, the difficulty of predicting when they will occur, and users’ tendency to uncritically trust confident-sounding outputs. Current general-purpose AI failures include providing false information, making basic reasoning errors, and degrading when deployed in new contexts. Documented harms from such failures include medical misdiagnoses, mistakes in legal briefs, and financial losses. Reliability challenges are particularly critical for AI agents, since failures can directly cause harm without human action or oversight. Multi-agent systems introduce further failure modes through miscoordination, conflicts, or undesired collusion between agents.

General-purpose AI systems face a range of reliability challenges

‘Reliability’ refers to the extent to which an AI system functions as intended by the developer or user. General-purpose AI systems experience a range of reliability issues, ranging from inaccurate or misleading content generation to failures performing basic reasoning. For example, while models have improved at recalling factual information, even leading models continue to give confident but incorrect answers at significant rates (Figure 2.10). In software engineering, general-purpose AI can now provide substantial assistance in writing, evaluating, and debugging computer code. However, AI-generated code often includes bugs, while coding agents regularly make errors. Such failures can introduce vulnerabilities into programs and security systems (see §2.1.3. Cyberattacks).

Reliability issues are particularly important to track in high-stakes settings, such as medicine, due to the accelerating use of AI and the potential for failures to result in severe harm. Relevant capabilities have improved quickly, with leading models now able to pass medical exams. Yet, real-world use reveals limitations that benchmarks miss. For example, in one study, models provided potentially harmful answers to 19% of medical questions posed. Such failures could result in misdiagnosis, inappropriate treatment, or wrongful denial of care.

AI agents pose novel reliability risks due to their autonomy

Because AI agents directly act in the real world, their failures have the potential to cause more harm than failures in non-agentic systems. Unlike AI systems that simply produce text or images for humans to review, AI agents can independently take actions that affect the world (see also §1.1. What is general-purpose AI?). AI agents can initiate actions, influence other humans or AI systems, and dynamically shape future outcomes. This expanded scope of influence introduces new risks and amplifies the importance of reliability, as failures could directly cause harm with no opportunity for human intervention. This may be especially important for agents deployed in strategic or safety-critical settings such as financial services, energy management, or scientific research.

Multi-agent AI systems introduce new kinds of reliability failures

Multi-agent AI systems introduce new kinds of reliability failures due to coordination failures or conflict between agents. In multi-agent AI systems, agents interact with each other while pursuing either shared or individual goals. For example, in a multi-agent system designed to conduct a research literature review, a lead agent decomposes the user’s query and assigns subtasks to specialised subagents, each responsible for researching a different aspect in parallel. While this allows for efficiency gains, it also means that errors can propagate between agents. If multiple agents are built on the same base model or incorporate the same tools, then they may also exhibit correlated failures. Empirical evidence for such failures in deployed systems remains limited, but these risks may grow as multi-agent systems become more common.

Box 2.4: Deliberate attacks can also cause AI systems to fail

This section focuses on unintended reliability failures, but malicious actors can also deliberately induce failures through attacks such as prompt injections. In a prompt injection attack, malicious instructions are presented to an agent indirectly via avenues like hidden instructions in websites or databases. These instructions can ‘hijack’ the agent, causing it to act against the user’s intentions. Such attacks are particularly difficult to defend against because they are delivered using external content outside the user’s or developer’s control. AI systems as targets of attack are discussed further in §2.1.3. Cyberattacks, and technical defences are covered in §3.3. Technical safeguards and monitoring.

Updates

Since the publication of the last Report (January 2025), commercial and research interest in AI agents has greatly increased. More AI agents are being deployed in the real world (Figure 2.11), most of which specialise in computer-use or software engineering applications. Recent releases such as XBOW hacking agent, Claude-4, and ChatGPT Agent demonstrate nascent autonomous capabilities such as creating slide decks based on Web searches. However, they cannot yet perform more complex tasks such as planning and booking travel since failure rates increase for longer tasks. Current research includes efforts to develop standards for how agents communicate with external tools and other agents. Examples include Google’s Agent2Agent and Agent Payments protocols, and Anthropic’s Model Context Protocol.

Evidence gaps

The main evidence gaps stem from the difficulty of reliably evaluating AI system capabilities, limitations, and failure modes (see §3.1. Technical and institutional challenges). Systematic evaluations of the reliability of AI agents are limited and lack standardisation. Certain issues, such as reliance on outdated information, may only manifest in real-world usage, making pre-deployment evaluations inadequate. Prior work has examined the reliability of agents and multi-agent systems in conventional software and earlier forms of AI. However, the applicability of this work to modern AI agents, which are often based on large language models, is unclear. Some researchers have raised concerns about the novel behaviours agents may exhibit in their interactions with each other, such as collusion or correlated failures, but empirical evidence remains limited. Efforts to address these gaps include the National Institute of Standards and Technology’s (NIST’s) new evaluations of agent-hijacking risks, the OECD’s AI Capability Indicators (243), and UK AI Security Institute’s Inspect Sandboxing Toolkit.

Mitigations

Techniques for improving AI reliability target both the model itself and the broader system in which it is deployed. These can reduce failure rates, but none can yet ensure the high reliability required in critical domains. An important technical measure is adversarial training, which exposes models to challenging inputs during training to help it develop more suitable, robust responses (see §3.3. Technical safeguards and monitoring). To reduce hallucinations, developers can apply retrieval-augmented generation (RAG), which supplements a model’s responses with information retrieved from an external database, helping ensure outputs are accurate and current, or specifically fine-tune models to be more factual or reason more effectively. Environment- or tool-based methods can also help developers monitor AI systems. For example, deployers could pilot AI systems in limited sandboxed environments to analyse potential failure modes before deploying them more broadly.

For AI agents specifically, researchers have proposed improving reliability through improved transparency, oversight, and monitoring. For example, monitoring agents’ interactions with external tools and with other agents would allow for more effective oversight of agent activities and incident analysis. Methods for collecting such information automatically, including in multi-agent settings, remain an active area of research.

Challenges for policymakers

Key challenges for policymakers include weighing the benefits of AI agent deployment against the risks of reliability failures, and ensuring that developers, deployers, and users have access to accurate information about agent performance and risk profiles. Deciding how to attribute liability for harms caused by AI agents poses a further challenge, particularly in multi-agent settings where it may be hard to identify when and how failures occurred. These challenges are compounded by the difficulty of evaluating agent reliability as agents gain autonomy and access to external tools. Uncertainty about how quickly agentic capabilities will emerge also makes planning for novel challenges difficult (see §3.1. Technical and institutional challenges regarding the ‘evidence dilemma’).