International AI Safety Report 2026

Key information

• A wide range of technical safeguards is used at different stages of AI development and use. These include techniques applied during model development to make systems more robust and resistant to misuse (such as data curation), deployment-time monitoring and control (such as content filtering and human oversight), and post-deployment tools to monitor the broader AI ecosystem (such as provenance and content detection).

• Technical safeguards have limitations and do not reliably prevent harmful behaviour in all contexts. For example, users can sometimes obtain harmful outputs by rephrasing requests or breaking them into smaller steps. Similarly, tools such as watermarking which are designed to identify AI-generated content can often be removed or altered, which limits their reliability.

• The limitations of individual safeguards mean that ‘defence-in-depth’ may be needed to prevent certain harmful outcomes. For example, a system might combine a safety-trained model with input filters, output filters, and content monitors.

• Since the publication of the last Report (January 2025), researchers have made progress on improving safeguards, but fundamental limitations remain. For example, the success rate of attacks designed to bypass safeguards has been falling, but remains relatively high. There are also fundamental limitations to how thoroughly open-weight models can be safeguarded.

• A key challenge for policymakers is the limited evidence on how effective safeguards are across diverse real-world uses of general-purpose AI systems. AI developers vary widely in how much information they share about their safeguards and monitoring. A further challenge is the potential trade-offs between applying stronger safeguards and maintaining system performance or usefulness.

AI developers can use several useful but imperfect technical safeguards to mitigate and manage risks from general-purpose AI systems, yet robustness challenges persist. Developers still cannot fully prevent general-purpose AI systems from performing even well-known and overtly harmful actions such as offering users instructions for committing crimes. For example, researchers have shown that state-of-the-art safeguards can be circumvented through adversarial prompting methods (i.e. ‘jailbreaks’), by having models break down complex harmful tasks into steps, and with simple model modifications. Researchers continue to work on safeguards against malfunctions and misuse. These methods vary widely in their purpose and effectiveness, and their impact ultimately depends on the broader sociotechnical and governance context in which AI systems are built and deployed.

Technical safeguards can broadly be divided into three categories: techniques for developing safer models; techniques used during deployment for monitoring and control; and techniques that support post-deployment ecosystem monitoring. Table 3.6 summarises the technical safeguards discussed, their effectiveness, and open challenges.

Developing safer models

A first line of defence against harms from general-purpose AI systems is to make the underlying model safer. This subsection covers safeguards that are ‘baked into the model parameters’ during the model development process (Figure 3.6).

Curating training data can limit the development of potentially dangerous capabilities

General-purpose AI models are useful precisely because they develop a broad range of knowledge and capabilities after processing training data, but some types of training data are disproportionately responsible for the development of potentially dangerous capabilities. For example, an AI model trained on virology papers might be better able to provide assistance in potentially harmful biology tasks (see also §2.1.4. Biological and chemical risks). Additionally, image/video generators trained on images of human nudity can also be misused for creating non-consensual intimate deepfakes (see also §2.1.1. AI-generated content and criminal activity).

Filtering training data is an effective mitigation against some undesired capabilities. However, it can be difficult to filter the large datasets used to train general-purpose AI models due to high costs, filtering errors, and negative impacts on the quality of the dataset. These challenges are exacerbated by the multilingual nature of internet text, cultural biases in content moderation, and the fact that whether a given piece of data is ‘harmful’ depends on contextual factors. Nonetheless, filtering potentially harmful material from training data shows promise for making models more reliably safe, including making open-weight models more resistant to harmful tampering. The relationships between training data contents and emergent model capabilities are not yet fully understood, and filtration seems to be more effective for limiting harmful capabilities when applied to broad domains of knowledge compared to narrower behaviours. See §3.4. Open-weight models for further discussion.

Methods for training general-purpose AI models to be helpful and harmless mainly rely on human feedback

It is difficult to train and evaluate models to reliably align with high-level principles such as being helpful, harmless, and honest. In practice, developers aim to accomplish this by fine-tuning AI models using demonstrations and feedback from humans. For example, the principal paradigm for fine-tuning AI models, known as ‘reinforcement learning from human feedback’, is based on training models to produce outputs that human annotators rate positively. However, positive feedback from humans is a flawed proxy for beneficial behaviour and is constrained by human error and bias.

This leads to several challenges: models fine-tuned by reinforcement leaning from human feedback sometimes pander to the user, a behaviour known as ‘sycophancy’; provide responses that are helpful in some contexts but harmful in others; provide responses that are difficult to evaluate for correctness; or perform actions whose helpfulness or harmfulness is a matter of opinion. Table 3.7 provides examples of these challenges. Some research aims to develop methods for helping humans better evaluate solutions to complex tasks with AI assistance. However, these methods currently have limited reliability, and the extent to which they are used to train today’s most advanced AI models is not publicly known.

Humans do not always agree on what behaviours are desirable, requiring methods to balance competing preferences

Humans do not always agree about what responses or actions AI models should or should not output. This makes it fundamentally challenging to develop models whose actions and impacts are broadly aligned with the interests of society. Some researchers study whose preferences are reflected in AI systems and work to develop ‘pluralistic alignment’ techniques that aim to strike a balance between competing preferences. For example, AI developers can design systems to avoid generating controversial answers by refusing to respond to certain requests, or align with the median viewpoint in some relevant sample of people, or personalise systems to individual users.

A common challenge for these approaches is that, in general, AI systems cannot equally align with everyone’s preferences, and that their downstream societal impacts will affect various groups of people differently. Some researchers have argued that most technical approaches to pluralistic alignment fail to address, and potentially distract from, deeper challenges, such as systematic biases, social power dynamics, and the concentration of wealth and influence.

AI developers use ‘adversarial training’ to improve model robustness

It is challenging to ensure that AI models robustly translate the beneficial behaviours they learn during training to real-world deployment contexts. Even models trained with a ‘perfect’ learning signal can fail to generalise successfully to all unseen contexts. For example, some researchers have found that chatbots are more likely to take harmful actions in languages that are underrepresented in their training data, which includes many languages predominantly spoken in the Global South.

In recent years researchers have also created a large toolkit of ‘adversarial attack’ techniques that can be used to make models generate potentially harmful responses. For example, a recent initiative crowd-sourced over 60,000 diverse examples of successful attacks against state-of-the-art AI models, which made them violate their companies’ policies on acceptable model behaviour. Table 3.8 shows examples of ‘jailbreak’ techniques that researchers have shown can make models comply with harmful requests.

One method to improve the robustness of models is known as ‘adversarial training’. It involves constructing ‘attacks’ (e.g. jailbreaks) designed to make a model act undesirably, and training the model to handle these attacks appropriately. However, adversarial training is imperfect. Attackers are consistently able to develop new successful attacks against state-of-the-art models. Since developers require specific examples of failure modes in order to train against them, the result is an ongoing ‘cat and mouse’ game in which developers continually update models in response to newly discovered vulnerabilities, and adversaries continually seek new attacks. Some researchers have proposed larger-scale adversarial training or new algorithms to improve robustness, but modern AI systems remain persistently vulnerable.

‘Unlearning’ techniques can mitigate specific harmful model capabilities

Another strategy for mitigating risks from general-purpose AI is to fine-tune models to lack capabilities in specific high-risk domains. For example, researchers are working to develop ‘machine unlearning’ algorithms that can specifically suppress abilities related to biothreats or to generating photorealistic images of nude human bodies. These methods can make models substantially safer, at the expense of limiting some positive uses of the unlearned capabilities. Limiting AI models’ knowledge in harmful domains has also been proposed as a way of designing ‘tamper-resistant’ open-weight models that can resist harmful fine-tuning. Thus far, however, this has been challenging to do robustly. See §3.4. Open-weight models for further discussion.

Some researchers are working on methods for stronger safety assurances through interpreting model internal states or mathematical verification

Some researchers are working on methods to more rigorously verify safety-related properties of models. In one approach, researchers aim to interpret the internal computations of models to either identify risks or to make more convincing arguments that the model is safe. For example, in a proof of concept, researchers showed that tools for analysing the internal computation of a language model could help evaluators identify harmful behaviours. In 2025, Anthropic also began analysing model internals as a way of studying model situational awareness and ‘intent’. However, these types of methods are currently not common or known to be competitive with other evaluation techniques.

A different approach for making stronger assurances of safety involves constructing mathematical proofs that a model will satisfy certain safety conditions. However, these proofs assume that the testing context matches the deployment context, and are untested against many types of adversaries. They also cannot currently be scaled to large models. Overall, there is significant debate among experts over the promise of interpretability and formal verification methods.

Deployment-time monitoring and control

In addition to safeguards implemented during model development, a second line of defence against harmful behaviours is external safeguards that focus on monitoring and controlling a model or system’s actions during deployment. Such safeguards help mitigate malfunctions and misuse, such as hallucinated outputs and harmful instructions.

Model deployers can use a variety of tools to identify and address high-risk model behaviours

When an AI system is running, a deployer can monitor for signs of risk and intervene if they appear. For example, they can inspect a model’s inputs for signs of adversarial attacks, filter inappropriate content from outputs, or monitor the system’s chain of thought for signs of harmful plans. Points where deployers can monitor and intervene on how people are using their systems include hardware, user interactions, inputs and outputs, internal computations, and chain of thought. There are also multiple actions deployers can take when risks are identified. These include logging information, filtering/modifying harmful content, flagging abnormal activity, system shutdowns, or triggering failsafes. Figure 3.7 illustrates examples of common monitoring and control mechanisms.

Because they are versatile and often effective, these mechanisms are widely used and can prevent many kinds of unintentional harms. However, these safeguards are imperfect, especially under malicious attacks optimised to make them fail. Recent research has also explored how monitoring can be unreliable if a system is optimised using the scores of a monitor, for example, by making chain of thought less reliable.

Humans in the loop allow for direct oversight in high-stakes settings

To reduce the chance of failures from AI agents (see §2.2.1. Reliability challenges), deployers can aim to design AI systems that work in cooperation with humans rather than fully autonomously. This is important for use cases where incorrect decisions can lead to significant harm, such as in finance, healthcare, or policing. However, having a ‘human in the loop’ is often impractical. Sometimes decision-making happens too quickly, such as in chat applications with millions of users. In other cases, human bias and error can amplify risks due to compounding errors. Humans in the loop also tend to exhibit ‘automation bias’, meaning that they often place more trust in the AI system than is warranted (see §2.3.2. Risks to human autonomy).

‘Sandboxing’ protects against risks from autonomous behaviours

AI agents that can act autonomously without limitation on the Web or in the physical world pose elevated risks (see §2.2.1. Reliability challenges). ‘Sandboxing’ involves limiting the ways in which AI agents can directly influence the world, making it much easier to oversee and manage them. For example, restricting an AI system’s ability to post to the internet or edit a computer’s file system can prevent unexpected harms from unexpected actions. However, these approaches cannot always be used for applications where an AI system must necessarily act directly in the world.

Ecosystem monitoring tools: model and data provenance

Model and data provenance tools are technical tools for studying the AI ecosystem, to improve awareness of the downstream uses and impacts of AI systems.

AI system provenance techniques help trace the uses and impacts of systems

Developers and deployers can use various techniques to study model usage and spread ‘in the wild’. For example, they can give models unique identifying behaviours or apply unique patterns to the weights of individual open-weight models. However, making these techniques more resistant to model modifications is an open problem. Researchers are also working on methods for ‘inferring model heritage’, helping to answer questions of the form: ‘Was model X a fine-tuned or distilled version of model Y?’ Finally, some developers are working toward protocols and infrastructure for AI agents to facilitate identification and verification when they interact with external systems.

AI content detection techniques help monitor the spread and impacts of AI-generated content

Watermarks, metadata, and other AI content detectors can help researchers track and study the real-world impact of AI-created content. First, data watermarks are subtle but distinct motifs inserted into digital media that can encode information about their origin. For text, they typically take the form of subtle biases in word choices and style; for images and video, subtle patterns over pixels; and for audio, subtle patterns in audio waves. Figure 3.8 illustrates these.

Aside from watermarks, AI-generated content can also be saved using file formats that store metadata about how they were generated. For example, many mobile devices save image and audio files using a file format that can store information about camera settings, time, location, etc. Similar metadata can be used to store information about whether data was generated by an AI system. Much like fingerprinting in criminal forensics, watermarks and metadata can be tampered with or removed, but are nonetheless useful.

Researchers are also working to develop AI-generated content detectors to help identify AI-generated content in the wild, even when no watermark or metadata is available. However, these identification techniques have a limited success rate.

Updates

Since the publication of the last Report (January 2025), progress has been made in developing AI systems with multiple effective layers of safeguards. As discussed in §3.2. Risk management practices, defence-in-depth is a core principle in risk management. For example, AI systems that combine safety-trained models with input filters, output filters, and other content monitors are increasingly studied and deployed. Recent research has also shown that, while model developers have made progress in increasing robustness to attempts to bypass safeguards, attackers still succeed at a high rate (Figure 3.9).

Evidence gaps

More evidence is needed to help researchers understand and account for the limitations of existing approaches. Technical safeguards for AI systems are being improved, but techniques suffer from limitations. For example, progress on improving the worst-case robustness of general-purpose AI systems has been slow, and there are fundamental limitations to how thoroughly open-weight models can be safeguarded and monitored (see also §3.4. Open-weight models). Meanwhile, not all technical safeguards are equally common, equally effective, or have been equally proven in the real world. For example, adversarial training is almost ubiquitously used on state-of-the-art models, while model interpretability and formal verification techniques have seen little use to date in production systems.

Challenges for policymakers

Key challenges for policymakers include deciding whether and how they should support research, development, evaluation, and adoption of technical safeguards and monitoring methods. This is challenging because scientists’ understanding of how best to practically safeguard mechanisms is still evolving and best practices are not yet established. For example, different developers apply different safeguards, and their approaches to technical risk mitigation more broadly vary widely. Finally, the existence of effective technical safeguards does not, on its own, ensure safety, as adoption and implementation can vary across developers and deployment contexts.