International AI Safety Report 2026

General-purpose AI systems are already causing real-world harm. Malicious actors have used AI-generated content to deceive and defraud; AI systems have produced harmful outputs due to errors and unexpected behaviours; and deployment is impacting labour markets, information ecosystems, and cybersecurity systems. Furthermore, advances in AI capabilities may pose further risks that have not yet materialised. Understanding these risks, including their mechanisms, severity, and likelihood, is essential for effective risk management and governance.

This chapter examines risks from general-purpose AI systems that arise at the frontier of their capabilities. It organises these risks into three categories: (1) Risks from misuse, where actors deliberately use AI systems to cause harm; (2) Risks from malfunctions, where AI systems fail or behave in unexpected and harmful ways; and (3) Systemic risks, which arise from widespread deployment across society and the economy. These categories are not exhaustive or mutually exclusive – risks may cut across multiple categories – but they provide a structured way to analyse different mechanisms of harm.

This chapter is not an exhaustive survey of AI risks, and inclusion here does not necessarily imply a risk is likely, severe, or requires policy action. The evidence base varies considerably across sections. In some cases there is clear evidence of harm and effective ways to address it. In others, both the effects of general-purpose AI and the effectiveness of mitigations remain uncertain.

Risks from malicious use

AI-generated content and criminal activity

Key information

• General-purpose AI systems can generate realistic text, audio, images, and video, which can be used for criminal purposes such as fraud, extortion, defamation, non-consensual intimate imagery, and child sexual abuse material. For example, there are documented incidents of scammers using voice clones and deepfakes to impersonate executives or family members, and trick victims into transferring money.

• Accessible AI tools have substantially lowered the barrier to creating harmful synthetic content at scale. Many tools are free or low-cost, require no technical expertise, and can be used anonymously.

• Deepfake pornography, which disproportionately targets women and girls, is a particular concern. Studies show that 96% of deepfake videos online are pornographic. 15% of UK adults report having seen deepfake pornographic images and 2.2% of respondents in a 10-country survey reported that someone had generated non-consensual intimate imagery of them.

• Systematic data on the prevalence and severity of these harms remains limited, making it difficult to assess overall risk or design effective interventions. Incident databases and investigative journalism collect individual cases, but comprehensive analysis is lacking. Embarrassment or fear of further harm can make individuals and institutions reluctant to report incidents of AI-enabled fraud or abuse.

• Since the publication of the previous Report (January 2025), AI-generated content has become harder to distinguish from real media. In one study, participants misidentified AI-generated text as human-written 77% of the time. In another study of audio deepfakes, listeners mistook AI-generated voices for real speakers 80% of the time.

• Key challenges for policymakers include underreporting, detection tools that cannot keep pace with generation quality, and difficulty tracing content to creators. Additionally, some content – such as child sexual abuse material – is harmful even when correctly identified as AI-generated, meaning detection alone cannot fully address these risks.

Malicious actors use general-purpose AI systems to create realistic fake content for scams, extortion, or manipulation (see Table 2.1). General-purpose AI has made it much easier to scale the creation of fake content that can be used to harass or harm individuals, such as non-consensual pornographic videos. However, while cases of serious harm have been documented, comprehensive public data on the frequency and severity of these incidents remains limited, making it difficult to assess the full scope of the problem. This section focuses on how AI-generated fake content can cause harm, especially to individuals, other than by manipulation, which will be discussed in §2.1.2. Influence and manipulation.

Criminal uses of AI content

Malicious actors use AI-generated content for criminal purposes such as fraud, identity theft, and blackmail. For example, scammers use AI tools to generate voice clones or deepfakes to trick victims into transferring money. Documented incidents include executives authorising transfers of millions to fraudsters, as well as ordinary people sending smaller amounts to impostors posing as a loved one. Criminals also use AI-generated content for identity theft (e.g. by using a victim’s impersonated voice or likeness to authorise bank transfers or trick technical system administrators into sharing information such as login credentials); blackmail, to demand money, secrets, or nude images; or sabotage, by damaging individuals’ reputations for professional, personal, or political purposes. Researchers have also noted that deepfakes may risk undermining the reliability of evidence presented in court proceedings. While the number of reported incidents is rising (Figure 2.1), systematic data on the frequency or severity of AI-enabled crimes is limited. This makes it difficult to assess how much AI increases risk overall, and to design effective mitigations.

AI-generated sexual content

AI-generated sexual content has become more prevalent, including non-consensual intimate imagery that overwhelmingly targets women and girls. The realism and complexity of images that AI systems can generate has improved significantly (Figure 2.2). When provided with photos of a person, AI tools can now generate highly realistic images or videos of them in a range of scenarios, including sexually explicit ones.

AI-generated sexual content disproportionately targets women and girls

One study estimated that 96% of deepfake videos are pornographic, that 15% of UK adults report having seen deepfake pornographic images, and that the vast majority of ‘nudify’ apps explicitly target women. In another survey of over 16,000 respondents across 10 countries, 2.2% of respondents said that someone had generated non-consensual intimate images of them. Sexual deepfakes are also used in intimate partner abuse, again disproportionately affecting women. Public polling shows that people overwhelmingly view the generation of such images as deeply harmful. While many systems have safeguards to prevent such uses, users can sometimes bypass these or find alternatives that lack safeguards.

A particularly concerning use of AI tools is to generate sexually explicit content involving minors. In 2023, a study found hundreds of images of child sexual abuse in an open dataset used to train popular AI models such as Stable Diffusion. Children can also perpetrate abuse against their peers using AI-generated content. The overall prevalence of such activities is unclear. However, the number of reported incidents is rising. For example, schools have reported student use of ‘nudify apps’ to create and share AI-generated pornographic images of their (mostly female) peers. In another small study, 17 US-based educators expressed increasing concern about AI-generated non-consensual intimate imagery in schools.

Updates

Since the publication of the previous Report (January 2025), AI-generated content has become harder to distinguish from real content. In one study, after a five-minute conversation, participants misidentified text generated by OpenAI’s GPT-4o model as human-written 77% of the time. Similarly, other studies show that humans struggle to identify deepfakes, often performing no better than chance. For audio deepfakes, a study found that people took AI voice clones to be the real speaker in 80% of cases, suggesting heightened risks of impersonation. However, multimodal AI outputs combining video, audio, and text appear easier to detect than text or audio alone.

Evidence gaps

A key evidence gap stems from the lack of comprehensive and reliable statistics to assess the frequency and severity of harm from fake content. While more studies are documenting the rise of fake content (especially sexual content) and providing strong evidence of the resulting harms, most evidence comes from incident databases, such as the AI Incident Database and OECD AI Incidents and Hazards Monitor rather than systematic measurement or population-level studies. Key empirical evidence gaps remain, and there is little expert consensus, specifically around the prevalence of AI-enabled extortion, child sexual abuse material in schools, and sabotage. Reluctance to report such incidents may be a contributing factor. For example, institutions and individuals often hesitate to report AI-driven fraud due to embarrassment or fear of further harm. There is a need for multiple pathways through which incidents can be detected or reported.

Mitigations

Countermeasures that help people detect fake AI-generated content, such as warning labels and AI detection tools, show mixed effectiveness. Certain AI and machine learning tools can be trained to detect anomalies in images and videos and thus to identify fake images, but their effectiveness remains limited. Similarly, ‘warning labels’ designed to alert users to potentially misleading content have only a modest impact. For example, a study found that warning labels on AI-generated videos improved participants’ accuracy at identifying AI-generated videos from 10.7% to 21.6%, with most people still failing to spot deepfakes. Beyond detection, prevention-focused techniques include gating access to AI models – for example, limiting access to vetted users – and safeguards, such as classifiers, filters, or rules that prevent models from generating harmful or misleading content (see §3.3. Technical safeguards and monitoring). However, in the case of open-weight models, malicious actors can bypass these measures (see §3.4. Open-weight models). Filtering sexual content from models’ training data is also emerging as an effective method for increasing barriers to generating non-consensual intimate imagery.

Watermarking and content logs are promising methods for verifying content authenticity, but face technical shortcomings and raise privacy concerns. Watermarking involves embedding a machine-readable digital signature into the content during creation, allowing for automated traceable verification of its origin and authenticity. Researchers have proposed using watermarks to help consumers identify that content is AI-generated, including for videos, images, audio, and text. However, skilled actors can remove standalone watermarks or deceive detectors, reducing their effectiveness, especially in the case of open-weight models (§3.4. Open-weight models). A complementary approach is to embed watermarks or secure metadata, such as verifiable records of origin and creation, in authentic media. For example, recording devices can be required to embed unique digital signatures that help distinguish recordings made using them from AI-generated content. Another approach involves maintaining logs of AI outputs and using them to identify newly generated AI content by comparison. However, this approach faces scalability issues, is vulnerable to evasion, and raises privacy concerns related to logging user interactions. While not foolproof on their own, new research shows that a combination of these mitigations within a broader ecosystem of standards and policies can compensate for their respective limitations and help users detect AI-generated content more reliably.

Challenges for policymakers

Key challenges for policymakers include unreliable statistics, technical limitations, and rapidly evolving technology. Underreporting and unreliable statistics make it difficult to assess the full scale of harmful AI-generated content and choose effective interventions. Tracing AI-generated content back to the individuals who created it is also challenging, especially when open-weight models are used. Detection and watermarking techniques have improved but remain inconsistent and face technical challenges. Technical developments in AI content generation can also undermine their effectiveness. For example, a study found that deepfake detection benchmarks – curated examples of AI-generated and real media designed to test the performance of deepfake detection tools – are outdated and perform about 50% worse on real-world deepfakes than on the benchmarks usually used to evaluate them. These limitations mean that multiple layers of techniques are likely needed to detect AI-generated content with a high degree of robustness. Finally, it is important to note that harm from AI-generated content can occur even when the content is clearly identified as synthetic (e.g. child sexual abuse material), meaning detection alone cannot address all risks.