International AI Safety Report 2026

Key information

• AI systems can cause harm by generating content that influences people’s beliefs and behaviour. Some malicious actors intentionally use AI-generated content to manipulate people, while other harms, such as dependence on AI, occur unintentionally.

• A range of laboratory studies have demonstrated that interacting with AI systems can lead to measurable changes in people’s beliefs. In experimental settings, AI systems are often at least as effective as non-expert human participants at persuading other people to change their views. Evidence on their effectiveness in real-world settings, however, remains limited.

• The content AI systems generate could become more persuasive in future due to improving capabilities, increased user dependence, or training on user feedback. The factors that shape how widespread, impactful, and potentially harmful this content will be are not well understood. Some evidence from theoretical work and simulations suggests factors such as distribution costs and the inherent difficulty of persuasion will limit the impacts.

• Since the publication of the previous Report (January 2025), evidence of AI systems’ capability to produce manipulative content has increased. The latest research suggests that people who interact with AI systems for longer and in more personal ways are more likely to find their content persuasive. Evidence has also grown that AI systems can have manipulative effects through sycophancy and impersonation.

• There is mixed evidence regarding the effectiveness of all proposed mitigation strategies. Manipulation can be difficult to detect in practice, making it challenging to prevent through training, monitoring, or safeguards. Efforts that aim to minimise manipulation risks could also curtail the usefulness of AI systems (e.g. as educational tools).

Hundreds of millions of people now interact with AI-generated content daily, through chat assistants, social media, customer service bots, companion apps, and other services. This content can shape their opinions, purchasing decisions, and actions. Much of this influence is benign or even beneficial, but AI-generated content can also be used to manipulate people: to change their beliefs or behaviours without their full awareness or consent.

Forms and harms of AI manipulation

Experts often distinguish ‘manipulation’ – influencing someone in order to achieve a goal without their full awareness or understanding – from ‘rational persuasion’: influencing someone using honest and rational arguments so that they authentically endorse their new beliefs. In practice, this distinction is contentious: researchers disagree about how to identify harmful manipulation and separate it from legitimate influence. As such, while this section is primarily focused on harmful manipulation, it also discusses other types of persuasion that some might regard as neutral or even beneficial.

Possible harms of AI manipulation range from individual exploitation to systemic erosion of trust

General-purpose AI systems can produce a range of persuasive content (Figure 2.3), and this content can create or exacerbate several risks. When this content is manipulative, many ethicists regard it as intrinsically harmful because people who are manipulated are not in control of their own behaviour (cf. §2.3.2. Risks to human autonomy). More directly, malicious actors can use AI to manipulate people into making harmful decisions. For example, criminals can use AI-generated content in social engineering to manipulate people into sending money or sensitive information (cf. §2.1.1. AI-generated content and criminal activity), while political actors may use AI systems to spread extremist views.

AI-generated content may also have unintended manipulative effects. For example, multiple studies have found that AI products that developers have optimised for user engagement (such as some AI companions) can foster psychological dependence, reinforce harmful beliefs, or encourage users to take dangerous actions (cf. §2.3.2. Risks to human autonomy). At a systemic level, the spread of AI-generated manipulative content could erode public trust in information systems and, in loss of control scenarios, help AI systems evade oversight and control measures (cf. §2.2.2. Loss of control). This section primarily focuses on the misuse of AI to manipulate, but much of the evidence discussed is relevant across these risks.

Effectiveness and scale of manipulative AI content

General-purpose AI matches human performance at influencing others in experimental settings

Several studies have found that, in experimental settings, AI-generated content can influence people’s beliefs at least as effectively as non-expert humans can. These studies generally measure people’s self-reported agreement with a statement before and after exposure to AI-generated content: either static text or a multi-turn conversation. A large number of studies have found that exposure to AI-generated content can significantly change people’s opinions and behaviour. Persuasiveness also increases with the scale of the model used (Figure 2.4). Some of these studies have compared AI systems to humans and found that AI systems are as or more convincing than non-expert humans (see Table 2.2), and can match the convincingness of human experts in writing static text. For example, in one study, people changed their beliefs about the correct answer to trivia questions by 17 percentage points after interacting with general-purpose AI systems, versus only 9 percentage points after interacting with other humans.

Real-world use of AI to influence people is documented but not yet widespread

Outside of laboratory settings, researchers have documented a range of examples of AI-driven influence. Malicious actors have attempted to use AI systems to alter people’s political opinions, or to make them share sensitive information or give away money (cf. §2.1.1. AI-generated content and criminal activity). Many companies are beginning to place sponsored content in AI chat conversations or deploy AI sales agents to sell products to users on their websites. AI companion apps have attracted tens of millions of users and some users have developed strong emotional dependence, delusions, or even taken their own lives after extended interactions with chatbots, though investigations into these incidents is ongoing (see §2.3.2. Risks to human autonomy). Consumers are also increasingly using AI to influence others. One study estimated that AI-written complaints were 9 percentage points more likely to secure compensation than human-written ones.

However, there is limited systematic evidence that real-world AI manipulation is currently widespread or effective relative to human-generated content. Investigations by AI providers into AI-powered influence operations have found little evidence that people widely shared the content, and only around 1% of content flagged as misleading on social media is classified as AI-generated. There are theoretical reasons why manipulation might be harder in the real world than in the lab. Distribution costs – getting content in front of people – are often larger than the cost of generating content. On the viewer’s side, the costs of being wrong and changing one’s beliefs are higher in real-world settings, and if individuals are exposed to multiple competing viewpoints, this could limit the impact of any one source.

Changes in coming years

Many factors could increase the manipulative capabilities of AI systems, but there is limited evidence on how large these effects will be. One study suggests that for each 10x increase in the computing power used to train models, persuasiveness increases by around 1.8 percentage points. There is mixed evidence on whether techniques such as personalisation will lead to increased persuasiveness, with some studies showing positive effects (~3 percentage points) and others small or null effects. Current training methods such as reinforcement learning from human feedback may reward models for manipulating users, inadvertently training models to produce more manipulative outputs. Moreover, studies have shown that explicitly training models based on feedback about whether or not the user was convinced can further increase persuasive effects. Novel interfaces, such as AI browsers, could amplify these risks by providing AI systems with more access to data and more influence over user actions. AI agents may pose greater manipulation risks since they can take actions such as conducting research, buying products or services, and interacting with third parties. For example, they could order presents for targets or blackmail them. If users continue to become more emotionally attached to AI systems and rely on them more for advice, the systems’ influence could further increase (see also §2.3.2. Risks to human autonomy).

Updates

Since the publication of the last Report (January 2025), the number of users engaging with AI systems has increased rapidly, with 700 million people using OpenAI’s ChatGPT every week, up from 200 million a year before. Additionally, tens of millions of individuals report using AI companion services (see §2.3.2. Risks to human autonomy). This has shifted both theoretical and empirical work from highlighting risks like broadcasting misleading content at scale, to more subtle forms of manipulation such as sycophancy and emotional exploitation.

Evidence gaps

There is limited understanding of how AI manipulation works, and whether AI systems are equally capable of inducing true and false beliefs in people. While some studies have demonstrated the durability and robustness of AI systems’ influence, more research is needed to assess these effects under realistic conditions, and to investigate the role of AI systems that distribute content, such as social media platforms. However, evaluating manipulation in realistic settings can be challenging due to ethical concerns. Lastly, more interdisciplinary and sociotechnical research is needed into how people’s relationships with AI will change as they interact more closely with it, and as AI systems are trained to adapt to people’s psychology.

Mitigations

Some proposed mitigations focus on training AI models to avoid producing manipulative outputs, but most of these show mixed success or require cumbersome evaluations. Models could be trained to generate true outputs, but this requires developers to define ‘truth’ (a thorny concept), and can backfire by inadvertently rewarding models for generating subtler deceptive outputs that are harder to detect. Models might also be trained to promote users’ autonomy or wellbeing, but this requires them to navigate between what users want in the moment (e.g. more engagement) and what they say they want, given more time to reflect (e.g. a more fulfilling life). Monitoring for manipulative outputs faces similar challenges in defining ‘manipulation’ and requires monitors to have access to model outputs.

Alternative mitigations, which focus on protecting users, provide some value but may not be sufficient on their own. Some researchers have suggested that improved education or AI literacy could mitigate manipulative effects, but there is limited evidence for these claims. Labelling content as AI-generated has not proven effective at reducing manipulation, and users who are knowledgeable about AI or interact with it frequently are just as likely to be deceived.

Challenges for policymakers

Policymakers face several challenges: manipulative AI outputs are difficult to identify and evaluate, and there is limited evidence on what makes AI-generated content more or less manipulative. It is challenging to precisely target manipulation through training or regulation: interventions which limit harms from manipulation will likely curtail beneficial educational, emotional, and commercial applications of AI. Capability evaluations are not an exact science and may over- or underestimate persuasive effects, making it challenging for policymakers to evaluate risks. In future, risks could increase sharply via training and dependence, or plateau due to real-world complications. Finally, proposed mitigations are not well-tested and face fundamental challenges. For example, training models to be truthful or promote autonomy requires defining these contested concepts.