본문 바로가기
읽기 · 학술 · 공개 라이선스 원문

International AI Safety Report 2026

Yoshua Bengio (Chair) et al. · 2026

튜링상을 받은 컴퓨터 과학자 요슈아 벤지오가 위원장을 맡고 30여 나라와 국제기구의 전문가 100여 명이 함께 쓴 2026년 국제 AI 안전 보고서입니다. 범용 AI가 지금 무엇을 할 수 있는지, 악용 · 오작동 · 사회 전체에 미치는 위험이 어디까지 확인되었는지, 위험을 줄이는 방법은 무엇인지를 증거에 따라 정리합니다.

출처 Bengio et al. (2026), Department for Science, Innovation and Technology (UK), DSIT 2026/001 · 라이선스 OGL v3.0 · 원문

장 20개 올림 이 장 2154낱말
← 앞 장 다음 장 →

이 장은 원문만 있습니다. 문장별 해석은 앞 장부터 차례로 붙입니다.

10장 · Risks — Biological and chemical risks

Key information

• General-purpose AI systems can provide detailed information relevant to developing biological and chemical weapons. For example, they can generate instructions, troubleshoot procedures, and provide guidance to help malicious actors overcome technical and regulatory obstacles.

• AI systems now match or exceed expert performance on many benchmarks measuring knowledge relevant for biological weapons development. For example, one study found that a recent model outperformed 94% of domain experts at troubleshooting virology lab protocols. However, substantial uncertainty remains about how these capabilities affect risk in practice, given material barriers to weapons production and the difficulty of conducting uplift studies.

• Major AI developers have released (some) recent models with heightened safeguards after being unable to exclude the possibility that they could meaningfully assist novices in creating biological weapons. These safeguards, such as stronger input and output filters, aim to prevent the models from responding to harmful queries related to weapons development.

• Since the publication of the previous Report (January 2025), AI ‘co-scientists’ have become increasingly capable of supporting scientists and rediscovering novel scientific findings. AI agents can now chain together multiple capabilities, including providing natural language interfaces to users and operating biological AI tools and laboratory equipment.

• A key challenge for policymakers is managing dual-use risks while promoting beneficial scientific applications. Some AI capabilities that can be misused in biological weapons development are also useful for beneficial medical research, and most biological AI tools are open-weight. This makes it difficult to restrict harmful uses without hampering legitimate research.

AI systems can now provide detailed scientific information and assist with complex laboratory procedures, including generating experimental protocols, troubleshooting technical problems, and designing molecules and proteins. These capabilities have the potential to accelerate drug discovery, improve disease diagnostics, and broadly support scientific and medical research. However, they may also assist threat actors in creating biological and chemical weapons. By combining and interpreting existing complex information on the internet that is relevant to weapons development, and tailoring advice to specific malicious activities, AI systems can lower existing expertise barriers, allowing more actors to cause harm. In 2025, several major AI developers released new systems with additional safeguards after they could not rule out the possibility that these systems could assist novices in weapons development (see Box 2.2).

Substantial uncertainty remains about how much AI systems increase the overall level of biological and chemical risks. Some experts argue that remaining barriers – including acquiring equipment, obtaining regulated materials, and executing complex procedures – still pose significant challenges for novices seeking to develop weapons. Risk assessment in this domain faces significant technical and legal challenges (see Box 2.3).

Box 2.2: Developer risk assessments and mitigations

Major AI developers conduct pre-deployment risk assessments of new models to determine when additional safeguards are needed (see §3.2. Risk management practices). In 2025, several developers released models with additional precautionary safety measures, such as input and output filters, to prevent them from responding to harmful queries relating to weapons development.

OpenAI uses its Preparedness Framework to track capability levels, designating models as ‘High capability’ if they could “amplify existing pathways to severe harm”. OpenAI treats GPT-5-Thinking and ChatGPT-Agent as ‘high capability’, and they have activated the associated safeguards for the first time as a “precautionary approach” given a lack of “definitive evidence”.

Anthropic uses a Responsible Scaling Policy, which defines AI Safety Levels based in part on capability thresholds related to knowledge and abilities in the chemical, biological, radiological, and nuclear domains. Claude Opus 4 was the first model that Anthropic released at AI Safety Level 3, noting that, while testing did not find definitive evidence that the model had reached relevant capability thresholds, the company could not rule out that further testing would do so.

Google DeepMind uses a Frontier Safety Framework with Critical Capability Levels in various domains. Gemini 2.5 Deep Think was their first model to trigger a Critical Capability Levels early warning alert for chemical and biological risk, prompting additional mitigations.

Box 2.3: Challenges in assessing biological and chemical risks

It is challenging to accurately assess how AI systems affect chemical and biological risks due to legal constraints and international treaties, as well as ‘information hazards’ – information that may be harmful to share. For example, if researchers carry out, or publish the results of, a study on AI assistance in weapons development, they may risk inadvertently violating national security laws or treaties such as the Biological Weapons Convention and Chemical Weapons Convention. This is especially the case for real-world ‘uplift studies’, which systematically compare how well people perform a given task when they have access to an AI model or system, relative to a relevant baseline such as merely having internet access. As a result, researchers often rely on ‘benign proxy tasks’: tests that measure how much an AI system helps with similar but harmless procedures, such as synthesising pharmaceuticals or culturing low-risk bacteria. Relevant data is also often classified, particularly when it relates to the use of AI systems by state actors. This evidence gap exacerbates substantial uncertainty about the magnitude of AI-related biological and chemical risks.

General-purpose AI and weapon development

General-purpose AI systems can provide and contextualise information relevant to creating biological or chemical weapons

General-purpose AI systems can provide information relevant to various steps in creating biological and chemical weapons (Figure 2.7). This includes providing detailed instructions for obtaining and constructing pathogens and toxins, simplifying technical procedures, and troubleshooting laboratory errors. Safeguards designed to prevent harmful uses have improved over time but remain imperfect. For example, researchers bypassed filters by claiming that they need the information for legitimate research, asking about lesser-known chemical weapons, or using alternative terms.

While such information is already accessible on the internet, general-purpose AI systems allow novices to access and contextualise relevant information faster than they could with internet searches alone. Multimodal capabilities also allow AI systems to provide tailored advice in real time via video and audio troubleshooting. They can also provide some kinds of ‘tacit knowledge’, the practical expertise that is usually only built from hands-on laboratory experience. For example, one study showed that OpenAI’s o3 model is able to outperform 94% of domain experts at troubleshooting virology lab protocols. These capabilities have led some experts to argue that access to general-purpose AI makes biological or chemical weapons development somewhat easier than internet access alone does.

Relevant capabilities have improved but evidence of real-world uplift is mixed

In a recently published real-world uplift study, general-purpose AI systems without relevant safeguards provided substantial assistance in bioweapon acquisition proxy tasks, compared to a baseline of internet access only. Previous uplift studies found no or small, generally statistically insignificant effects. However, these studies had potentially unrepresentative participants and small sample sizes, and they have quickly become outdated as AI capabilities have improved (Figure 2.8). The Frontier Model Forum – an AI industry consortium – has jointly funded an additional uplift study to assess real-world novice uplift, but has not yet reported their results.

Effects of AI tools

AI-enabled biological and chemical tools are AI models trained on biological or chemical data that can identify, categorise, or design novel biological or chemical entities. First, some such tools, such as ‘biological foundation models’, can be adapted to perform a wide variety of scientific tasks within their domain, placing them within this Report’s definition of general-purpose AI (see Introduction). Second, general-purpose AI agents can now operate more specialised tools, making them more accessible to lower-skill users through natural language interfaces.

These tools can accelerate biological and chemical research, including research with misuse potential. For example, Google DeepMind’s AlphaProteo can generate novel protein designs. AI-generated designs often fail to function as intended, requiring real-world testing to identify working candidates. However, since testing AI-generated designs is much faster than generating designs manually, these tools can still accelerate research overall. AI agents can further speed up workflows by automating the cycle of iteratively designing and testing proteins.

Tools are increasingly accessible through chat interfaces and integrations

Natural language interfaces are making these tools increasingly accessible. Developers are integrating chat interfaces into chemical and biological design software, allowing inexpert users to operate sophisticated tools. There is little research on how much more accessible these integrations make such tools, and the effect on overall risk – particularly for novices versus those with existing expertise – is unclear.

AI tools can be adapted to design pathogens and toxins

Biological foundation models can generate designs for novel pathogens. Recently, researchers demonstrated that a biological foundation model could generate a significantly modified virus from scratch. This study represents the first instance of genome-scale generative AI design, albeit with the important caveat that the generated virus infects bacteria rather than humans. Some models can also generate designs for novel pathogens more harmful than their natural equivalents.

Experiments have shown the potential for similar risks with narrower chemical and biological tools. For example, some tools have been specifically designed for toxin creation and can generate modified designs for known toxins, such as ricin. In one early demonstration, a tool designed to reduce molecular toxicity was repurposed to increase it with trivial modifications. However, legal barriers and treaty obligations pose challenges for researchers seeking to study the effectiveness of AI-designed toxins or harmful proteins (see Box 2.3). Such tools also have many beneficial applications, including predicting pathogen properties and designing components for therapeutic purposes. Developers are releasing many more of them over time (Figure 2.9) and integrating them with natural language interfaces to make them more accessible to users without specialist expertise.

Some AI-enabled biological tools are restricted, but others are widely accessible

Access to AI-enabled biological tools varies. Some, such as Google DeepMind’s AlphaProteo, are restricted to select researchers. Others, such as ConoDL, are open-weight and widely accessible. One recent study found that 23% of the highest-performing tools had high misuse potential due to dangerous capabilities and accessibility, and 61.5% of these were fully open source, making them accessible to potential malicious actors. Another study found that only 3% of 375 biological AI tools surveyed had any form of safeguards.

Updates

Since the publication of the previous Report (January 2025), some AI companies implemented additional risk mitigations for their new models (see Box 2.2). Furthermore, AI ‘co-scientists’ are increasingly capable: they can meaningfully support top human scientists and rediscover novel, unpublished scientific findings. Multiple research groups have developed specialised scientific AI agents capable of performing tasks including literature review, hypothesis generation, experimental design, and data analysis. Controlled studies and new benchmarks suggest that AI systems can provide substantially more weapons development assistance than the internet alone, but larger studies are needed to confirm these results.

Evidence gaps

The primary evidence gaps relate to translating demonstrated capabilities into risk estimates. Comprehensive studies measuring how AI systems affect actual weapons development are rare, expensive, and constrained by legal and ethical considerations (see Box 2.3). Chemical risk evaluations have received relatively less attention than biological risk evaluations. Across both chemical and biological risk evaluations, results are reported with varying levels of detail or withheld entirely due to sensitivity concerns. Evaluations also generally assess the capabilities of individual tools, making them less applicable to real-world end-to-end workflows which might involve multiple AI systems. As such, it is unclear whether these evaluations under- or overestimate risk. Finally, there is ongoing debate about whether harmful AI capabilities primarily empower malicious actors with existing expertise (increasing their efficiency) or enable novices with little prior knowledge.

Mitigations

A range of technical mitigations are being developed, both within and outside of AI models, to address these risks. For general-purpose AI systems, major developers have implemented safety controls designed to refuse harmful requests. Technical mitigations for specialised biological and chemical AI tools tend to lag behind those for general-purpose AI systems. Other safeguards include excluding pathogen data from training, restricting access to high-risk tools, training models to refuse queries involving pathogenic viruses, and watermarking outputs. However, many of these safeguards have not been thoroughly tested, and can be removed from open-weight models.

Another focus for technical mitigations is screening DNA synthesis requests in order to prevent malicious actors from acquiring material necessary for bioweapons creation. Using synthetic DNA is likely the most straightforward way to create modified pathogens and it allows malicious actors to avoid using infectious source material. Screening is complemented by extending infectious disease surveillance frameworks to better detect novel threats and intentional attacks. Biological risks – whether AI-enabled or not – can probably be at least partially mitigated by improving biosecurity directly through reducing indoor pathogen transmission, developing broad-spectrum antivirals, and improving laboratory biosecurity and biosafety globally. Greater facilitation for data-sharing between relevant actors could aid in identifying and addressing potential threats. Using AI to improve pathogen detection and vaccine and drug development is likely a key mitigation strategy, especially given the limitations of current safeguards.

Challenges for policymakers

The dual-use nature of AI for biological and chemical capabilities poses challenges to policymakers wanting to limit the risk of potentially harmful uses while enabling beneficial research. The open availability of biological AI tools presents a difficult choice: whether to restrict these tools or to actively support their development for beneficial purposes (see §3.4. Open-weight models).

이 글은 위 출처의 원문을 그 라이선스(OGL v3.0)에 따라 옮겼습니다. 참고문헌 목록과 인용 번호, 그림 · 표 · 각주 · 수식 · 부록은 빼고 절별로 나누어 실었으며, 그 밖의 문장은 원문 그대로입니다. 원 저작자와 발행 기관은 이 화면을 만들거나 검토하지 않았고, 이곳의 해석과 낱말 풀이는 공식 번역이 아닙니다.
Contains public sector information licensed under the Open Government Licence v3.0.