On the Opportunities and Risks of Foundation Models

In March 2021, we created an informal community at Stanford University of students, faculty, and researchers interested in some aspect of foundation models. From the very beginning, the community included not just AI researchers, but those eager to apply foundation models to their domain (e.g., healthcare and law), as well as those who were interested in societal concerns (e.g., ethics and economics). As discussions progressed, we noticed that there were many gaps in mutual understanding — how the technology worked, how industry develops foundation models, how to think about the ethical concerns, etc., and existing literature only covered bits and pieces. We wanted to therefore provide a fuller picture of foundation models, identify opportunities and risks, and establish a constructive vision for the future responsible development of foundation models.

The writing of this report was an experiment: we had over 100 people from different backgrounds come together to write a single report covering a wide range of aspects of foundation models. A large part of this report is a survey of existing work, but through many discussions, we have unified it in one report to highlight all the interdisciplinary connections.

Structure

The report is divided into 26 sections, each discussing one aspect of foundation models. The sections are grouped into four parts: capabilities (§2: capabilities), applications (§3: applications), technology (§4: technology), and society (§5: society), although there are many connections across sections. These connections highlight an integrated approach in which the technologies and capabilities are developed in a way that is sensitive to real societal concerns, while being inspired by and grounded out in applications.

While we have sought to capture most of the important topics surrounding foundation models, this report will inevitably be incomplete, especially as the field evolves quickly. For example, many applications (e.g., natural sciences, music, finance, agriculture) are not included, though they are as likely to be affected as the applications we have chosen to discuss. It would also be interesting to study how foundation models relate to research in neuroscience, cognitive science, and psychology to explain intelligence and aid efforts in computational social science to understand society.

Overview of capabilities

Foundation models acquire various capabilities that can power applications. We have chosen to discuss five potential capabilities: the ability to process different modalities (e.g., language, vision), to affect the physical world (robotics), to perform reasoning, and to interact with humans (interaction). Finally, we conclude with a philosophical discussion of potential limits on their capabilities.

§2.1: Language

NLP as a field has blazed the trail for foundation models. While these models dominate standard benchmarks, there is a clear gap between the capabilities these models acquire currently and those that characterize language as a complex system for human communication and thought. In response to this, we emphasize the full range of linguistic variation (e.g., different styles, dialects, languages), which poses an opportunity and challenge given some variants are data-limited. Further, child language acquisition is more sample efficient than the training of foundation models; we examine how signals beyond text and grounding may help to bridge this gap. Both of these characteristics of language provide clear directions for future foundation models research.

§2.2: Vision

Computer vision led the adoption of deep learning in AI (Russakovsky et al., 2015), demonstrating that models pretrained on large annotated datasets can transfer to numerous downstream settings. Now, pretraining on web-scale raw data instead of curated datasets, foundation models are on the rise in computer vision (e.g. Radford et al., 2021). These models have shown promising results for standard tasks in the field, like image classification and object detection, and training on multimodal and embodied data beyond images may enable progress on significant challenges (e.g., 3D geometric and physical understanding, commonsense reasoning). We also discuss some of the key challenges in modeling (e.g., the ability to scale effectively to videos) and evaluation (e.g., the measurement of higher-order capabilities) along with the applications (e.g., ambient intelligence for healthcare) and societal considerations (e.g., surveillance) that will determine the impact of foundation models for computer vision going forward.

§2.3: Robotics

A longstanding goal of robotics research is to develop “generalist” robots capable of performing myriad tasks across physically diverse environments. Unlike language and vision, which have led the way with foundation models both due to the abundance of raw data to train these models on and the availability of virtual applications to apply these models to, robotics faces fundamental challenges due to being anchored to the physical world. The principal challenge in developing new types of foundation models for robotics — different in nature than their language and vision counterparts — is acquiring sufficient data of the right form that is conducive to learning: we explore how plentiful data (e.g., generic videos of humans, amongst others) that is not specific to particular environments and across modalities (e.g., language, vision) may help to bridge this gap. These new robotic foundation models could allow for easier task specification and learning, ushering in new applications (e.g., better robotic assistance for household tasks) and heightening the importance of robustness and safety (e.g., formal safety evaluation).

§2.4: Reasoning and search

Reasoning and search problems such as theorem proving and program synthesis have been long-standing challenges in AI. The combinatorial search space renders traditional search-based methods intractable. However, humans are known to operate intuitively even in the most mathematical of domains (Lakoff and Núñez, 2000), and indeed existing work such as AlphaGo have already shown that deep neural networks can be effective in guiding the search space. But humans also transfer knowledge across tasks, facilitating much more efficient adaptation and the ability to reason more abstractly. Foundation models offer the possibility of closing this gap: their multi-purpose nature along with their strong generative and multimodal capabilities offer new leverage for controlling the combinatorial explosion inherent to search.

§2.5: Interaction

Foundation models show clear potential to transform the developer and user experience for AI systems: foundation models lower the difficulty threshold for prototyping and building AI applications due to their sample efficiency in adaptation, and raise the ceiling for novel user interaction due to their multimodal and generative capabilities. This provides a synergy we encourage going forward: developers can provide applications that better fit the user’s needs and values, while introducing far more dynamic forms of interaction and opportunities for feedback.

§2.6: Philosophy of understanding

What could a foundation model come to understand about the data it is trained on? Focusing on the case of natural language, we identify different positions on the nature of understanding and explore their relevance for our central question. Our tentative conclusion is that skepticism about the capacity of future foundation models to understand natural language may be premature, especially where the models are trained on multi-modal data.

Overview of applications

At present, foundation model research is largely confined to computer science and AI, with the impact of foundation models and the applications they support largely being centered in the tech industry. Moving forward, foundation models present clear potential to transform and extend the reach of AI across many sectors beyond the tech industry, suggesting a more pervasive effect on people’s lives. While there is a multitude of applications and domains to consider, we we have chosen three applications — healthcare, law, and education — because they represent foundational pillars of our society. For foundation models to significantly contribute to these application domains, models will require specific capabilities (§2: capabilities) as well as technical innovation (§4: technology) to account for the unique considerations in each domain. Further, since these domains are critical to societal function (§5: society), applying foundation models in these domains requires engaging with deeply sociotechnical matters such as those those pertaining to data (§4.6: data), privacy (§4.7: security), interpretability (§4.11: interpretability), fairness (§5.1: fairness) and ethics (§5.6: ethics).