On the Opportunities and Risks of Foundation Models

The technological foundations of foundation models give rise to the capabilities (§2: capabilities) that determine their potential. To understand the technology used in development, we consider the data (§4.6: data), model architectures (§4.1: modeling) and systems (§4.5: systems) used to train (§4.2: training), and further adapt, (§4.3: adaptation) these models alongside the theory (§4.10: theory) that should be developed to understand this paradigm. To then understand the resulting models, we discuss how to evaluate (§4.4: evaluation) and interpret (§4.11: interpretability) alongside the importance of robustness (§4.8: robustness), security and privacy (§4.7: security), and long-term AI safety (§4.9: ai-safety) for ensuring the reliability of these models when deployed in society (§5: society).

Modeling

The emerging paradigm of foundation models has attained impressive achievements in AI over the last few years, as models such as BERT (Devlin et al., 2019) shine at a wide spectrum of language understanding tasks: from textual classification and entailment to question answering and reading comprehension, while GPT-3 composes rich and fluent tales about unicorns (Brown et al., 2020) and DALL-E shows signs of visual creativity, generating from scratch strikingly-realistic pictures of avocado chairs (Ramesh et al., 2021).

These and other instances of recent foundation models not only achieve remarkable performance across a multitude of diverse downstream tasks and applications (Rajpurkar et al., 2018; Wang et al., 2019a), but also manifest noteworthy behaviors of interpretability (Karras et al., 2020), robustness (Devlin et al., 2019), controllability (Patashnik et al., 2021) and generalization (Brown et al., 2020). What does it take for a model to demonstrate such qualities? What architectures are capable of consuming large quantities of potentially multimodal information and translate them into rich knowledge of the world? And overall, what desirable properties should a network possess to give rise to a foundation model?

Here, we identify and discuss five such properties, spanning expressivity, scalability, multimodality, memory capacity, and compositionality, that we believe are essential for a foundation model in order to: (1) distill and accumulate knowledge from various sources and domains, (2) organize it in an effective and scalable representation, and (3) flexibly generalize it towards novel contexts. For each of these properties, we motivate their necessity, provide examples of contemporary models that incorporate them, and explore key challenges and promising avenues for future research and development. See Figure 17 for an overview diagram.

Expressivity

Expressivity concerns with the theoretical and practical capacity of a network to model the data distribution it is trained over and represent it in a flexible manner. Prior works have proposed formal expressivity measures to characterize the complexity of functions a network can compute, or more precisely, approximate, which is essentially affected by its depth, width, connectivity, and structural patterns (Raghu et al., 2017).

As the No Free Lunch theorem suggests, there is no single model or algorithm that suits best for all cases (Wolpert and Macready, 1997), and so, for our purposes, we are particularly interested in identifying which models could effectively capture the facets of natural information, such as human language or real-world images (Goodfellow et al., 2016). These modalities are either continuous (as in vision) or discrete (as in language), are distinctly hierarchical and high-dimensional, and present a complex set of relations and interactions among their constituent elements, whether these are pixels, words or physical objects.

Indeed, recent breakthroughs in generative modeling provide strong evidence for the high expressivity of neural networks, as they successfully express distributions of textual (Brown et al., 2020; Devlin et al., 2019; Lieber et al., 2021; Wang and Komatsuzaki, 2021), auditory (van den Oord et al., 2016), and visual (Karras et al., 2020; Brock et al., 2018) domains, and generate samples of high fidelity, diversity and realism.

Inductive Biases

Much of the success of neural networks over the last decade in modeling natural data is owed to the networks’ high depths, as could be roughly measured by the number of stacked non-linear layers they are composed of, or the number of computational steps they take during their chain-of-reasoning. Great depths play a crucial role in enhancing networks’ expressivity, allowing them to form powerful hierarchical and distributed representations that could generalize from the training data to new unseen examples (He et al., 2016b; Levine et al., 2020).

The universal approximation theorem (Lu et al., 2019b) indeed states that even simple multilayer perceptrons (MLPs) can represent a broad set of functions, while different inductive biases, as those implemented in Recurrent Neural Networks (RNNs) or Convolutional Neural Networks (CNNs) (Goodfellow et al., 2016), can improve the learning efficiency and enhance the capacity of a given network to model different forms of information: sequential data, common to language, speech and time-series, for the former, or spatially-invariant information, as in images or videos, for the latter.

Transformer Networks & Attention

Meanwhile, transformer networks (Vaswani et al., 2017), introduced more recently, demonstrate the importance of capturing long-range dependencies and pairwise or higher-order interactions between elements. They build on the self-attention mechanism (Vaswani et al., 2017; Bahdanau et al., 2014) that enables shorter computation paths and provides direct means to compare elements far-across the input data (such as a pronoun and its antecedent in a sentence, or two sentences that refer to the same topic).

From another perspective, the multiplicative interaction embodied in both attention as well as gating structures (as in LSTMs (Hochreiter and Schmidhuber, 1997) or Mixture-of-Experts (Shazeer et al., 2017)) offers a more flexible alternative to the rigid fixed-weight computation of MLPs and CNNs, dynamically adapting the computation to the input at hand. This proves especially useful for language modeling, where, for instance, given a sentence like “She ate the ice-cream with the X", while a feed-forward network would always process it in the very same manner, an attention-based model could adapt its computation to the input — updating the contextual representation of the word “ate” if the prepositional phrase (PP) attachment X is “spoon”, or instead link it to the “ice-cream" if X refers e.g., to “strawberries" (Zavrel et al., 1997).

General-Purpose Computation

A final notable advantage of attention over prior architectures stems from its stronger generality, where it is not strongly tied to a particular task or domain, as is the case for the local receptive field of convolution or the sequential assumption of recurrent networks, both reflecting inherent properties specific to the vision and language modalities respectively. We hypothesize that the general-purpose nature of attention and transformers contributes to their broad applicability for a wide range of research problems and applications (Liu et al., 2019; Dosovitskiy et al., 2020; Hudson and Zitnick, 2021).

This contrast captures a more general trade-off between task-specialization and expressivity: models with stronger structural priors can leverage them to improve sample efficiency on the particular tasks that benefit from these assumptions; while conversely, models that integrate weaker inductive biases learn more slowly, but can in turn scale to higher volumes of data and adapt to a diverse set of domains, since they do not rely on restrictive or task-specific suppositions. As both data and compute turn more accessible, we observe that the exploration of models with a minimal set of inductive biases that can “let the data speak for itself" seems to serve as a more promising approach for future research in the field.

Challenges & Future Directions

Notwithstanding the stellar progress and accomplishments of neural networks in general, and foundation models in particular, in terms of expressivity, notable challenges still remain. Leading approaches (Choromanski et al., 2020; Dosovitskiy et al., 2020) keep struggling with modeling of extremely long-range dependencies, such as those occurring in books, movies, or even DNA sequences, which may be attributed to the quadratic computation of contemporary transformer-based approaches (Wang et al., 2020c; Lin et al., 2021).

This challenge essentially reflects the trade-off between efficiency and expressivity: where explicit modeling of long-distance interactions through short and direct computation paths improves expressivity on the one hand, but comes at the expense of scalability due to computation entailed by the increased connectivity on the other (Child et al., 2019; Kitaev et al., 2020; Choromanski et al., 2020). Models such as the GANformer (Hudson and Zitnick, 2021) and the Perceiver (Jaegle et al., 2021b; Jaegle et al., 2021a) explore ways to balance these two properties and propose transformers with linear complexity that rely on bipartite or bottleneck attention, so to improve computational efficiency while maintaining high-expressivity. We believe that identifying an effective equilibrium between these two objectives offers an interesting avenue for future research.

Another important research direction relates to the expansion of foundation models, which, so far, have mainly focused on the language domain (Peters et al., 2018; Devlin et al., 2019; Brown et al., 2020), to different modalities, such as the structural (Scarselli et al., 2008; Veličković et al., 2017) and perceptual (Tolstikhin et al., 2021; Jaegle et al., 2021b; Tan and Le, 2021), each involving a unique set of associated challenges. Likewise, we believe that exploring architectures for reasoning (§2.4: reasoning), which demands iterative computation chains and interaction with symbolic information, constitutes a valuable goal for future foundation models research.