On the Opportunities and Risks of Foundation Models

Once a model completes gathering the information after training, there are multiple ways to retrieve particular facts or memories necessary for downstream applications and tasks. Some employ explicit prompting techniques that query the model’s knowledge through input sequences (Petroni et al., 2019; Kassner et al., 2021; Jiang et al., 2020) while other approaches involve implicit recollection and reshaping of the prior knowledge through an adaption phase (Bosselut et al., 2019; Hwang et al., 2021). A third category of methods goes a step further and combines neural-based computation with symbolic aggregation and retrieval of information from either unstructured textual repositories (Karpukhin et al., 2020; Lewis et al., 2020b; Khattab et al., 2020) or even structured resources such as knowledge graphs (Zhang et al., 2019a; Peters et al., 2019; Liu et al., 2020e; Verga et al., 2020; Yasunaga et al., 2021).

However, there is trade-off between the strong memorization skills offered by retrieval mechanisms on the one hand and the richer representations learned when there is an information bottleneck on the other. Indeed, over-reliance on retrieval reduces the opportunities to learn how to represent information in compact and abstract manners, distill key insights and concepts out of the vast amounts of input information the model is exposed too, and, basically, separate the wheat from the chaff. For instance, the in-context learning abilities of GPT-3 possibly emerge as a by-product of enforcing the network to represent the input sequential data through its bounded memory architecture (Brown et al., 2020). Overall, While they certainly have some merits (Guu et al., 2020), models that rely on external retrieval mechanisms may not learn to generalize as effectively as bounded, compact and abstract representations.

Knowledge Manipulation

Finally, when considering large-scale learning over long durations, it is crucial to note the dynamic nature of knowledge, where facts’ correctness and validity can change over time as the world keeps evolving — and what was true or relevant yesterday may not be so tomorrow. It is therefore crucial for a model to represent its knowledge in a manner that supports efficient update or manipulation of facts as part of its lifelong learning.

Compositionality

Compositionality can be defined as the principle according to which the meaning of the whole is derived from the meaning of its constituent parts, and the rules applied to combine them (Janssen and Partee, 1997; Bottou, 2014). It is a crucial ingredient of human intelligence (Lake et al., 2017), underlying our capabilities to plan, reason and learn readily and efficiently from a handful of examples. Compositionality may hold the key to achieve out-of-distribution — or specifically — combinatorial generalization. Drawing on classic ideas from symbolic AI, it encourages and enhances desirable properties within neural networks, such as interpretability, controllability and data-efficiency (Lake et al., 2017), and can take different forms, characterizing variety of elements:

Model

Compositionality can be reflected at the model level, in terms of its architectural properties, structure, and degree of modularity — which can increase training and inference efficiency of large neural models (Shazeer et al., 2017). It also links to themes of interpretability and multimodality, as it relates to the interfaces between the different modules the model is composed of, what modes of interactions they employ, and how transparent they are.

Computation

Models such as Module Networks (Andreas et al., 2016) and Mixture-of-Experts (Shazeer et al., 2017) go further along this direction, exhibiting not only structural modularity, but also compositional computation, supported by the specialization of sub-networks to different operations, in a manner that adapts and tailors the model behavior to the input at hand. While some methods rely on concatenation of hand-engineered modules (Andreas et al., 2016), alternative approaches enable the network specialization to naturally emerge through learning (Shazeer et al., 2017). Other models, such as MAC (Hudson and Manning, 2018) and Dynamic Memory Networks (Xiong et al., 2016) perform an explicit iterative computation, where a given task is decomposed into multiple reasoning steps, performed one by one, manifesting sequential progression from a set of initial facts to novel inferences and conclusions.

Training & Data

Not only can the model or its computation be compositional, but so can be the data or training processes too (Andreas, 2020). Instead of training one model over a complete dataset, one could split, or decompose it into subsets, train different models on each one independently, and ultimately recombine them at test time through various ensemble techniques (Dietterich, 2000). Such approaches could have far-reaching implications on the training and deployment procedures of foundation models, in both practical and even societal regards.

Representation

We have discussed compositionality of different elements, such as the model, the computation, the training schemes or the data. But most notably, the learned representation itself, which emerges over the course of the model training and adaptation, can also be compositional (Andreas, 2019). Indeed, a promising manner to represent knowledge is through structured, potentially graph-based, object-oriented representations (Zhang et al., 2019a; Wang et al., 2021a), that center around identifying entities and event nodes and forming connections, analogies and relation edges among them. It reflects a natural way to organize information about the world, where inputs from different modalities can be channeled and aggregated around semantic multi-faceted concepts. Such representations could support multi-hop reasoning and inference (Washington et al., 1995; Sun et al., 2020b; Yu et al., 2020c), and potentially also enable stronger out-of-distribution generalization through recombination.

However, compositionality can also hinder the expressivity of the representation, and impede its capacity to account for idiosyncrasies, exceptions, and contextual correlations (Misra et al., 2017a). In other words, the whole can sometimes be greater than the sum of its parts, where for instance, red wine is not the same as red onion. But while many approaches that have dominated over the last decade tend to focus mostly on one end of the spectrum, and learn monolithic distributed representations, we believe that exploring manners to reach a better balance between contextuality and compositionality is a promising avenue for future research.

Summary

We have introduced five properties that we believe are essential for the next generation of foundation models, in order to effectively distill the large amounts of information around us so to successfully address downstream tasks: expressivity — to flexibly capture and assimilate real-world information, scalability — to adeptly handle high volumes of high-dimensional data, multimodality — to consume, process and potentially produce content from different sources and domains, memory capacity — to effectively store and retrieve the acquired knowledge, and finally, compositionality, to foster successful generalization to novel tasks, settings and environments. We believe that the realization of the full potential of foundation models, as is envisioned and discussed in detail throughout this report, will rely on research of new architectural and modeling advances to fulfill these desiderata.