On the Opportunities and Risks of Foundation Models

In this section, we describe how US law may influence, constrain, or foster the creation and use of foundation models. We note that the legal landscape surrounding algorithmic tools remains uncertain. We highlight issues pertaining to (1) model training, (2) liability for model predictions, and (3) protections for model outputs.

Though understanding how the law affects foundation models is crucial, it is important to recognize that the law cannot be the only lens through which we evaluate the construction, maintenance, and use of foundation models. Ethical frameworks are necessary to understand where legally permissible applications of foundation models may still be ill-advised for the harms they inflict and are discussed in more depth in §5.6: ethics and §5.1: fairness. Studying the potential for misuse and possible security concerns (see §5.2: misuse and §4.7: security) is critical for preventing harmful outcomes ex ante, as opposed to the ex post treatment that legal mechanisms often provide.

Training

Training foundation models will require accumulating vast amounts of multi-modal data, raising questions around data collection and data use.

First, the ability for model creators to grow datasets via web scraping will be governed by the manner in which courts will interpret terms of service provisions and, notably, the U.S. Computer Fraud and Abuse Act (CFAA), which criminalizes accessing a server “without authorization” (Wajert and Rottman, 2019). Courts are in conflict on these questions, and recent cases have sought to clarify the circumstances under which web scraping may be barred. The restrictiveness of data access would fundamentally affect the diversity of data practitioners can use to train foundation models (Levendowski, 2018).

Second, much of the data contained in training sets will be copyrighted and potentially protected by intellectual property law. However, copyright law recognizes exceptions when individuals may be permitted to use copyrighted material. Some scholars believe that the legal permissibility of training datasets will largely rest on whether courts interpret the process of model training as “transformative” under fair use doctrine (Lemley and Casey, 2020). Though the question of what qualifies as transformative is highly context dependent, the general rule is that transformative uses are those “that add something new, with a further purpose or different character, and do not substitute for the original use of the work" (Office, 2021). Already, the recently released Github Copilot tool is bringing these arguments to the fore (Gershgorn, 2021).

Finally, some training datasets may run afoul of privacy laws. Illinois, for instance, enables individuals to sue for improper collection or use of biometric data (e.g., retina or iris scans, fingerprints, voiceprints, or scans of hand or face geometry). Foreign privacy laws like the E.U.’s General Data Protection Regulation (GDPR) — which will affect American model creators if datasets contain information from E.U. citizens — would require data subjects to be informed about the purpose of data collection. Further issues could arise for laws like the California Consumer Protection Privacy Act (CCPA), which provide individuals with a “right to be forgotten,” raising questions as to whether model creators will need to “remove” training data from models (Villaronga et al., 2018; Ginart et al., 2019).

Output liability

Though foundation models themselves are task agnostic, fine-tuned models — or the representations learned by foundation models themselves — may be used for traditional prediction tasks. Where these tasks form components of larger decision-making systems, foundation models will thus influence actions, decisions, or policies. When these result in harm, model creators — and the individuals operating them — may be legally responsible.

Embedding foundation models in physical systems (e.g., self-driving cars, electric grid management, medical diagnostics, etc.) may result in physical harm to individuals. Here, courts will likely resolve questions of liability under tort doctrine (Lemley and Casey, 2019; Selbst, 2020). Key open questions include the interplay between the liability of users, foundation model providers, and application developers, as well as the standards courts will use to assess the risk profile of foundation models. Deployments in particularly sensitive domains (e.g., medicine) will require regulatory approval, and the development of standardized processes to assess safety (Wu et al., 2021g).

Fine-tuned foundation models that classify individuals in ways that correlate with protected attributes (e.g., race, gender) may face challenges under civil rights laws. Scholars have noted that claims for disparate treatment resulting from foundation models may be brought in the context of hiring, housing, or credit lending (Gillis and Spiess, 2019; Scherer et al., 2019). Exactly how courts will adjudicate these issues is far from clear. Scholars have noted for instance, that the courts’ traditional views on “discrimination” would actually prevent machine learning practitioners from implementing many algorithmic fairness techniques (Xiang, 2021; Ho and Xiang, 2020).

U.S. law recognizes special privileges and limits on governmental entities. Thus, the use of foundation models by governmental entities --- at a local, state or federal level --- will implicate special considerations, in addition to equal protection claims. The use of models for risk assessment --- or in other settings which result in a deprivation of life, liberty, or property --- will invite procedural due process claims. When models are used by administrative agencies (e.g., the Environmental Protection Agency) for instance, plaintiffs may allege that such use violates basic standards of due process, reasonableness / non-arbitrariness, and transparency.

Legal protections for outputs

Model outputs — and by extension the model creators responsible for the models — may also be afforded certain legal protections. First, content produced by generative models may implicate free speech issues. The extent to which courts will find First Amendment protections for machine generated content is unclear. Scholars have discussed a number of open questions, including whether “AI speech” is protected (Massaro et al., 2016) or if model outputs are in effect the human programmer’s speech (Kajbaf, 2019). Others have noted the possibility of disclosure requirements (akin to safety disclosures for pharmaceutical drugs or other substances), also implicating speech doctrine, under which models would be forced to share with listeners that their content is machine generated (Lamo and Calo, 2019). These issues could have wide ranging consequences, affecting whether individuals can use foundation models to mass produce speech, or whether model creators could be held liable for content generated by foundation models.

Second, there is uncertainty regarding who may assert ownership over model outputs. Existing copyright law does not recognize computer programs as authors, and hence, does not afford copyright protection to “work” created by computer programs (Grimmelmann, 2015). As a result, scholars have advocated for a variety of approaches. Some have argued that, depending on the circumstances, both the human creator of a program and its human user may have viable claims to being the “author” of the program’s output (Ginsburg and Budiardjo, 2019).

As models are increasingly used in the process of “creation” — from artistic endeavors to more mundane settings like news filings — disputes over the ownership of machine generated content will become more commonplace.

While our analysis above only skims the surface of the legal issues implicated by foundation models, the resolution of these questions will be critical to the construction, use, and deployment of foundation models, or, to borrow Larry Lessig’s phrase, how “code is law” (Lessig, 2000).

Economics

Foundation models have the potential to substantially improve overall living standards by increasing productivity and innovation. These models can be deployed to substitute for human labor, augment humans, or help in the discovery of new tasks and opportunities, which can lead to increased concentration of ownership and power, or more decentralization. On a broader level, the result can be either increased inequality due to potential centralization (§5.1: fairness, §5.6: ethics), or more broadly shared prosperity due to the easier adaptation of foundation models for a wide range of applications (§1: introduction). The ultimate outcomes on all these dimensions are not dictated solely by technology or economics, but by the choices and actions of technologists, policymakers, managers, workers, and other members of society.

Foundation models can be thought of as what economists refer to as a general-purpose technology (Bresnahan and Trajtenberg, 1995). General-purpose technologies refer to technologies like the steam engine and electricity, which drive waves of transformation and productivity growth due to their pervasiveness, improvement over time, and ability to spawn complementary innovations (a host of products and services that revolve around one core product). While foundation models may not be pervasive at the moment, they seem poised to be the basis of widespread technological innovations, and have the key hallmarks of a general-purpose technology. As a result, these models are likely to be economically important. In considering the impact of foundation models on the economy, we will focus on three broad areas of impact: productivity, wage inequality, and ownership.

Productivity and Innovation

Foundation models are likely to substantially increase both productivity and innovation. Productivity growth is one of the main contributing factors to boosting living standards, as it increases the wealth of nations and addresses a host of challenges from poverty and healthcare to the environment and education.

Productivity is defined as output per unit input. One way to boost productivity is to reduce the denominator; for instance, enabling a company’s advertisements to be written with fewer copywriters or fewer labor hours per copywriter lowers the number of units of input. Productivity can also be boosted by increasing the numerator, for instance by enabling a software developer to write more code in a given time. If the growth in the numerator is great enough, this can lead to more people developing software, not fewer (Autor, 2015). In many tasks, we have already observed machine learning systems increasing productivity. For instance, an autocomplete system for clinical documentation reduces keystroke burden of clinical concepts by 67% (Gopinath et al., 2020). Likewise, the potential for foundation models to affect productivity spans almost every industry and many occupations. Considering language alone, an analysis of U.S. occupations using the US Department of Labor’s O*NET database shows that many occupations involve the types of language-related work that could be affected by foundation models. Approximately 13% of occupations have a primary task that is related to writing, and the total wage bill of these occupations (annual salary multiplied by the number of individuals employed in the occupation) is over 675 billion dollars. However, the potential impact of foundation models extends beyond language. They will also have effects on diagnostic imaging in medicine, graphic design, music, and many other tasks where people are creating something that is similar to something else that already exists (Winkler et al., 2019; Ramesh et al., 2021).

Perhaps the most profound, if still speculative, effect of foundation models is their potential to enhance creativity and boost the rate of innovation itself. For instance, DALL·E (Ramesh et al., 2021) could transform the market for illustrations much as inexpensive cameras revolutionized photography. If these models enable humans to develop new ways to write new songs and novels (§2.5: interaction), discover variants of drug molecules (§3.1: healthcare), extend patents (§3.2: law), build innovative software applications, or develop new business processes, then not only the level of productivity, but the rate of growth of productivity would be increased. In this way, foundation models have some of the characteristics of the ideas or blueprints in Paul Romer’s growth models (Romer, 1990), or even meta-ideas (ideas about ideas) which, unlike most other goods, are non-rival, thus speeding growth.

It is worth noting that changes in productivity are not always visible in the official statistics, because many aspects of input and output are difficult to measure (Brynjolfsson and Collis, 2019). As a result, the benefits and costs of foundation models will not be fully captured by traditional productivity metrics, nor by related metrics like gross domestic product (GDP) or price levels (the average of current prices across the entire spectrum of goods and services). This is especially true for general purpose technologies historically, since they are catalysts for a cascade of secondary innovations that often transform the set of goods and services in the economy, and even the nature of production and innovation over a period of years or even decades.

Wage inequality

Even if foundation models increase average productivity or income, there is no economic law that guarantees everyone will benefit. In part, this is because not all tasks will be affected to the same extent. More importantly, the effects of foundation models on the demand for labor (and thus employment and wages) can be either positive or negative, regardless of productivity growth (Brynjolfsson and McAfee, 2011; Brynjolfsson and Mitchell, 2017). When a technology substitutes for human labor in completing tasks, it tends to reduce demand for the workers doing those tasks. This depresses employment and wages. However, when a technology complements labor, or facilitates the creation of new opportunities or tasks, it tends to increase labor demand (Acemoglu and Restrepo, 2019). Employment can (and often does) go up, even as productivity increases. For instance, the invention of the airplane created the demand for an entirely new occupation, the airline pilot. In turn, the development of jet engines was complementary to human pilots, further increasing demand for them. Similarly, the effects of foundation models on employment, wages, and income inequality will differ depending on how they are used.

While the industrial revolution mainly transformed physical work, foundation models are likely to transform tasks involving cognitive work, like content creation and communication. In general, since foundation models are intermediary assets that often possess strong generative capabilities, we envision that they will be able to augment humans in many creative settings, rather than replace humans as there are still significant limitations in using these models stand-alone for open-ended generative tasks (See et al., 2019). As we describe in §2.5: interaction, foundation models may also power systems that users can leverage to co-construct novel forms of art or more efficiently prototype new applications. Fluid human-machine and human-in-the-loop interaction will require advances in interface design (§2.5: interaction) as well as fundamental improvements in the interpretability (§4.11: interpretability) and robustness (§4.8: robustness) of these models, so that humans can understand model behavior and expect models to perform well in diverse contexts.

Centralization

Another key determinant of foundation models’ economic impact is who owns data and models. In particular, pushing the frontier of foundation models has thus far primarily been the purview of large corporate entities. As a result, the ownership of data and models are often highly centralized, leading to market concentration (§5.6: ethics). In turn, this can lead to significant centralization of decision rights and power, reducing income and opportunities for those who don’t have ownership. This centralization of power can lead to an equilibrium where fewer people have social and economic mobility and opportunity, a situation that Brynjolfsson (2022) calls "The Turing Trap". To counterbalance this centralization, there have been grassroots efforts to open source AI research such as Masakhane, EleutherAI, and HuggingFace, or build foundation models through distributed training. However, it likely that the gap between the private models that industry can train and the ones that are open to the community will remain large due to foundation models’ dependence on massive amount of data and computational resources (§5.3: environment).

Other considerations

This short chapter is not meant to be comprehensive of all the economic effects of foundation models. In addition to affecting productivity, wage inequality, and ownership, foundation models may also have significant effects on job quality and job satisfaction. For instance, they may increase job satisfaction by automating repetitive, uninteresting parts of work, or decrease satisfaction by increasing the pace of work, thereby inducing more frequent burnout. As discussed in §5.1: fairness and §5.6: ethics, they can also amplify and perpetuate bias, often in unexpected ways, or be used as a tool for reducing it. Foundation models can facilitate global trade and remote work, just as earlier uses of machine translation systems had significant effects in these areas (e.g. Brynjolfsson et al., 2019). There may also be significant environmental effects (§5.3: environment), as well as unexpected and unanticipated effects on the rate and direction of occupational change and business transformation in an economy. More broadly, given the emergent capabilities of foundation models, we should expect new unknown unknowns to arise that are difficult to predict, and which may have substantial follow-on effects.

In summary, foundation models are poised to be an important general-purpose technology of our era. They have potential to increase living standards substantially, but also pose risks of increasing inequality and concentrating power. The economic implications of these technologies are not predetermined, but rather depend on how technologists, policymakers, managers, workers, and other stakeholders answer challenges such as:

• • How can we harness the potential of foundation models to boost productivity?

• • Can we develop models that enhance creativity and boost the rate of innovation?

• • Will the benefits and control rights be limited to a few or widely shared?

Understanding the economic potential of these systems is the first step to guiding them in directions that match our values.