On the Opportunities and Risks of Foundation Models

From family court to criminal justice and from environmental policy to corporate transactions, the reach of the law is vast. In the United States, there are over 1.3M lawyers (American Bar Association, 2021) and annual revenues for legal services exceed $300B (MarketLine, 2021). Yet “access to justice” remains far out of reach for most. Legal services can be prohibitively expensive. Roughly 86% of low-income individuals with civil legal problems in the United States, for instance, report receiving inadequate or no legal help (Legal Services Corporation, 2017). Even when counsel is appointed, lawyers might be strained by increasingly large caseloads. Studies have shown that public defenders, for example, are often overworked and underfunded (Lefstein and Spagenberg, 2009; Schumm, 2012; American Bar Association, 2004). The U.S. Department of Justice reported that in 2007, 73% of county-based public defender offices exceeded the maximum recommended limit of cases received per attorney and 15 of 19 reporting state public defender programs exceeded the maximum recommended limit of felony or misdemeanor cases per attorney (Farole and Langston, 2010; Langston and Farole, 2010). Even in a country with one of the highest per capita rates of attorneys, justice can appear out of reach. U.S. President Jimmy Carter once opined, “Ninety percent of our lawyers serve ten percent of our people. We are overlawyered and underrepresented” (Carter, 1978). According to a leading voice in access to justice, technology may provide a path forward (Rhode, 2014), a view echoed by many others (Cabral et al., 2012).

What role might foundation models play in the law? A major promise is that foundation models can improve access to justice and government services by leveling procedural and financial barriers to legal services. The challenges posed by legal applications can, in turn, motivate basic research questions for foundation models. Many legal applications pose unique challenges to computational solutions. Legal language is specialized and legal outcomes often rely on the application of ambiguous and unclear standards to varied and previously unseen fact patterns. At the same time, due to its high costs, labeled training data is scarce. Depending on the specific task, these idiosyncrasies can pose insurmountable obstacles to the successful deployment of traditional models. In contrast, their flexibility and capability to learn from few examples suggest that foundation models could be uniquely positioned to address the aforementioned challenges.

Throughout this section, foundation models may take as context many modalities as evidence: audio during trial proceedings, video and images during discovery, and text in conducting legal research. Yet, the majority of legal tasks in which reliance on foundation models will be beneficial involve text-based inputs and outputs. As such, we mainly focus on text-based domains while only briefly discussing others. To ground the discussion, Figure 13 describes the stages of a civil lawsuit in the United States and where foundation models might come into play in this process. Figure 14 shows the logic flow required to generate just part of one paragraph of a legal brief, which might serve as a concrete example of a task that foundation models might one day be used for.

An Important Consideration. Before proceeding, we note that the ethical, legal, and fairness considerations expanded on in §5.6: ethics, §5.4: legality, and §5.1: fairness are particularly important to examine before using foundation models in an applied legal or government context, as these applications often have important, real-world consequences to those affected (Surden, 2020). Foundation models must also be thoroughly scrutinized before deployment, as discussed in §4.4: evaluation. For example, the legal system places particular emphasis on — and may even mandate — transparency, accountability, and explainability. Consequently, it is questionable whether current models are positioned to solve many of the most pressing, legal problems. Nonetheless, the need to expand and improve access to legal and government services provides a worthy goal for foundation models.

Opportunities in law

Legal applications can range from the use of machine learning in government contexts (Engstrom et al., 2020; Coglianese and Ben Dor, 2020; Re and Solow-Niederman, 2019) to aiding lawyers in their provision of legal services (Zheng et al., 2021; Huang et al., 2021b; Ostendorff et al., 2021; Vold and Conrad, 2021). We note that prior work has also surveyed machine learning-assisted legal tasks in text-based domains (Zhong et al., 2020; Chalkidis et al., 2020), although it has been noted that recent legal AI research has focused on geographic regions outside of the U.S. (Zheng et al., 2021). While many of the topics we discuss here may be applicable to different legal systems, due to the expertise of our team we focus primarily on the U.S. In particular, we concentrate on three broad categories of legal applications that may benefit from foundation models in the U.S. legal system: private law or civil justice (claims between private individuals, arising out of, for instance, contracts, property or torts), criminal law (i.e., the prosecution of individuals for criminal behavior), and (non-criminal) public law (e.g., the regulation of private behavior by government agencies).

Civil law

In U.S. civil proceedings, parties must typically find and pay attorneys to be represented. As a result, many individuals, especially those with low income, struggle to secure adequate legal representation (Rhode, 2004). Foundation models have the potential to improve access to justice by reducing the cost, improving the quality, and extending the reach of legal services. In Figure 13, we describe the process by which a civil lawsuit is filed in a U.S. court and where foundation models may play a role in aiding both attorneys and judges.

Even before an attorney is involved in the legal process, clients may benefit from the deployment of foundation models. Recent work has used machine learning models to identify the relevant legal issues contained in a plain-language description of facts presented by a client. Tools like these can help provide a recommendation for the type of legal action needed to address the issue at hand or to recommend a specialized attorney. A number of other similar efforts have sought to increase access to justice by providing information tailored to a client’s particular needs (Cabral et al., 2012; Brescia et al., 2014; Queudot et al., 2020; Westermann et al., 2019).

Once a client speaks with an attorney, prior to civil litigation, the attorney may seek to avoid a costly trial. At this stage, they can rely on foundation models to evaluate contracts, review terms of service, find relevant patents, and conduct other pre-litigation processes in order to ensure that their clients are at an advantage (Betts and Jaep, 2017; Elwany et al., 2019; Lippi et al., 2019; Lee and Hsiang, 2019; Hendrycks et al., 2021c; Hegel et al., 2021). Notably, recent work has both described the challenges and benefits of using foundation models for contract review (Leivaditi et al., 2020; Hegel et al., 2021; Hendrycks et al., 2021c). In addition to reviewing and drafting legal documents, client interactions and documents can be translated to reduce costs and barriers to the provision of legal services (Cuéllar, 2019). But translation of legal documents requires precision and an understanding of highly technical language, which makes collecting training data costly. Additionally, translating client statements or trial proceedings often requires an understanding of local dialects and language. This, too, makes it difficult to collect enough ground truth translation data to train on. As a result, traditional supervised methods rarely achieve the level of accuracy required in the legal domain (Vieira et al., 2020). Foundation models may improve performance in this area over fully supervised mechanisms by adapting quickly in these low-resource contexts.

During litigation, foundation models can help lawyers to conduct legal research, draft legal language, or assess how judges evaluate their claims (Zheng et al., 2021; Huang et al., 2021b; Ostendorff et al., 2021; Vold and Conrad, 2021; Chalkidis et al., 2020; Chalkidis et al., 2019). This could potentially reduce the costs of and improve legal services. For example, recent work has utilized pretrained models for the recommendation of relevant citations and holding statements when writing legal texts (Zheng et al., 2021; Huang et al., 2021b; Ostendorff et al., 2021). Other work uses pretrained models for improved legal question answering to power commonly used legal search engines and help lawyers conduct legal research (Vold and Conrad, 2021). A wide variety of work has also examined automated contract drafting and review, a task that could similarly benefit from foundation models (Hendrycks et al., 2021c; Betts and Jaep, 2017). Perhaps most compelling, foundation models may help assist lawyers generate legal briefs (written arguments). The models might find novel arguments or identify problems in attorney-written portions of the brief. For example, Tippett et al. (2021) predict the outcome of a legal proceeding based on features extracted from the filed briefs. Foundation models can be leveraged to use raw language as inputs rather than extracted features. This might provide attorneys with more informative recommendations as to how their brief could be improved to ensure a favorable outcome.

After opening and reply briefs are filed, parties then begin the discovery process, which has already used simple machine learning models for the better part of a decade (Grossman and Cormack, 2010). Attorneys use these systems to label whether a document should be produced to the opposing party. The documents are multi-modal in nature, often containing video, images, audio, and text. Current systems are costly because they used supervised learning and active learning to label the documents as responsive (Grossman and Cormack, 2010; Oard et al., 2018; Yang et al., 2021). Instead, few-shot or zero-shot document retrieval capabilities that might be possible with foundation models would help ease concerns about the large costs of the current process. To avoid the possibilities of gamesmanship in the discovery process, Cui (2018) has proposed a zero-shot (or few-shot) adaptation process that can only be operationalized through the use of foundation models.

After discovery, once the trial begins, foundation models could help parties prepare for trial by predicting what the judge might focus on during questioning (Dickinson, 2018), adapting to the current context from judges’ prior published opinions. In the courtroom, foundation models might be used to examine audio and video of courtroom proceedings to determine if outcomes were biased against the defendant because of their race or dialect.

Once the trial concludes, foundation models could help judges and law clerks to properly evaluate legal claims from both parties using similar technologies, or the use of contextual embeddings from foundation models might assist in statutory interpretation (Nyarko and Sanga, 2020; Choi, 2020). Recent work (without reliance on foundation models or NLP) has examined whether an appeals decision can be predicted from a set of extracted features, like citation counts and the appearance of key words (Katz et al., 2017; Boniol et al., 2020). It is possible that such models could be improved using foundation models and applied to help judges draft decisions by flagging obvious mistakes in their opinion, as has been discussed in the context of adjudicative agencies (Engstrom et al., 2020; Ray and Lubbers, 2014). They can also be used to identify racial biases in legal opinions and help judges revise their opinions accordingly (Rice et al., 2019).

Criminal law

One particularly contentious area has been the use of risk scores in government settings, particularly in criminal law. Some may want to use language-based foundation models to aid in making charging decisions or parole decisions based on a given text-based narrative of the events. Careful consideration must be taken before using foundation models for risk scoring due to the potential for biases, especially when language data is included (Bender et al., 2021; Berk et al., 2021; Laufer, 2020). But foundation models may play a role in many other dimensions of criminal justice. The same tools as in civil litigation, above, can also be used by prosecutors and defense attorneys. This can help appointed attorneys perform their job more efficiently and reduce unnecessary overhead. As a result, they may be able to balance already heavy caseloads more effectively. For example, public defenders are often viewed as being overworked and underfunded, which would lead to avoidable procedural errors. Foundation models can help reduce some of these resource constraints by identifying errors and automating simple tasks. However, they are not a solution on their own.

In other areas, foundation models can act as an oversight mechanism to reduce structural inequities. Pretrained models have been used for processing parole hearing transcripts to find instances of anomalous outcomes (Bell et al., 2021). Recent work has also removed linguistic cues for a suspect’s race in police reports to promote race-blind charging decisions and avoid racially biased prosecutions (Chohlas-Wood et al., 2020). Other work has helped identify disrespectful police communications (Voigt et al., 2017). In these contexts, it is very costly to label data since annotators must be given access to sensitive data and appropriate background checks are often required. To reduce these costs, foundation models can be used to pretrain and adapt quickly to downstream tasks where labels are scarce.

Public law

Government agencies regulate vast parts of society, and foundation models have wide potential applicability across public law. This includes: analyzing public comments in the notice-and-comment process, assisting patent examination, retrieving relevant documents in response to Freedom of Information Act requests, aiding in mass adjudication, among many others. Recent work has surveyed these government applications in a variety of contexts and we refer the reader to the relevant sources for in-depth discussion (Engstrom et al., 2020; Coglianese and Ben Dor, 2020). In many of these applications, foundation models can improve the quality, efficiency, utility, and accessibility of government services: labels are scarce, resources are constrained, and contexts are constantly shifting. As such, the adaptability and flexibility of foundation models are often required to improve efficiency and performance. To give an illustrative example of just one such application, existing work has leveraged NLP for facilitative moderation in public comment forums. In this use case, predictive models help lay-users improve arguments and identify misstatements in their comments. Such a system has already been deployed in the U.S. Department of Transportation rulemaking process (Park et al., 2012), although it can likely be improved through the linguistic reasoning capabilities of foundation models. But government agencies must comply with constitutional, statutory, and administrative obligations (see §5.4: legality), so additional care is needed in these settings.

How can foundation models uniquely help?

The above examples of legal applications are unique in several ways. First, the cost of annotating data is very high. Often, the expertise to create high-quality labels can only be found in attorneys, who may charge hundreds of dollars per hour. Even after labels are obtained, certain data may be sensitive and cannot be pooled together to training a large language model. Given recent progress in few-shot learning (Brown et al., 2020), foundation models are among the most promising paths for learning models with limited annotations.

Second, legal decision-making requires context at various scales: knowledge of all historical decisions and standards, knowledge of the case law that remains relevant in the present, and knowledge of the nuances of the individual case at hand. Foundation models are uniquely poised to have the potential to learn shared representations of historical and legal contexts, as well as have the linguistic power and precision for modeling an individual case.

What are foundation models lacking that requires more research?

To illustrate the deficiencies current foundation models need to overcome in order to be realistically deployed, we consider as an example the automatic creation of a legal brief to submit to a court.

A brief lays out the arguments to a judge before a hearing. Once a party has filed an opening brief, the opposing party files a response. The judge then evaluates the briefs and asks questions of both parties at a hearing before making a decision. Figure 14 visualizes the structure of such a legal brief with some of its characteristic features.

An automated brief generation mechanism might take as context relevant documents and facts of a case (as specified by an attorney) as well as a rough sketch of the desired outcome. It would then generate a legal brief with complex legal arguments to submit to the court.

Long Documents and Narratives. To achieve this goal, the model must be able to read long contexts and produce long narratives. Legal documents tend to be far longer than documents in any other context. The average U.S. Supreme Court opinion contains around 4,700 words, a brief on the merits to the Supreme Court can have as many as 15,000 words, a law review article often contains 20,000 to 30,000 words, parole transcripts can be hundreds of pages long (Bell et al., 2021), and trial records can be even longer. Current foundation models have struggled with such long contexts and outputs (see §4.1: modeling for more discussion).

Retrieval, Concept Drift, Argument Formation, and Logical Reasoning. In addition to reading case-specific documents, the foundation model must retrieve the relevant case law and understand which case law is still valid and which has been overruled, taking into account potential concept drift since it was trained. More work in editing grounded information in foundation models will be required as case law evolves (De Cao et al., 2021). Using retrieved legal standards, the foundation model must then understand how to weave them into a persuasive argument. Emerging research has studied ways of using foundation models to measure, detect, and generate persuasive texts (Duerr and Gloor, 2021; Li et al., 2020a; Longpre et al., 2019), which may provide stepping stones towards this goal.

But legal briefing also requires the ability to identify a relevant legal rule and determine how it applies to a new situation. For example, Holzenberger et al. (2020) provide a dataset and methodology for evaluating this ability in current models based on the tax code. Given a set of tax code rules and a description of someone’s situation, the model must determine how the rules apply to the individual in order to determine what their tax liability is. Holzenberger et al. (2020) found that domain experts can express the relevant tax law as a prolog solver to determine an individual’s tax liability with perfect accuracy. However, they found that simply using a BERT model, even when pretrained on in-domain legal text, was not sufficient to allow the model to deduce and apply the relevant rules, a process also referred to as “legal reasoning.”

These challenges can further be illustrated through a real exchange with GPT-3, demonstrating that current models are unable to perform even comparatively simple tasks involving legal reasoning.

Precision. The foundation model also cannot create fake facts in the process, a problem that has come up in current models (Gretz et al., 2020; Zellers et al., 2019b). Specificity and truthfulness are of heightened important in legal contexts, where imprecise statements can have drastic, unanticipated consequences, and false statements can lead to sanctions against attorneys.

Few-shot Learning. Some challenges facing foundation models beyond those described in the above example include few-shot learning, which is still in its infancy (Perez et al., 2021). Research on few-shot learning techniques and increasing access to legal corpora can work in tandem. Because foundation models need to be precise, not only in factual truth, as discussed above, but also in technical legal language, it remains unclear to what extent information obtained from one corpus can be utilized in another corpus of a different domain. Few-shot learning thus remains important.

Adaptation. Some gains have been observed from domain-adaptive pretraining on unlabeled legal corpora. These gains appear to be most pronounced when the pretraining corpus is highly relevant to the downstream task and labeled training data is limited (a setting which is common in the law) (Zheng et al., 2021). It has not yet been comprehensively studied whether this extends to a diverse set of legal tasks, but leveraging unlabeled domain-specific corpora for self-supervised training of foundation models may provide complementary improvements to few-shot methods.

Access to Clean In-Domain Data. Some recent efforts have sought to create large labeled datasets for more challenging legal benchmark tasks through automation (Zheng et al., 2021) or manual annotation by volunteer legal experts (Hendrycks et al., 2021c). These efforts have demonstrated that larger language models that are pretrained on more data achieve performance gains on certain challenging tasks, compared to more limited gains observed in other settings (Chalkidis et al., 2020; Elwany et al., 2019; Zhong et al., 2020). This work suggests that larger legal benchmark datasets may be necessary to observe further gains from applying transfer learning techniques to foundation models. However, creating benchmark datasets for tasks that are legally meaningful and difficult from an NLP perspective can itself be challenging, as human expert annotation can be costly and automated methods that utilize conventional tokenization and sentence segmentation techniques can fail to account for unique aspects of legal text, such as the structure of legal citations (Bommarito et al., 2018; Savelka et al., 2017). As a consequence of these challenges, many existing legal domain-specific labeled datasets are small, not publicly available, or reflect simpler tasks that have been solved by methods often pre-dating the development of foundation models.

Much available legal data may also be unrepresentative. Since only a fraction of cases end up in legal opinions, it is unclear whether the disputes in publicly available data are representative of the typical disputes presented to a model in practice (Priest and Klein, 1984). Costly training data for more representative scenarios may be concentrated in the biggest law firms. These law firms may have the ability to retain and accumulate data across many cases and clients. One concern then is that foundation models could concentrate power even more among the few actors that have the resources to train models on in-domain data — unless the models can generalize sufficiently well.

Reliability. Finally, we again note that even if foundation models could successfully perform all tasks in the legal domain, deployment remains a major challenge: a failure of a foundation model in the law will have real, damaging consequences to both clients and attorneys (see also discussion on fairness, legality, and ethics in §5.1: fairness, §5.4: legality, and §5.6: ethics). For this reason machine translation software has already been deemed unreliable for use as evidence in some courts, although it continues to be relied upon in others.

Given all of these complexities, legal briefing and reasoning is likely beyond the capabilities of current models, but appears to be within the future realm of possibilities. As such, these serve as a potential lode star for the ongoing development of foundation models.