On the Opportunities and Risks of Foundation Models

The capabilities (§2: capabilities) of foundation models indicate that they have the potential to transform various sectors and industries, extending the role AI plays in society (§5: society). Among the myriad applications where foundation models may be applied, we will focus on three disciplines — healthcare (§3.1: healthcare), law (§3.2: law), and education (§3.2: law) — that are all foundational to societal function. Within each, we discuss the opportunities that foundation models pose for this domain alongside challenges (e.g., interpretability; §4.11: interpretability) and concerns (e.g., privacy; §4.7: security).

Healthcare and biomedicine

Healthcare and biomedicine are an enormous application area in society, for instance, with expenditures accounting for 17% of gross domestic product (GDP) in the US (Swensen et al., 2011; van Hartskamp et al., 2019; Keehan et al., 2020). Both healthcare (which focuses on the delivery of care to patients via diagnosis, treatment, and health administration) and biomedical research (which focuses on the scientific understanding of disease and the discovery of new therapies) demand significant expenses, time, and comprehensive medical knowledge (Yu et al., 2018; Korngiebel and Mooney, 2021). We envision that foundation models can be a central storage of medical knowledge that is trained on diverse sources/modalities of data in medicine (Krumholz et al., 2016; Soltanian-Zadeh, 2019; Suresh et al., 2020) (Figure 12 left), and can be queried/updated interactively by medical professionals (e.g., healthcare providers and biomedical researchers access published findings and upload new publications) (Ionescu et al., 2020) and queried by the public. As foundation models have strong adaptation capabilities (e.g., fine-tuning, prompting (Brown et al., 2020)), they can be efficiently adapted to various individual tasks in healthcare and biomedicine (e.g., question answering app used by patients (Klasnja and Pratt, 2012; Zhu et al., 2019; Daniel et al., 2019; Liu et al., 2020a), clinical trial matching system (Ni et al., 2015; Harrer et al., 2019; Beck et al., 2020) accessed by researchers and patients; Figure 12 right). This way, foundation models can be a central interface that supports various interactions between data, tasks, and people in healthcare and biomedicine, thereby advancing the efficiency and accuracy of healthcare/biomedical applications (Elbattah et al., 2021). We elaborate these opportunities in §3.1.1: healthcare-tasks and §3.1.2: biomed-tasks.

At the same time, healthcare/biomedical applications pose unique challenges that motivate further research in foundation models, such as integrating multimodal data in healthcare/biomedicine (Miura et al., 2021; Liu et al., 2021a) and observing ethical and legal regulations in medicine (privacy, safety and explainability) (Guan, 2019; Xu et al., 2019). We elaborate these challenges in §3.1.3: healthcare-biomed-challenge.

Opportunities in healthcare

Foundation models may improve the delivery of care to patients through healthcare providers and hospitals. Currently, healthcare cost increases every year (Keehan et al., 2020), and studies estimate that 30% of healthcare spending may be wasteful due to administrative inefficiency and preventable medical errors (Kocher, 2021). Moreover, as the demand for healthcare increases, the society faces a serious shortage in healthcare providers (Kirch and Petelle, 2017). This inefficiency and shortage in healthcare necessitate developing fast and accurate interfaces for healthcare providers and patients, such as automated aid systems for diagnosis/treatment, summarization of patient records, and answering of patient questions (Davenport and Kalakota, 2019; Nie et al., 2018; Wang et al., 2021b). In particular, in an urgent pandemic crisis such as COVID-19, fast diagnosis/screening (e.g., automatic analysis of chest X-ray images) as well as automated question answering for patients (e.g., symptom checking and care) and the public (e.g., disease prevention) are vital to reduce the spread of diseases and allocate healthcare resources for critical patients, saving more lives (Lalmuanawma et al., 2020). As foundation models have a strong capability to serve as an integrated knowledge reservoir, they can be queried and adapted to various individual tasks in healthcare. Below are examples of important tasks in healthcare that would benefit from foundation models.

Interface for healthcare providers

Foundation models can improve the efficiency and accuracy of care by providers. Healthcare providers spend unnecessary time editing electronic heath records (EHRs) (Kocher, 2021), and preventable medical errors (e.g., hospital readmissions, surgical errors) cause wastes in healthcare (Shrank et al., 2019; Shah et al., 2020). Foundation models can be adapted as an efficient and accurate interface into EHRs (clinical notes, lab value histories and imaging files) (Li et al., 2020c; Steinberg et al., 2021; Percha, 2021), helping healthcare providers create summaries of patient visitation (Krishna et al., 2020), retrieving relevant cases and literature, and suggesting lab tests, diagnosis, treatments and discharges (Zhang et al., 2019b; Rasmy et al., 2021). Foundation models can also be adapted to help a surgical robot monitor and achieve accurate surgeries (Diana and Marescaux, 2015; Agrigoroaie and Tapus, 2016; Yu et al., 2019). See §2.3: robotics for more discussions on foundation models for robotics.

Interface for patients

Foundation models can be adapted to serve as an interface to patients, providing relevant information about clinical appointments (Bates, 2019), answering patient questions related to preventive care (Demner-Fushman et al., 2020), along with relevant medical explanatory information (e.g., text and graphics that explain conditions) (Chaix et al., 2019), and helping assistive-care robots for patients (Jeong et al., 2015; Abdi et al., 2018). See §2.5: interaction for more discussion on foundation models for user interaction. Foundation models can also serve as an interface with the general public to answer questions related to public health and pandemic prevention (such as the COVID-19 case) (Bharti et al., 2020; Herriman et al., 2020). At the same time, we note that the interface must guarantee factual accuracy to ensure public trust in medical advice (Kreps and Kriner, 2020) (see §3.1.3: healthcare-biomed-challenge).

Opportunities in biomedicine

Foundation models may facilitate biomedical research such as discovery of drugs and understanding of diseases, which ultimately translates to improved healthcare solutions (Hanney et al., 2015). Currently, biomedical discovery requires significant human resources, experimental time and financial costs. For instance, drug development involves a complex process, from basic drug research of protein target identification and potent molecule discovery to clinical development (e.g., clinical trials) to the final drug approval, which typically takes over 10 years and costs more than one billion dollars (Wouters et al., 2020). Facilitating and accelerating biomedical discovery using existing data and published findings is an imperative problem in biomedicine (Yu et al., 2018). In particular, a novel disease outbreak such as COVID-19 costs millions of lives and trillions of dollars (Lalmuanawma et al., 2020; McKibbin et al., 2020); if we can speed up drug development for new diseases, that would be very helpful. Foundation models can be particularly helpful for biomedical discovery in two aspects. First, foundation models have a strong generative capability (e.g., coherent text generation in GPT-3), which can help generative tasks in biomedical research such as generating experimental protocols (clinical trials) and designing molecules that work (drug discovery) given existing data (Kadurin et al., 2017; Harrer et al., 2019). Second, foundation models have a potential to integrate diverse data modalities in medicine, which enables investigating biomedical concepts (e.g., disease) from multiple scales (using molecule-, patient- and population-level data) and multiple knowledge sources (using imaging, textual and chemical descriptions). This facilitates biomedical discoveries that are difficult to obtain if using single-modality data (Lanckriet et al., 2004; Aerts et al., 2006; Kong et al., 2011; Ribeiro et al., 2012; Wang et al., 2014; Wang et al., 2015c; Ruiz et al., 2020; Wu et al., 2021h). Foundation models also enable transfer knowledge across modalities. Lu et al. (2021a) showed how a transformer model trained on natural language (a data-rich modality) could be adapted for other sequence-based tasks such as protein fold prediction, which is a long-studied predictive task in biomedicine (Jumper et al., 2020). Below are examples of important tasks in biomedicine that will benefit from foundation models.

Drug discovery

To discover a drug or a therapeutic that treats a disease, researchers must first identify a target (e.g., proteins, genes, RNA causally implicated in the disease) and must then search for molecules (e.g., chemical compounds, antibodies) that bind to the target and treat the disease. Typically, identifying the appropriate target and generating a corresponding molecule requires years of expensive wet lab experiments (Hughes et al., 2011; Schenone et al., 2013; Schneider, 2018). Foundation models’ generativity can improve the search space and efficiency (see §2.4: reasoning), which not only reduces the amount of experiments but also helps to discover new and better drugs (Jin et al., 2018; You et al., 2018; Walters and Barzilay, 2020; Stokes et al., 2020). Moreover, the simultaneous solution of related drug discovery problems (i.e., target identification, efficacy prediction, side effect prediction, and others) by a single foundation model may improve the solutions to each of them (Ramsundar et al., 2015; Camacho et al., 2018; Duran-Frigola et al., 2020; Huang et al., 2021a). As an example, one area where foundation models have shown significant potential for impacting therapeutic design is the modeling of proteins using language models. Successful applications range from predicting viral mutations that can escape a vaccine-induced immune response to predicting protein docking potential for better design of therapeutic antibodies (Bepler and Berger, 2021; Hie et al., 2021; Tsaban et al., 2021; Wu et al., 2021b; Rives et al., 2021).

Personalized medicine

Personalized medicine aims to select the optimal treatment for individual patients based on their health history, genetics, imaging, and other personal measurements (Collins and Varmus, 2015; Ashley, 2016). For instance, given a set of drugs and a patient genome, foundation models may help predict which drug is likeliest to treat the patient with minimal side effects (Whirl-Carrillo et al., 2012; Tatonetti et al., 2012; Gerstung et al., 2017; Grinfeld et al., 2018; Adam et al., 2020). Foundation models are uniquely powerful in their ability to integrate multimodal patient data ranging from the EHR (Rajkomar et al., 2018) to medical imaging (Bera et al., 2019; Ouyang et al., 2020) to drug and molecular measurements (Gottlieb et al., 2011; Ruiz et al., 2020) to make an optimal prediction.

Clinical trials

Clinical trials study efficacy and safety of treatment or drug candidates. Conventional clinical trials are inefficient and costly: 80% of trials fail due to inability to show efficacy/safety or problems with patient matching (Ali et al., 2020; Liu et al., 2021c). Foundation models can help in the following: predicting potential failures and design promising clinical trial protocols (e.g., patient eligibility criteria) based on existing studies; and automating matching of eligible patients based on patient individual profiles, which are multimodal data including EHRs, gene sequence, etc. (Harrer et al., 2019).

Challenges and future research in foundation models

While there are potential opportunities for foundation models to help, healthcare/biomedical applications also pose unique challenges that motivate further research in foundation models.

Multimodality

Medical data are highly multimodal, with various data types (text, image, video, database, molecule), scales (molecule, gene, cell, tissue, patient, population) (Kong et al., 2011; Ruiz et al., 2020), and styles (professional and lay language) (Lavertu and Altman, 2019; Li et al., 2019). Current self-supervised models are developed for each modality (e.g., text (Lee et al., 2020b), image (Chaitanya et al., 2020), gene (Ji et al., 2021), protein (Jumper et al., 2020)), and do not jointly learn from diverse modalities. To learn the inter-modality and cross-modality information from these diverse multimodal medical data, we need to investigate both feature-level and semantic-level fusion strategies in the training of foundation models. If done effectively, this has a potential to unify biomedical knowledge and facilitate discoveries as discussed in §3.1.2: biomed-tasks.

Explainability

Explainability — providing evidence and logical steps for decision making — is crucial in healthcare and biomedicine (Holzinger et al., 2019), and is made obligatory under the General Data Protection Regulation (GDPR). For instance, in diagnosis and clinical trials, patient symptoms and temporal relevance must be explained as evidence. This helps the resolution of potential disagreement between the system and human experts. Explainability is also needed for informed consent in healthcare (Amann et al., 2020). However, current foundation models’ training objectives do not include explainability, requiring future research in this direction (Linardatos et al., 2021). Incorporation of knowledge graphs may be a step to further improve model explainability (Roberts et al., 2020; Xu et al., 2020; Jin et al., 2021). Readers are refered to §4.11: interpretability for more discussion on explainability.

Legal and ethical regulations

Healthcare applications must observe legal and ethical regulations with guarantees, such as patient safety, privacy and fairness. For instance, regarding safety, predictions made by foundation models must be factually accurate with established medical knowledge, and must quantify uncertainty or choose to defer to an expert when uncertain (Challen et al., 2019; Mozannar and Sontag, 2020). For privacy, the use of patient health records must observe the privacy laws, such as HIPAA (Act, 1996) in the case of the US. Federated learning is one potential solution to keeping the raw, sensitive data private in the training of foundation models (Chamikara et al., 2021). For fairness, researchers will need to be mindful of common pitfalls or otherwise risk exacerbating existing social inequalities (Chen et al., 2019; Wiens et al., 2019; Chen et al., 2020b). They must ensure that the training and evaluation data for foundation models is sufficiently representative of different sexes, races, ethnicities and socioeconomic backgrounds; an area where medical datasets and clinical trials have had a long history of bias (Martinez-Martin et al., 2020; Kaushal et al., 2020). Research is also needed to debias and regularize models to ensure fairness when representative data is scarce (Zhao et al., 2020a). Foundation model developers also need to consult with ethics and law researchers, and observe regulations in the specific circumstances (e.g., country, region) where they are deployed. We also refer readers to §4.7: security, §4.8: robustness, §5.1: fairness, §5.4: legality for details on privacy, robustness, fairness and legality.

Extrapolation

The process of biomedical discovery involves extrapolation. For instance, foundation models must be able to quickly adapt to new experimental technologies (e.g., new assays, new imaging techniques such as high resolution microscopy) or new settings (e.g., new target diseases such as COVID-19) (Jaroch et al., 2018; Benam et al., 2019). The ability to leverage existing datasets and extrapolate to new settings is a key machine learning challenge in biomedicine (Snell et al., 2017; Ma et al., 2021b). While GPT-3 exhibits some extrapolation behaviors (e.g., generating new text not seen before), its mechanism is unclear and still in its infancy. Further research is needed for improving the extrapolation capability of foundation models, especially when considering the diverse range of data modalities and tasks that is inherent to healthcare and biomedicine but is not commonly studied in current GPT-3 and related models. Also see §4.8: robustness.