The Past, Present, and Future of the Cognitive Theory of Multimedia Learning

The cognitive theory of multimedia learning (CTML) represents my continuing and evolving attempt to understand how meaningful learning works. Meaningful learning occurs when the learner engages in appropriate cognitive processing during learning, including attending to the relevant information in a lesson (i.e., selecting), mentally organizing the incoming information into a coherent cognitive structure (i.e., organizing), and connecting it with relevant knowledge activated from long-term memory (i.e., integrating; Mayer, 2021, 2022). Meaningful learning is indicated by performance on transfer tests, which involves being able to use the learned material in new situations. CTML focuses on how people learn meaningfully from academic material containing words and graphics. It focuses on techniques that prime appropriate cognitive processing during learning.

The development of new theories such as CTML is shaped by existing theories. For example, Camp et al. (2022) point out that CTML builds on classic conceptual frameworks from cognitive psychology, including dual-coding theory (Paivio, 1986), multi-stage model of memory (Baddeley, 1986), and cognitive load theory (Sweller, 1999, in press; Sweller et al., 2011). In particular, Camp et al., (2022, p. 17) note:

The power of theories is that they have the potential to remain valid and relevant through generations, while the power of new theories is that they build upon and expand those old theories. You might say that the old theories are the giants upon whose shoulders new theories, and thus new giants, stand. The cognitive theory of multimedia learning (CTML)...can be regarded as one such new giant.

The creation of CTML is clearly a team effort within the field of educational psychology. My modest goal in this essay is to share some introductory insights gleaned from my efforts at theory-building over the past decades, describe the evolution of CTML, summarize the current state of CTML, and speculate on possible future directions for CTML. First, in line with Greene’s (2022) call to carefully examine the process of theory development in educational psychology, I share seven insights about theory building that are exemplified in my decades-long attempts to explain how people learn from multimedia lessons.

Insight 1: Theory Building Depends on Intellectual Curiosity

CTML is a manifestation of my lifelong quest to understand how meaningful learning works. I began my journey on one of those amazing, crisp autumn days in Ann Arbor in 1969, as I started my graduate career in psychology at the University of Michigan. Soon within my first year in graduate school, under the mentorship of James Greeno, I developed a deep curiosity about how meaningful learning works. How can we teach so people can take what they have learned and use it productively in new situations? This seemingly simple question about teaching for transfer has been a driving force in my lab across more than four decades. Specifically, CTML focuses on some fundamental questions about learning and instruction with multimedia materials: How do people learn academic material consisting of words and graphics (i.e., multimedia learning)? How can we help people learn academic material consisting of words and graphics (i.e., design of multimedia instruction)? My curiosity about these kinds of questions is the engine that drives the development of CTML.

How did I get started on this theory-developing journey? During my first year of graduate school, while I was struggling to build my identity as a research psychologist, I had the good fortune to take a course entitled, Models of Thinking, taught by my advisor, Jim Greeno. That course got me thinking about how people are able to come up with creative solutions to problems, which lead me to a basic question: “How can we help people learn in ways so that they can take what they have learned and apply it to new situations?” This question about teaching for transfer, which formed in my mind during that course, has stuck with me all these years and has driven my research. I soon discovered that this is a classic, albeit elusive, issue both in psychology and in education that dates back to early days of research in learning and instruction.

Insight 2: Theory Building Is Grounded in Old Ideas

The search for theories of meaningful learning has a long history, including the insightful work of the Gestalt psychologists such as Wertheimer (1959) and Katona (1940); the groundbreaking work of developmentalists such as Piaget (1926) and Vygotsky (1978); and the creative work of memory researchers such as Bartlett (1932). In the field of educational psychology, a focus on instruction for meaningful learning has its roots in generative theories of learning by pioneers such as Wittrock (1974, 1989) and Ausubel (1968). Some key ideas rising from this work are meaningful learning as assimilation to schema (i.e., connecting incoming information with existing knowledge), meaningful learning as a generative activity (i.e., actively attending to relevant material, organizing it into a coherent structure, and relating it to relevant prior knowledge), and meaningful learning as knowledge construction (i.e., building mental representations in working memory). The development of CTML represents my attempts to understand and clarify these intriguing ideas based on empirical testing.

How did I get this insight? As an undergraduate, my goal was to read every classic psychology book, but in graduate school, my goal became more focused on reading every classic book related to how learning works and, particularly, books that could help me better understand my driving question of how to teach for transfer. I found amazing used bookstores in Ann Arbor, where I spent a lot of my spare time on a treasure hunt for books by the likes of Piaget (1926), Bartlett (1932), Wertheimer (1959), Katona (1940), Ebbinghaus (1913), Ausubel (1968), and many others. I was moved by the introductory quotation I found on the cover page to Ebbinghaus’ nineteenth century book, Memory: “From the oldest subject we will build the newest science.” Even as a beginning graduate student, I felt privileged to contribute in even a small way to this newest science of how the human mind works, which in my case turned out to the science of learning and instruction.

Insight 3: Theory Building Is Not a Straight, Planned-Out Path

My search for a theory of meaningful learning has not taken a straight, planned-out path, but rather has progressed in an irregular pattern of small steps based on fortuitous findings over decades, with invaluable contributions from a long list of collaborators and colleagues. CTML was not based on a systematic step-by-step career plan, and it certainly did not pop out of my head one day all in one nice polished package. It came out in fits and starts, with a long series of revisions, refinements, clarifications, deletions, and additions.

Where does this idea come from? I never set out to have a long-term career plan of systematic research on multimedia learning, but rather I have much shorter 2- or 3-year plans targeted on specific research questions. For example, back in the 1980s, after reading all of David Ausubel’s (e.g., Ausubel, 1968) writings, I became obsessed with advance organizers—material that is presented before a lesson that is intended to improve learning through activating relevant prior knowledge. That line of research led me to the unexpected conclusion that visual advance organizers could be effective in helping learners relate a new concept with familiar prior knowledge, which got me thinking about the power of connecting visual and verbal representations as a route to meaningful learning. As that idea incubated in my mind throughout the late 1980s and 1990s, I became interested in how to incorporate illustrations in text. That fateful exploration brought me to the formulation of what was to become the multimedia principle—people learn better from words and pictures than from words alone. In short, I often do not know where my theory-building is going but I simply follow the fruitful paths that my current research takes me.

Insight 4: Theory Building Is an Engineering Problem

Based on this journey, I have come to realize that theory building in educational psychology is much like solving a practical engineering problem. However, instead of building an increasingly better device to carry out some function, our theory-building task is to build a progressively better explanation of some educationally relevant phenomenon (such as how students develop meaningful learning that transfers to new situations). I just keep tinkering with the theory trying to make it work better based on new research evidence and logical reasoning. One indication of working better is the rate at which the work is cited by others and incorporated into their frameworks, as documented by Camp et al. (2022).

Where does this insight come from? When I look at the evolution of CTML, particularly through the ways I represented it visually, I see that I started with a core idea—there are several cognitive conditions for meaningful learning—and then continually tweaked the idea based on the research evidence that it generated and the new ideas I came across. When I read Paivio’s (1986) work on dual-coding, I realized its relevance and tried to incorporate that idea. When I read Miller’s (1956) and Baddeley’s (1986) and Sweller’s (1999) work on limited working memory capacity, I realized that fundamental idea had to be part of my theorizing. When I read Wittrock’s work on generative learning, it validated my thinking that appropriate cognitive processing during learning was a key to meaningful learning and led to my incorporating the SOI model (based on selecting, organizing, and integrating). As in engineering, my approach has been to keep redesigning what I have to make it better—which in my case means making it better able to explain a wider set of findings. In this way, my work began with what Kuhn (1962) would call a paradigm shift—a shift from behaviorist to cognitive views of how learning works—but then for all these ensuing years has involved continually improving on my original idea.

Insight 5: Theory Building Is an Iterative Process Involving the Persistent Interplay Between Research and Theory

As shown in Fig. 1, consistent with Greene (2022), I start with the kernel of a theoretical idea (i.e., theory), which leads me to a testable question (i.e., research question) that I examine in series of experiments (research design), which generates a pattern of findings (research evidence) that helps me improve my theoretical account (theory). The development of CTML represents many turns around the circle presented in the figure (both clockwise and counterclockwise). These turns depend on valuable discussions with colleagues concerning how to frame the next version of the theory, how to generate useful research questions (and know when to give up on unfruitful ones), how to design impactful studies, and how to interpret the results. The result is a series of iterations of a theory of how meaningful learning works, as described in the following sections.

Where did I get this insight? Let me give an example. Most of the early work by my collaborators and me on instructional design principles for multimedia learning focused on the instructional goal of reducing extraneous processing—that is, reducing the learner’s cognitive processing that is not directed towards learning the content so that the learner can use cognitive resources to make sense of the material. This led, for example, to the coherence principle in which people learn better when we remove unneeded visual and verbal material from a lesson. However, for some lessons, even when we eliminated unneeded material, students still had trouble learning it, perhaps because they just did not want to put out effort to make sense of the material. This led us to develop a new kind of instructional design goal, which I called fostering generative processing—motivating the learner to actively engage with the material. This goal suggested a whole new set of multimedia learning techniques based on prompts to engage in generative learning activities during learning, such as writing a summary, drawing an illustration, creating a graphic organizer, and explaining to others. In this way, the research evidence (i.e., not being able to improve learning through reducing extraneous material), led to a new theoretical idea (i.e., fostering generative processing) which led to new research avenues (i.e., incorporating generative learning activities).

Insight 6: Theory Building Depends on Persistence in Collecting New Research Evidence

Also in line with Fig. 1, building the cognitive theory of multimedia learning depends on a persistent commitment to experimental comparisons using a value-added design. In value-added experiments, we compare the learning outcomes (and, when possible, learning processes) of people who learn from a base version of a multimedia lesson with those of people who learn from the lesson with one feature added. For example, we can compare learning from a narrated animation on lightning formation versus learning from the same lesson with the words presented as printed captions at the bottom of the screen. Theory development depends on a strong foundation of research evidence, which flows from a willingness to replicate effects detected in value-added studies.

Where did this insight come from? Let me give you an example. When Roxana Moreno was a graduate student in my lab, she came to my office with the preposterous (to my way of thinking) idea that using conversational language in an online science lesson would improve learning outcomes over our existing lessons using traditional, formal language. In spite of my skepticism, we worked out plans for a series of rigorous experiments, each one of which came back with strong positive results. After replicating the effect multiple times with multiple content topics, I finally allowed myself to share the personalization principle with the larger research community, that is, the idea that people learn better when instructors use conversational wording rather than formal wording. This opened my eyes to the idea that online multimedia learning depended not only on cognitive processing, but also social and affective processing, which has led to several ongoing new directions for our research (Horovitz & Mayer, 2021; Lawson & Mayer, 2022; Lawson et al., 2021).

Insight 7: Theory Building Is a Team Activity

CTML would not have happened if I was tasked with working alone. I have had the pleasure of working with dozens of collaborators over the years who have helped build CTML. The theory benefits from collaborations with students, campus colleagues, and fellow educational psychologists from near and far, including visitors from around the world who have contributed to our lab over the years. The theory has been shaped by advances and feedback from the larger community of scholars, many of whom are represented in the various editions of The Cambridge Handbook of Multimedia Learning (Mayer, 2005, 2014; Mayer & Fiorella, 2022). CTML also has benefitted from ideas from competing theories such as cognitive load theory (Paas & Sweller, 2022; Sweller, 1999, in press; Sweller et al., 2011) or the integrated model of text and picture comprehension (Schnotz, 2022, 2023). Part of theory building is being able to convince your peers and to be convinced by them.

Where did this insight come from? As I look over the hundreds of research papers on multimedia media that have my name on them, I see that the vast majority were co-authored with graduate students or visitors to my lab. I seek to meet on a weekly basis with our lab research team—graduate students, postdocs, and visitors. I also meet regularly with students, visitors, colleagues, and anyone else I can find who is interested in talking about improving multimedia instruction. I cherish these meetings because they are essential in giving me a chance to work out new ideas about multimedia learning.