The Past, Present, and Future of the Cognitive Theory of Multimedia Learning

The cognitive theory of multimedia learning is an evidence-based description of how people learn from multimedia instructional messages. A multimedia instructional message is instructional material consisting of words (e.g., printed text or spoken text) and graphics (e.g., illustrations, photos, animation, video, or immersive virtual reality) intended to foster new knowledge or skills in a learner. A multimedia instructional message can be presented in print (e.g., as a book), on a computer screen (e.g., as an instructional video or narrated animation or a simulation game), or in virtual reality via a head-mounted display (e.g., as an interactive simulation). The theory yields implications for the design of effective multimedia instructional messages, which are rendered as design principles.

Some of the advances in CTML have been fostered by advances in educational technology including instructional video, animation technology, technologies for creating onscreen agents, immersive virtual reality, and educational games, but it was not my intention to study educational technology per se. In fact, the theory began by studying learning from printed text and illustrations rendered on paper, and my focus has always been on how to design effective instruction involving words and graphics. In short, my focus is on how to help people learn academic content rather than on the capabilities of the latest educational technologies.

In this section, I summarize the current state of the cognitive theory of multimedia learning including the guiding assumptions, the memory stores, the cognitive processes, the demands on cognitive capacity, and three instructional goals. I also summarize 15 evidence-based multimedia instructional design principles based on CTML. More detailed descriptions are available in Mayer (2021, 2022).

The cognitive theory of multimedia learning is represented in Fig. 8. A multimedia instructional message enters the learner’s cognitive system through their eyes and ears. Printed words and graphics are held briefly in visual sensory memory, and spoken words are held briefly in auditory sensory memory. As these images fade, the learner can pay attention to some of the material, which is transferred to working memory for further processing. In working memory, the learner can organize the pictorial material into a pictorial model and the verbal material into a verbal model and integrate corresponding pictorial and verbal representations with each other and with relevant knowledge from long-term memory. The outcome is meaningful knowledge that is stored in long-term memory and can be applied to new situations.

Guiding Assumptions of the Cognitive Theory of Multimedia Learning

The cognitive theory of multimedia learned as represented in Fig. 8 is based on three guiding assumptions derived from cognitive science: dual channels, limited capacity, and active processing. The dual-channels assumption is that humans have separate but interacting channels for processing auditory/verbal information and pictorial/visual information (such as narration and animation, respectively). The limited-capacity assumption is that humans can process only a few pieces of information in each channel at one time. The active-processing assumption is that meaningful learning occurs when the learner engages in appropriate cognitive processing during learning, including selecting relevant material to attend to in a lesson, mentally organizing the incoming material into a coherent representation in working memory, and mentally connecting it with corresponding representations and with relevant prior knowledge activated from long-term memory.

Three Memory Stores in the Cognitive Theory of Multimedia Learning

CTML has three memory stores, which are represented as boxes in Fig. 8 sensory memory, working memory, and long-term memory. Sensory memory holds complete visual images (in visual sensory memory) that enter through the eyes and complete auditory images (in auditory sensory memory) that enter through the ears, but the images fade rapidly within a fraction of a second. Working memory holds pictorial/visual and auditory/verbal pieces of information that the learner has attended to before they decay from sensory memory. This information can be re-arranged, but only a few pieces of information can be processed in each channel at any one time. Long-term memory is the learner’s permanent storehouse of knowledge, parts of which can be activated and brought into working memory during learning.

Five Cognitive Processes in the Cognitive Theory of Multimedia Learning

CTML has five cognitive processes, which are represented as arrows in Fig. 8: selecting words, selecting images, organizing words, organizing images, and integrating. Selecting words refers to attending to relevant parts of the printed text, and selecting images refers to attending to relevant parts of the presented graphics. Organizing words refers to arranging the relevant words in to a verbal model in working memory, and organizing images refers to arranging the relevant parts of the graphics into a pictorial model in working memory. Integrating refers to making connections between corresponding verbal and pictorial representations in working memory as well as relevant knowledge from long-term memory. Meaningful learning depends on the learner engaging in appropriate cognitive processing involving selecting, organizing, and integrating. Instructional design is intended to guide these processes.

Three Demands on Cognitive Capacity

A central tenet of the cognitive theory of multimedia learning is that working memory capacity is limited, but there are three demands on that limited cognitive capacity during learning: extraneous processing, essential processing, and generative processing. Extraneous processing is cognitive processing that does not support the instructional goal; the amount of extraneous processing depends on the degree of poor instructional design, such as presenting extraneous verbal or pictorial information in a lesson. Essential processing is cognitive processing to mentally represent the presented material in working memory; the amount of essential processing depends on the inherent complexity of the material for the learner, such as presenting many inter-related concepts in a fast-paced lesson. Generative processing is cognitive processing aimed at making sense of the incoming information; the amount of generative processing depends on the learner’s level of motivation to exert effort to understand the lesson. The three kinds of processing each require some of the learner’s limited cognitive capacity, so capacity that is used for extraneous processing takes away from capacity that could be used for essential and generative processing, and cognitive capacity that is used for essential processing takes away capacity that could be used for generative processing.

Three Instructional Goals

The three demands on cognitive capacity give rise to three instructional goals: minimize extraneous processing, manage essential processing, and foster generative processing. Consider a situation in which a poorly designed lesson causes the learner to allocate almost all of their cognitive capacity to extraneous processing, so they do not have adequate remaining cognitive capacity to engage in needed essential and generative processing. In this case, an important instructional goal is to minimize extraneous processing. This can be accomplished, for example, by eliminating unneeded words and graphical elements from a lesson.

Consider another situation in which extraneous processing has been reduced, but the lesson is so complex that the amount of needed essential processing exceeds the learner’s cognitive capacity. In this case, an important instructional goal is to manage essential processing. This can be accomplished, for example, by breaking a continuous lesson into manageable chunks that can be paced by the learner.

Finally, let us assume that we have minimized extraneous processing and managed essential processing, so cognitive capacity is available for generative processing, but the learner is not motivated to exert effort to understand the material. In this case, an important instructional goal is to foster generative processing. This can be accomplished, for example, by prompting the learner to engage in a generative learning activity such as writing a summary or self-testing during pauses a lesson.

Fifteen Multimedia Instructional Design Principles

Table 6 summarizes 15 principles supported by research by my colleagues and me over the years. The first set of five principles addresses the instructional goal of reducing extraneous processing; the second set of four principles addresses the instructional goal of managing essential processing; and the third set of six principles addresses the instructional goal of fostering generative. Each principle is subject to boundary conditions including for whom the principle applies, for which kind of lesson the principle applies, and under what circumstances the principle applies. These are described in more detail in Multimedia Learning (Mayer, 2021). Many more principles are described in the wider literature, such as in The Cambridge Handbook of Multimedia Learning (Mayer & Fiorella, 2022).