Cognitive Architecture and Instructional Design: 20 Years Later

Since the publication of the 1998 article, there has been an ongoing research effort to examine issues related to the measurement of cognitive load. Therefore, one would expect that in the past 20 years substantial progress has been made regarding these issues. Here, we evaluate this progress and present a concise overview of three major developments regarding the measurement of cognitive load. This evaluation is based on previous reviews of cognitive load measurement, such as the ones presented in the theoretical articles of Paas et al. (2003, 2008); and van Gog and Paas (2008). In addition, it takes account of a recent reconceptualisation of germane cognitive load as referring to the actual working memory resources devoted to dealing with intrinsic cognitive load (Sweller et al. 2011). As a result of this reconceptualisation, only intrinsic and extraneous cognitive load are distinguished as basic categories of cognitive load.

The first important development is related to the further specification of the subjective measurement technique that was originally introduced by Paas (1992) to provide an overall measure of cognitive load. Although this measure has been extensively and successfully used showing good psychometric properties, some researchers remain sceptical about its capacity to measure cognitive load, even when comparisons with physiological measurement techniques have shown that the subjective rating scale is just as valid and reliable and easier to use as objective techniques (e.g. Szulewski et al. 2018). The main advantages of the subjective technique over physiological techniques are its sensitivity and its simplicity. In contrast to physiological measurement techniques, the subjective rating scale is sensitive to small differences in invested mental effort and task difficulty. Whereas the simplicity of the subjective rating scale is considered its major strength, because it can be easily used in research and practice, it is also considered by many as its major weakness. Due to its simplicity, it provides an overall measure of cognitive load (i.e. intrinsic plus extraneous load) and therefore cannot easily be used to differentiate between the different types of cognitive load. This has resulted in a search for techniques that can be used to differentiate between the different types of cognitive load as well as a search for ‘objective’ measurement techniques that can be used as an online measure of cognitive load. Before these two research developments are discussed in more detail, we first will discuss the ongoing further specification of the subjective rating scale technique.

Paas et al. (2003) and van Gog and Paas (2008) have noted that the subjective rating scale is used in many different ways than originally suggested by Paas and van Merriënboer (1994a, 1994b). Among other differences, researchers have used different verbal labels (i.e. ‘task difficulty’ instead of ‘invested mental effort’), used fewer categories (e.g. 5 or 7 instead of 9) and used only one measurement after all learning or test tasks, instead of an average of multiple measurements after each learning and test task. Whereas the first two examples would require more research into the psychometric properties of the adapted rating scales, the latter example has been further investigated by Gog et al. (2012) and Schmeck et al. (2015). They compared the original way of measuring load using an average score based on multiple measurement after each learning and test task to an adapted approach using only one measurement after the learning phase and one after the test phase. The question was whether this would lead to a different estimate of the magnitude of cognitive load. Results showed that the one measure after the instruction or test phase was always higher than the average of multiple measurements during the learning or test phase. Although the exact reason for the difference is not clear, the higher single score after the learning or test phase could be suggestive for depletion of working memory resources (Chen et al. 2018).

There are still a lot of questions that need to be answered in future research with regard to the optimal use of the subjective ratings scale. What is the effect of time on task on perception of invested mental effort or task difficulty? What are the effects of age and gender on the ratings? Do we need to present participants with a baseline before they start to rate? More research is needed to answer these and other questions and further specify the conditions for effective use of the rating scale.

The second development is related to extending the subjective technique by designing questionnaires that can differentiate between the different types of cognitive load. Whereas several researchers have tried to measure only changes in one specific type of cognitive load (e.g. Ayres 2006), others have investigated methods to measure the different types of cognitive load (e.g. Cierniak et al. 2009). One notable development was described by Leppink et al. (2013, 2014), who investigated the usefulness of a new psychometric instrument in which the different types of cognitive load were represented by multiple indicators. The authors concluded that the results of both studies provided support for the assumption that intrinsic and extraneous cognitive load can be differentiated using their 10-item psychometric instrument. Although it is clear that more empirical evidence is needed, for example, by using different questions and looking at different domains, the results so far are promising regarding the instrument’s capability of distinguishing between different types of cognitive load.

The third development is related to the ongoing efforts to find more objective measures of cognitive load. To this end, cognitive load researchers have been using secondary task techniques and physiological measures of cognitive load. Based on the assumption of a limited working memory capacity, secondary task techniques use performance on a secondary task as an indicator of cognitive load imposed by a primary task. It is assumed that low or high performance on the secondary task are indicative of high and low cognitive load imposed by the primary task. A recent example of a secondary task technique is the rhythm method developed by Park and Brünken (2015), which consists of a rhythmic foot-tapping secondary task. Korbach et al. (2017) showed that this technique was sensitive to hypothesised differences in cognitive load between a mental animation group, a seductive detail group and a control group. However, secondary task techniques have been criticised for their intrusiveness (i.e. imposing an extra cognitive load that may interfere with the primary task; Paas et al. 2003) and inability to differentiate between different types of cognitive load.

Physiological techniques are based on the assumption that changes in cognitive functioning are reflected by physiological variables. Whereas several researchers have proposed to use neuroimaging techniques, such as functional magnetic resonance imaging (fMRI; e.g. Whelan 2007), a technique that has actually been used in cognitive load research is electroencephalography (EEG; Antonenko and Niederhauser 2010; Antonenko et al. 2010). Using a hypertext-based learning environment, Antonenko and Niederhauser (2010) showed that several aspects of the EEG reflected hypothesised differences in cognitive load. A more frequently used technique for measuring cognitive load is based on eye-tracking variables, such as pupil dilation, blink rate, fixation time and saccades. For example, van Gerven et al. (2004) showed that pupil dilation is positively correlated with cognitive load in young adults, but not in old adults. Although, it has become increasingly easy to use physiological measures due to the development of mobile measuring devices, much more research is needed into these techniques and their potential to measure cognitive load.