We’ve all been in a similar situation. Imagine you are on the highway, driving to a doctor’s appointment, and you are running late. It’s also raining. Your children are in the back seat, and they are making too much noise. Your cellphone keeps going off.
What are the odds you’re going to get into an accident?
What is Mental Workload?
The study of mental workload (also known as cognitive workload) is one of the most important variables in psychology, ergonomics, and human factors for understanding performance. Research articles about mental workload have almost tripled since the 1980s [1]. Despite the popularity of research in this area, there are many possible definitions for cognitive workload, depending on how you look at it.
To illustrate: picture a simple system where you (the operator) are learning to drive for the first time (the task).
One way to think about workload is by objectively looking at the task itself. This is often referred to as taskload. For example, driving a car with a manual transmission has a much higher taskload than driving with an automatic transmission. This is because when you use a stick shift, you also have to think about engaging the clutch and shifting gears at the right time (this could be a conscious, or non-conscious cognitive process).
These additional tasks require you to process more information. If the demands of these tasks exceed your capability to handle them all at once, then the vehicle will stall (likely eliciting honks from the cars waiting behind you too).
However, another way to think about workload is through your subjective experience of driving. There is a tremendous amount of variability in how different people can handle the same taskload, depending on the individual’s prior experience, general aptitudes, history of training, and other extraneous factors. For example, if you learn to drive alongside a friend who plays racing games all day, their perception of the task may be very different from yours, even though the taskload remains the same. Similarly, if you go to driving school, and spend time practicing each day, driving a stick shift will become much easier, and you won’t need as many cognitive resources to operate the vehicle properly. Your subjective workload of driving a stick shift will decrease. This is sometimes referred to as an increase in workload efficiency [2].
So, a good way to define mental workload as a whole is as the product of factors that contribute to one’s workload efficiency for a given taskload. But, this definition may differ slightly depending on the field you are in and the research question you are asking.
Understanding this formula is extremely important for human factors researchers because of its practical applications: reducing human errors which can increase both productivity and safety in the field [3].
How can we measure Cognitive Workload?
Just as there are many possible definitions for mental workload, there are many different ways to measure it. No one sensor can give a complete view of how someone is reacting to a task. In a sense, this makes mental workload the quintessential use case for multimodal biosensors!
Here is a quick overview of the three most popular sensors used to study mental workload:
Electrodermal Activity / Galvanic Skin Response
An increase in mental workload is often accompanied by an increase in physiological arousal, so it is no surprise that many papers use skin conductance as an indirect measure of workload.
For example, one study by the New England University Transportation Center at MIT asked participants to do a cognitive task while driving in a car simulator [4]. The more challenging the memory task, the greater the increase in absolute Electrodermal Activity (EDA), which eventually reached a plateau after which performance in both the cognitive and driving tasks began to suffer.
Additional studies have also shown that the frequency content of the EDA signal will also change when participants do arithmetic, reading and memory tasks [5, 6].
Heart Rate
Similar to EDA, differences in heart rate are also observed with an increase in mental workload. Heart rate will increase as a task gets more difficult [7], or if additional tasks are added [8]. Heart rate in terms of beats-per-minute, as well as heart rate variability, are popular metrics, especially in automotive and aviation research.
Electroencephalography (EEG)
Finally, EEG is one of the most popular metrics for studying workload because it records the electrical activity of the brain itself, but it is also the most complex to analyze. Alpha band power is one of many possible metrics used to classify workload using EEG. For example, alpha activity has also been shown to decrease in pilots during the take-off and landing phases of flight [9]. Some commercially-available headsets, like the Emotiv EPOC and the ABM B-Alert X-series headsets, have their own proprietary measures of mental workload [10].
These sensors are often used in combination with one another or with other sensors like respiration or pupillometry, and are often reinforced with subjective measures like standard workload questionnaires for self-report, like the NASA Taskload Index [11]. These sensors are also often combined with eye tracking more generally, providing an understanding of where attention was placed when mental workload increased or decreased.
Who Studies Mental Workload?
The study of mental workload has traditionally fallen within the domain of researchers in psychology, human factors and ergonomics, primarily with applications that are known as “safety-critical” such as automotive, aviation, air traffic control, space flight, and defense. More recently, studies of workload have also become of interest to researchers in the fields of media and communications, human-machine interaction, medical simulation and training, behavioral economics and finance.
With so many potential applications, there is no universal solution to studying mental workload, and no single standard method that is recommended. Luckily, there is a large body of literature describing mental workload from multiple angles and using many different tools, but this can be almost overwhelming. With so many choices, which is the right combination of sensors for you?
You’ll notice that mental workload studies that are mentioned in this article employ many different kinds of taskloads: arithmetic tasks, reading tasks, memory tasks, control interface tasks, and specialized tasks like flight or driving, either in a simulator or in a real vehicle. When selecting the right sensors to use, it’s always a good idea to look at the prior scientific literature and see what hardware and paradigms were used for your specific use case. This will enable you to make the most scientifically-validated choice for the kind of research question you are trying to answer, as well as give you a sense of what each method can and can’t provide.
Mental workload research remains immensely valuable for many current disciplines, and is poised to become an important variable in new and emerging fields. While advances in machine learning and artificial intelligence will continue to inform an understanding of human behavior and interactions in different settings, this work will require a solid foundation built on validations of mental workload.
References
[1] Young, M. S., Brookhuis, K. A., Wickens, C. D., Hancock, P. A. (2014). State of Science: Mental Workload in Ergonomics. Ergonomics, 58(1):1-17.
[2] Xie, B., Salvendy, G. (2000). Review and reappraisal of modelling and predicting mental workload in single- and multi-task environments. Work. Stress, 14 (1), 74–99. https://doi.org/10.1080/026783700417249.
[3] Moray, N.E., (1979). Mental Workload: its Theory and Measurement. Plenum Press, New York.
[4] Mehler, B., et al. (2009). Impact of incremental increases in cognitive workload on physiological arousal and performance in Young adult drivers. Transport. Res. Rec.: J. Transport Res. Board 6–12 2138 Transportation Research Board of the National Academies.
[5] Shimomura, Y., Yoda, T., Sugiura, K., Horiguchi, A., Iwanaga, K., Katsuura, T. (2008). Use of Frequency Domain Analysis of Skin conductance for Evaluation of Mental Workload. J Physiol Anthropol, 27(4): 173-177.
[6] Nourbakhsh, N., Wang, Y., Chen, F., Calvo, R. A. (2012). Using galvanic skin response for cognitive load measurement in arithmetic and reading tasks. OzCHI’12 Proceedings of the 24th Australian Computer-Human Interaction Conference, 420-423.
[7] Reimer, B., Mehler, B., Coughlin, J. F., Godfrey, K. M., Tan, C. (2009). An On-Road Assessment of the Impact of Cognitive Workload on Physiological Arousal in Young Adult Drivers. Proc Automotive ui’09, Acm, 115-118.
[8] Fournier, L.R., et al. (1999). Electrophysiological, behavioral, and subjective indexes of workload when performing multiple tasks: manipulations of task difficulty and training. Int. J. Psychophysiol. 31, 129–145
[9] Wilson, G.F. (2002). An analysis of mental workload in pilots during flight using multiple psychophysiological measures. (Int. J. Aviat. Psychol). 12 (1), 3–18
[10] Berka, C., Levendowski, D. J., Lumicao, M. N., Yau, A., Davis, G., Zivkovic, V. T., … Craven, P. L. (2007). EEG Correlates of Task Engagement and Mental Workload in Vigilance, Learning, and Memory Tasks. Aviation, Space, and Environmental Medicine, 78(5), 14.
[11] Hart, S. G., Staveland, L. E. (1988). “Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research” (PDF). In Hancock, Peter A.; Meshkati, Najmedin (eds.). Human Mental Workload. Advances in Psychology. 52. Amsterdam: North Holland. pp. 139–183. doi:10.1016/S0166-4115(08)62386-9. ISBN 978-0-444-70388-0.
