Creating great products and services requires an understanding of what works, and what doesn’t for the user. Creating this understanding is however a hard problem to solve. The entire field of UX (user experience) or HCI (human-computer interaction) research exists in essence to solve this problem.
There are numerous reasons why this research is so difficult, including the diversity of settings or products that the user is exposed to, the continually-changing nature of any experience, as well as the fact that each participant will react in their own individual terms. There is a lot that goes on with any interaction.
Getting a user experience right is of course extremely powerful for a company (and the factors that lead to this are worthy of study in their own right), which is why a variety of methods are used in an attempt to generate an understanding of what works, and what doesn’t.
Below we will go through some of the methods that are traditionally used in user experience research, and also describe how new biosensor-based methods are helping shape this field by providing data that has often been crucial, yet inaccessible to user experience researchers.
User Experience Data
Attempts to understand user experience often use a variety of methods, including:
- Observation – in which a researcher follows the behavior and actions of a user (e.g. does the user smile when interacting with the product). This approach can provide a lot of data, but is laborious to record, and highly subjective.
- Think-aloud – in which the user is asked to report on their ongoing experience of interaction with the product. This approach is intended to provide insight into the emotional response of the participant, but ultimately impacts the flow of the experience, and as with the observation approach, it is easily biased.
- Heuristic Evaluation – a structured set of guiding tasks that are intended to prompt answers to critical areas of concern for designers. The exact formula can differ depending on the method used, but the principle is the same. This method can provide consistency in which actions or responses are required of each participant. While this method benefits from providing a well-defined structure, it is also ultimately subjective as it relies on the explicit thoughts of participants.
- Interviews / questionnaires – a set of structured or unstructured questions that must be implemented either during the experience (and thus interrupt the flow of the experience) or afterwards with retrospection (which fails to capture information about the ongoing experience). This approach can provide important qualitative data (particularly with regards to explicit memories of the user experience), but is easily biased, even unintentionally.
- Task analysis metrics – measures of explicit task performance, such as how fast a product is found on a webpage, how long it takes to register for a service, or how many key presses were used to download some information [1]. This provides an objective measure of efficiency, but misses the crucial emotional component of user experience.
As with many research approaches, having more data is usually a good thing (as long as you know what to do with it), and it’s clear that a combination of approaches can help build a cohesive view of user experience. Many of these measures attempt to provide an understanding of the emotional experience, as this is so central to a product’s success.
As John McCarthy and Peter Wright argue in their book “Technology as Experience”: “any account of what is often called the user experience must take into consideration the emotional, intellectual, and sensual aspects of our interactions with technology” [2]. Any successful product needs to be both intuitive to use and also enjoyable in its use.
While the approach of using the aforementioned measures has been (and will continue to be) incredibly valuable in creating an understanding of user experiences, their subjective nature ultimately means that certain aspects of the experience won’t be reliably captured.
Modern Approaches to UX Research
To bridge this gap, modern UX work and research has increasingly used biosensors to provide data that is more objective, consistent, and doesn’t require any interruption of the experience. This approach utilizes tools that collect physiological data, that can then be related to the user’s experience – the emotional, cognitive, and attentional processes that are otherwise so difficult to objectively capture.
This is largely a passive process, in that the data is captured without any conscious effort from the participant. While devices may need to be attached to the participant (such as electrodes to two fingers for measurements of galvanic skin response), the devices are largely unintrusive and non-invasive.
But what are biosensors? Broadly speaking, a biosensor is any device that measures biological activity. For UX research, this can involve:
- Eye trackers for measuring the movement of the eyes, which provides information about attention
- Facial expression analysis, based on video recordings of expressed emotions
- fEMG (facial electromyography), a method to detect facial muscle movement which can then be related to valence
- GSR (Galvanic Skin Response, otherwise known as Electrodermal Activity or EDA) which measures physiological and emotional arousal
- ECG (electrocardiography) which is used to measure heart rate activity. This data can also provide other measures such as heart rate variability, or interbeat interval, which have been used to provide information about emotional arousal
- EEG (electroencephalography), a device for measuring brain activity, which can provide information about feelings of approach or avoidance, among other measures
While this approach has clear methodological benefits, such as providing continuous and unbiased measurements, the collected data can also be quickly translated into actionable insights for UX improvements. Below, we will go through some examples of research that have used biosensors to generate a better and more reliable understanding of user experience.
User Experience Research
A study by Nacke and Lindley [3] used a combination of methods, including EEG, ECG, fEMG, GSR, and eye tracking. They tested how participants experienced gameplay, finding correlations between the collected signals and other measures (both subjective and objective) of the experience. This study was particularly useful as a proof-of-concept, showing the validity of the measures to provide instructive, real-time, and objective data regarding the interaction.
Other gameplay research carried out at the University of Sussex used both observation and biosensor-based methods, and ultimately state that their “results show that observation-based techniques can expose the majority of issues relating to usability, however the biometrics-based approach enabled researchers to discover latent issues in relation to players’ feelings, immersion and gameplay experience and, in certain categories of issue, reveal up to 63% more issues than observation alone” [4]. While observations methods proved necessary, the combination with biosensors provided a new level of understanding.
Pilot research from the University of Southern Indiana [5] looked at the use of biosensors in collecting objective data while users examined business dashboards. The dashboards must be designed in such a way to relay the most useful and relevant information as quickly as possible, without requiring strenuous cognitive effort from the user, so that the best possible decisions can be made.
By using eye tracking, facial expression analysis, GSR, and EEG, (with iMotions) the study investigated how attention and cognitive effort could be tracked using such biosensors, ultimately stating that they were “making a call for more direct and objective measures of user responses in business dashboard context”.
Further research used eye tracking to investigate user attention when using a medical device [6]. The researchers used eye tracking glasses and the iMotions software. They note that “Normally, the device manufacturer uses verbal self-reporting protocols, interviews and observations in their formative usability tests during product development”.
Using eye tracking “provided five unique insights and findings”, including information about when the user missed buttons, when text was skipped, and a better understanding of the visual distraction during device use. They also note that using eye tracking provided faster results, a greater level of detail to findings, and that this is done so without distracting or influencing the user experience.
Researchers have also used a combination of eye tracking, ECG, GSR, and facial expression analysis to triangulate attention, emotional arousal, and expressed emotions to create “emotional heatmaps” on webpages [7]. These heatmaps are formed from the processed signals to create a visual overlay that relays information about how someone feels when they looked at a certain point of the screen.
The researchers state how “This triangulated approach makes it possible to visually analyze users’ various emotional and cognitive states for specific areas of a given interface (e.g., cognitive load combined with emotional valence)”, a powerful tool for understanding user experience.
In conclusion, UX researchers are increasingly using biosensors to advance their understanding. These methods are best used in combination with traditional methodologies that provide the foundation of understanding, while the biosensors provide the data for a deeper comprehension of the user experience.
Eye Tracking
The Complete Pocket Guide
- 32 pages of comprehensive eye tracking material
- Valuable eye tracking research insights (with examples)
- Learn how to take your research to the next level
References
[1] Yao, L., Liu, Y., Li, W., Zhou, L., Ge, Y., Chai, J., & Sun, X. (2014). Using physiological measures to evaluate user experience of mobile applications. In Engineering Psychology and Cognitive Ergonomics (pp. 301-310). Springer International Publishing.
[2] McCarthy, J., & Wright, P. (2007). Technology as experience. Cambridge, Mass: MIT Press.
[3] Nacke, L. & Lindley, G.A. (2008). Flow and immersion in first-person shooters: Measuring the player’s gameplay experience. Proceedings of the 2008 ACM conference FuturePlay, Toronto, Canada.
[4] Mirza-babaei, P., Long, S., Foley, E. (2011). Understanding the Contribution of Biometrics to Games User Research. DiGRA ’11 – Proceedings of the 2011 DiGRA International Conference: Think Design Play, Utrecht, The Netherlands.
[5] Bacic, D. (2017). Understanding Business Dashboard Design User Impact: Triangulation Approach Using Eye-tracking, Facial Expression, Galvanic Skin Response and EEG Sensors. In: AMCIS 2017., Boston, MA.
[6] Koester, T., Brøsted, J. E., Jakobsen, J. J., Malmros, H. P., Andreasen, N. K. (2017). The Use of Eye-Tracking in Usability Testing of Medical Devices. Proceedings of the 2017 International Symposium on Human Factors and Ergonomics in Health Care.
[7] Georges, V., Courtemanche, F., Senecal, S., Baccino, T., Fredette, M., & Leger, P. M. (2016). UX Heatmaps: Mapping User Experience on Visual Interfaces. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 4850-4860). ACM.
