How to do Website UX and Usability Testing: Guide to Advanced Methods

User Experience (UX) is a broad concept that encompasses many aspects of a user’s interaction with your product and company; namely “how they feel about you.” Usability is more the practical side of things, the main question being answered is “ease of use.” The combination of the two leads to a better understanding of emotional experience and interaction with your brand and hopefully adds to a memorable, positive experience. But, how do you test for both?

Every website, app, or tangible product should go through some degree of UX & usability testing. Since just about everything can fall under this category, we are going to focus on web-based interactions and highlight some main considerations when embarking on advanced UX & usability testing with biometrics. The primary elements of advanced UX & usability will focus on how to approach different modalities, experiments, and considerations for web-based testing.

Content:

Define your scope: The importance of formulating the right business question

At iMotions we of course highly encourage the use of biosensor tools for UX testing, but it should not be the first consideration when developing the scope of your testing. The most important first step to take is to frame your business question as clearly as possible, and then apply the appropriate biometric tool. In short, the research question should always dictate the tool.

Understanding the ‘business impact’ of your business question can help you get a better understanding of how your experimental design can help you reach your goal. Using advanced biometric research to gain insights into your different website designs can help you answer questions such as conversion rates and interaction rates with elements on your website.

When deciding what kind of study will give you the most conclusive answer to your business question, it is important to choose the right hardware and biometric tool that will fit your study best. Below we outline the most popular modalities for web-based UX & usability testing as well as a couple of additional tools to complement your biometric findings. We will go into more detail further below after a brief introduction of the hardware modalities that are generally used for UX & usability testing.

Biometric Tools for web-based UX & usability testing

How to use Eye Tracking in usability web testing

Since web-based products are exclusively screen-based, eye tracking is an almost unavoidable tool for this kind of test. Below are some valuable metrics for eye tracking that can give you some useful information on how users use your website.

Heat Maps: Eye tracking heat maps are color-coded visualizations of where the audience is looking at a given point in time. Visual attention is aggregated across the entire audience and displayed as “heat” overlaid on a stimulus. This heat can help identify what areas of the content draw the most visual attention, and what areas or elements may go missed entirely.

Areas of interest (AOI): AOI’s can be identified to determine how quickly specific elements are noticed/which go missed and how long people spend on different elements.

Navigation paths: Navigation paths outline, on average, the visual navigation of the audience through the stimulus.

Eye Tracking

The Complete Pocket Guide

• 32 pages of comprehensive eye tracking material
• Valuable eye tracking research insights (with examples)
• Learn how to take your research to the next level
  

Download Free Guide

How to use Facial Expression Analysis (FEA) in UX

Whereas eye tracking can tell you where and for how long a respondent is looking at a screen, facial expression analysis can tell how they likely feel about it.

Facial expressions are often discrete, fleeting events. They are also heavily context-dependent – you might not smile at a website (unless the content is funny), but you might be more likely to furrow your brow if a site is frustrating or confusing.

Facial expressions are reported as a percentage time during a piece of content or a segment within that content when facial expressions are detected above a certain threshold.

Greater time expressing brow furrow may suggest that people are confused by the content, concentrating, or struggling with the assigned task.

How to use GSR or Electrodermal Activity (EDA) in UX

An EDA “peak” is a biological indicator that something relevant happened at that moment – an emotional connection was made. EDA Peaks per Minute indicate, on a rate basis, how many emotional events occurred every 60 seconds. The higher the number, the more of an emotional response the users were having while interacting with the content. Since websites can be fairly neutral stimuli in and of themselves, there are certainly going to be studies where EDA might not yield much information, so refer back to your research question to determine whether this tool can be of use for you.

For example, there will probably be strong EDA peaks when evaluating a user’s response to inflammatory content on social media, or for assessing the magnitude of frustration with a design when given a specific task to accomplish (paired with FEA data). However, if you are employing free-form navigation while participants are casually surfing two different designs, you might not see a difference.

Usability testing with Electromyography and Facial Electromyography (EMG & fEMG)

While this modality is primarily used in device testing, ergonomics, and human factors, it has its uses in web-based testing. EMG measures the electrical activity of muscles in respondents. If attached to facial muscles (fEMG) it can act as another way of measuring valence (positive or negative reactions) in your respondents and is sometimes preferred over webcam-based FEA due to its sensitivity (at the cost, however, of being more invasive). EMG in more product, device, or environment-based usability testing can be used to measure grip, stress, gait, posture, etc., which can be valuable measuring points, given the aim of your experiment.

Additional tools for UX & Usability Testing

To make sure that you are getting as accurate feedback as possible, several non-biometric self-reporting options are available to use to strengthen your findings.

Open-ended question survey: Ask respondents to elaborate on their user experiences. This can also help you conclude more concisely on the respondent’s biometric data. Questions such as; “How did the task I set you make you feel as you were performing it?” will often lead your respondent to give an answer that should be following with the biometric results that the person gave.

Scaling questions: Ask your respondents to rate a specific task from 1-10 (or 1-100). Questions such as; “on a scale from one to ten, how difficult was it to navigate to/submit/log onto X or Y?” these types of questions will give you a good insight into how their physical and psychophysical responses correlate with their answers.

Behavioral Measures: Set up other measurement points to concretize the results gathered from your respondents is always beneficial. You can establish objective annotations such as live-markers, “clicks used” that you mark as a researcher while a participant is performing a task.

Step by Step Guide on Advanced UX and Usability Website testing ( 4 Steps)

Now that you have a brief overview of the possibilities of getting data that gives insights into emotional and practical UX & usability tools, setting up your research to gather a holistic view of the user’s experience requires some well thought-out processes.

• Step 1: Setting up your study respondent pool
• Step 2:Designing your Study
• Step 3:Refining your hypothesis
• Step 4: Aligning your hypothesis with your deliverables

Step 1: Setting up your study respondent pool:

You have your research question and the appropriate modalities you want to use to answer your business question, so now it is time to find your respondents.

Clarifying the respondent group characteristics for your experiment is central to an approved study design. This is important because it is necessary to exclude side effects that could alter the outcomes of your data collection process. When designing your respondent pool, you have to make sure that there is coherence between age, gender, education level, income, etc. Aspects such as life experience and health issues must also be included in your decision process. Simply put, a website aimed at catering to a specific demographic must be UX tested by that same demographic.

There is a consensus that 5-7 participants are the perfect amount for website UX testing when not using biometric equipment. 5-7 participants will find between 75-95% of usability problems present on a website design, so adding more participants will usually just mean you will be wasting your time. It is better to conduct a succession of smaller experiments, with groups of 5, aimed to target the entire website UX flow.

• The first group finds the problems.
• The second group redesigns the issues.
• The third group tests the redesign again.

That way, you have only used 15 (to 21) participants, but you have covered your entire design process.

This consensus does not stand when using biometric measuring tools. The recommended participant pool when conducting eye tracking studies is 30 and above. This is because the versatility of biosensor studies allows for a wider breadth of study objectives to be explored beyond simply finding sticking points on a website design, and given the granularity of the data from biosensor sources, you will need more participants to derive any usable insights from what your testing group is doing.

Check out: Biometric Research: Respondent Protocols – How to Create the Ideal Experimental Setup

It is important to remember that not all respondents will give usable feedback, which is why it is a good idea to add 25% more participants than you think you would need. This is also true for the other modalities mentioned above – more is generally better when it comes to biometric research.

Potential Business Use Cases for Website UX Testing

Depending on your business question, there are several ways to practically go about answering it. These are some of the popular methodologies for UX testing:

– Site Navigability: Evaluate how website users navigate the different pages both naturally and with specific tasks.

– Website Comparisons (A/B test): Compare users’ experience across the website designs. Re-exposure to forced stimuli can be used to provide more granular insights.

– Biometric Informed In-Depth Interviews: Allow users to naturally surf while interviewers assess live biometric feed to inform more personal questioning during in-depth interviews.

Step 2: Designing your Study – Examples of good and less good study designs

When setting up your study there are several things to keep in mind. Keeping your study design simple, with few differences to the individual website designs is generally the better way to go. This allows for A/B testing, which is the best way to make sure that priming will not happen.

Good study design example:

As mentioned earlier in this article, having a good and clearly defined study question is very important. An example of a good question could be: “Which website layout makes the path to purchase easier?” This question is good because it lays the foundation for a very concise and simple study design. A design that could look like this:

2 versions of the website, with 2-3 specific differences in layout or architecture
A/B test design means that respondents see either one prototype or the other
Participants are given the same task, to follow through on a shopping journey and purchasing decision

So, why is this a good study design? First of all, having a between-subjects (A/B test) design prevents any priming effects from participants seeing both designs. Controlling for a task, especially with a specific outcome or goal in mind, allows the researcher to directly compare the user journey between website versions. This keeps the number of variables small and it allows the researcher to isolate specific ways to optimize the website.

Narrowing down the stimuli (or what you are exposing your user to) for A/B testing, based on psychophysical reactions, helps you create a positive experience with your product or website among other behavioral observations that can be recorded for quantifiable data.

Less Good Study Design example:

To reiterate the difference between good and less good study design, here is an example of one that does not set the foundation for a concise and simple study.

Again, any study design starts with a question. This time we could ask: “How do users respond to each website layout?” At first glance, this seems like a fine study question to ask, but it comes with significant flaws when followed up like this:

2 different pages with many differing variables across layout and architecture
Respondents see Prototype A, followed by Prototype B, in that order.

Why does a study design like this need improvement? Firstly, having many differences between the two prototypes makes it difficult to isolate what elements are good or bad for the user experience. Secondly, having participants see the same order of websites can introduce priming and other effects on the data.

Read more about: What is Researcher Bias? (And How to Defeat it)

Step 3: Refining your hypothesis – what are you actually testing

When you have decided on and confirmed the study design, you can start solidifying more specific hypotheses.

Using the good study design question and your study design framework from earlier “Which website layout makes the path-to-purchase easier?” you can start to formulate some hypotheses that you would like to have confirmed or disproven. In this case, some hypotheses could look like this:

A site design that makes relevant links most visible will make the purchase process easier.

Users will show prolonged engagement with a more effective website layout – as people get bored or disengaged with an inefficient design, engagement will decrease over time.

An inefficient site design will produce more pain points or points of frustration for the user.

The longer the customer journey and the more interactions the user has with the page, the most frustrating the path-to-purchase.

Step 4: Aligning your hypothesis with your deliverables – What metrics am I measuring?

It’s important not only to identify the modality, but also specific metrics you intend to use to answer your business questions.

Using the modalities described at the beginning of this article, we can apply them to the hypotheses posited in the above section.

	Eye-Tracking: A site design that makes relevant links most visible will make the purchase process easier. Method/Metrics Solution: 1-3 Areas Of Interest drawn on features relevant to moving the customer journey forward on both websites. Heatmaps to help identify alternate paths.

	GSR/EDA: Users will show prolonged engagement with a more effective website layout – as people get bored or disengaged with an inefficient design, engagement will decrease over time. Method/Metrics Solution: GSR Peaks pr. Minute for websites A versus B.

	Facial Expression Analysis: An inefficient site design will produce more pain points or points of frustration for the user. Method/Metrics Solution: Amount of time spent in Brow Furrow, for websites A versus B as an indicator of negative valence.

	Behavioral Measures: The longer the customer journey, and the more interactions the user has with the page, the most frustrating the path-to-purchase. Method/Metrics Solution: Track numbers of clicks and how much time was spent on completing each task.

 

iMotions setups to execute your study

Setups for website UX testing can take several forms. Depending on the scope of your research, this can be more or less hardware heavy, and it can also impact how your study will be set up using iMotions. If you are curious about what possibilities for UX and usability testing setups we can offer at iMotions, please reach out to us and we will help you get the best solution for your research.

Remote UX and Usability Testing

The latest feature from iMotions is our Online Data Collection tool. It offers the ability for you to collect data online as well as in the lab, allowing users to interact with your website from home, no matter where they might be in the world. This allows you to organize participant testing without needing people to physically show up in the lab while simultaneously opening up the entire planet in your search for participants.

---