10 Steps to Run a Research Study
When it comes to human behavior research, study design and execution don’t require superhero powers. You can do it! Just keep a couple of things in mind and you will be heading for research happiness in no time.
The more care you invest right from the outset of your study, the smoother the overall ride will be. Follow our 10 step guide to make your study a true winner. Guaranteed.
STEP 1: Define your research question
Start with the obvious: Define the actual question of interest. What is it you’re trying to achieve with your study? Which outcomes do you expect?
Usually, the best research questions are breaking new ground – they address something that has not been answered before. At the same time, they contain references to the existing state of knowledge – they pick up the threads that were dropped by already existing studies in the field.
Research questions can be theoretical and abstract. The more precise and quantifiable they are phrased, the easier it will be to measure and analyze the effects, while at the same time controlling for unwanted side effects.
In a perfect world, research questions comprise a cause and an effect:
Surely, “how to improve an ad” seems to be a relevant question. However, a more quantifiable, measurable alternative might be something like “how does the actor’s gender in the ad (the cause) affect the viewer’s attention and engagement(the effect)?”. You get the idea.
STEP 2: Specify stimuli and sensors
Let’s take our ad example again: Here, you might want to show different versions of the advertisement to the target audience – one with a male, one with a female actor. How should you measure “action and engagement” exactly?
The good news is, there’s several ways to go about it.
Let’s see. For starters, you could utilize eye tracking software to measure where and how long people look, or how many saccades and fixations they perform while watching the video.
In addition, GSR measures would certainly be helpful as they reveal great deal about people’s emotional arousal. Offering a deep glimpse into emotional valence, the recording of facial expressions could compliment your insights.
Finally, you certainly can’t go wrong with EEG – it is an excellent way to peek into people’s heads and assess engagement and attention levels based on their brain activity.
STEP 3: Design your study
Study design requires careful consideration as sequence effects can alter the results. Imagine a tasting study where respondents eat mixed pickles just minutes before tasting the new ultimate 10-layer white chocolate cheesecake (and let’s assume they don’t fall into a sugar coma).
It’s rather likely the respondents will rate the cake significantly lower, just because the mixed pickles were messing with their tasting sensors.
To counteract sequential effects, it is always good advice to randomize stimuli and present them in varying sequences such that the combination of stimuli across your respondent group is never the same. This simple strategy effectively minimizes the risk that one stimulus affects the rating of others as the sequence is different across people.
STEP 4: Implement your study
While step 3 was purely theoretical, it’s now all about coding up the paradigm in software. Gladly, you don’t need hacking skills to code up your experiment.
There’s terrific software available allowing you to drag and drop stimuli, switch devices on and off and set up the entire respondent management in the blink of an eye. Ask our expert team at iMotions about how our research software can make your lab life a lot easier.
STEP 5: Write a test protocol
Running a study implicates thinking about the general workflow. This includes the overall “respondent handling” from seemingly obvious things like opening the door and saying “hi” to consenting and instructing before the start, debriefing and paying after the study.
Consenting is a noteworthy step as respondents have to be informed that they can terminate at any time and participate at free will. Universities generally require the consenting to follow established routines and protocols (as specified by the ethics committee of the university, for example).
If you’re in doubt about who to contact, ask colleagues in the organizational management team for assistance.
STEP 6: Run a pilot test
Counter-intuitively, this step doesn’t involve testing pilots in flight simulators (although that certainly would be fun). Rather, pilot tests are small-scale trials where a few respondents are tested to see if the study works properly.
Ideally, respondents should have the same background as the people you want to test in large scale afterwards (comparable age, gender, socioeconomic status, ethnicity etc.). Interview pilot respondents to learn about any problems or struggles they encountered with instructions, the stimulus set or the study flow.
STEP 7: Optimize your study
In this stage, implement any changes based on your collection of ideas and suggestions during the pilot phase. Spend time on this as it is the last chance to apply changes before rolling out the study.
There’s nothing more annoying than skipping one issue and ending up with data from 100 respondents that you cannot use (you definitely can make better use of your time).
STEP 8: Collect data in the field
Let’s start the fun – it’s time to collect data! Rule of thumb: Don’t rush it. Handle every respondent with the same care, respect, and positive attitude. Keep in mind that your data will be as good as your respondents’ motivation.
Providing instructions and explaining biosensor technologies in a friendly way will massively increase your respondents’ willingness to spend time and effort to make your data shine.
Collecting data includes following of guidelines and standards for sensor setup and testing
- Sensors should always be placed and configured the very same way across the respondent pool to make the analysis easier and straightforward
- Have an eye for your equipment during data collection, and clean up afterwards – don’t leave the lab behind in chaos
STEP 9: Analyze
Be prepared to live on instant Ramen and pizza deliveries to your office for a while, because chances are you will spend a significant amount of time pre-processing and analyzing your data.
Selecting the appropriate statistical procedure needs careful consideration. If your data fulfills the statistical requirements (normal distribution, minimum number of respondents etc.), you might run a t-test, an ANOVA, or even non-parametric statistics.
Luckily, today’s software renders this task quite simple, generating stats and plots with just a few clicks.
STEP 10: Write the report
There is just one last thing to cross off your list before you can pop that bottle and celebrate: You need to write up your findings.
Your report should comprise an introduction and a thorough description of all relevant elements of the study (biosensors, stimuli, study procedure etc.) as well as statistical results with expressive graphics and figures.
However, simply describing statistics is not the end of the road – numerical results require interpretation along with concrete implications and ideas for future research.
On a side note: It goes without saying that your report should be free of typos and grammatical errors (even if you happen to be a linguistic genius – outsmart your “blind eye” and have someone proofread your report).
Excited to become a research superhero? Contact our experts at iMotions to receive more advice on multimodal study design and execution utilizing eye tracking software, facial expression analysis, GSR, EEG, and more.