If you happen to know your way around in psychology research, chances are you come across the term questionnaire on a daily basis.

Maybe you have already developed a number of questionnaires during your research career and know the ins and outs of good questionnaire design rather well, or you are just about to dive into your first scientific study for which you need to come up with smart questions to back up your research hypothesis.

Whether you are a professional or newbie to the field – when it comes to designing questionnaires, asking the right questions in the right ways is crucial to collect exactly the data you need.

So really, what are questionnaires?

Questionnaires are a set of written questions with a number of predefined answers to gather standardized information about the opinions, preferences, experiences, intentions, and behavior of a large group of respondents, typically devised for the purposes of a scientific study.

While questionnaires provide a comparatively cheap, prompt, and efficient means of obtaining large amounts of information, questionnaire design is a multistage process that requires your distinct attention to a number of aspects at the same time to gather the information you seek.  Why exactly is that?


The power of asking the right questions

Depending on the kind of information you aim to acquire, questions need to be asked in varying degrees of detail and in specific ways.

Given the same topic, it’s rather likely that different researchers will come up with different questionnaires that vary widely in their choice of questions, a line of questioning, use of open-ended questions, and length.

Question(naire) everything – what makes a good questionnaire?

Basically, well-designed questionnaires are highly structured to allow the same types of information to be collected from a large number of respondents in the same way and for data to be analyzed quantitatively.

open and closed questionnaire formats

Now how should you go about it? Clearly, you wouldn’t want to just try a shot in the dark and start writing questions haphazardly. Despite the relative ease of conducting questionnaires, designing them is definitely no fishing expedition.

Before you know how to exactly phrase your questions, you need to wrap your head around a number of other things first in order develop a reasonable set of questions (and possible answers) suitable to deliver the insights you need.

Unfortunately, there are no hard and fast rules about questionnaire design. No worries, we got you covered. Our roadmap for questionnaire design will help you develop spot-on questions that get it all right!

5 steps to good questionnaire design

#1: Identify your research aims and the goal of your questionnaire

What kind of information do you want to gather with your questionnaire? What is your main objective?

Ideally, there are already existing questionnaires with published results on the validity and statistical evaluation of all tested questions that you can use (or steal a couple of ideas from).

While this is quite common in academic research, it might be rather hard to reuse existing questionnaires for commercial applications. In this case, you might have to scan papers and internal reports for key metrics of interest and create a questionnaire that specifically addresses these aspects.

#2: Define your target respondents

Clearly, you can’t test everyone – it’s rather plausible that there have to be certain restrictions with respect to the target audience of your questionnaire. The selection of groups is a key factor for maximizing the outcomes of your study.

To put it another way: You can run multiple questionnaire sessions over a longer period of time with a single group (longitudinal design), or you can present your questionnaire once to two or more groups (cross-sectional design).

While the former allows you to analyze how the questionnaire results of the group change over time, the latter delivers insights into differences among groups.

#3: Develop questions

Smart questions are the linchpin of every questionnaire. To make them work, they have to be phrased in a way that prevents any misunderstandings or ambiguities.

Quite frankly, it’s a lost cause trying to analyze data from a questionnaire where people seemed to be confused either mixing things up, selecting incorrect answers or not being able to read or understand the questions at all.

It makes a significant difference whether you want to hand a questionnaire to children, adults or elderly respondents. You certainly do well to consider the cognitive, attentional, and sensory competencies of your target group – handing out long questionnaires with a huge amount of questions in small letter print and complicated phrasing might be information (and visual) overkill for any respondent group.

#4: Choose your question type

There’s a wide variety in how to phrase questions. In explorative questionnaires, you will find mainly open questions, where respondents can fill in any answer (this makes sense whenever you try to gain an understanding of the topics associated with your research question).

By contrast, quantitative questionnaires primarily include closed-questions, which have been predefined by the researcher either in form of multiple choice answers or rating scales.

Here’s one example:

Open question: “What did you like about the webinar?”

Closed question:“The webinar was useful.”

[  ] strongly agree

[  ] Agree

[  ] Cannot decide

[  ] Disagree

[  ] Strongly disagree

You get the drift.

As is usually the case, both types of questions have benefits and drawbacks that are worth considering in order to come up with a solid questionnaire design that does the trick for you.

Besides open and closed-format questions, there are several other types of question you can sprinkle into your questionnaire. Stay tuned for next week’s post as we will shed light on the different categories and their characteristics so you know exactly when and how to use them.

#5:  Design question sequence and overall layout

After optimizing each question separately it is time to improve the overall flow and layout of the questionnaire.

Are there transitions from one question to the next? Are follow-up questions placed correctly? Are skip-rules implemented so that respondents can skip questions that do not apply to them?

#6: Run a pilot

This stage is crucial for evaluation and optimization purposes. Any questionnaire should be handed to a representative sample of your target audience before you push it to the masses.

During piloting, you can identify issues in readability and understanding, in phrasing and overall arrangement. Pilot respondents should be monitored and interviewed closely.

You certainly want to avoid any inappropriate or problematic questions. Also, keep in mind to evaluate your pilot data statistically to make sure that the analytic procedures of interest truly can be applied to the data.

Questionnaires and biometrics - the perfect match for piloting

Make sure not to miss next week’s blog post to get the inside scoop on the different types of questions you can use to boost your questionnaire.

In the meanwhile, have a look at how surprisingly easy it is to optimize surveys or reach out to our experts at iMotions to receive more information on questionnaire design. We’re happy to help!