If you want to find out something about a person, you’d usually just ask them. If you want to ask a few questions for a group of people, maybe you’d get together as a group. If you want to do research on the answers, you’d give them a questionnaire.

Questionnaires are a crucial part of research. There are many other tools that are used to find out about how people think, feel, and act, but the act of asking remains central to finding out what people explicitly think.

While questionnaires have likely been used for hundreds of years [1], the first recorded instance arose from the result of the work of Adolphe Quetelet, a French polymath, in 1835. He was interested in applying the same rigorous methodologies applied to natural science as to the humanities. By recording – through questionnaires (well, technically surveys) – the physical characteristics of soldiers, he essentially invented the field of sociology. This all goes to say: questionnaires can be powerful things.

Despite the lengthy and illustrious history of questionnaires, they are still not used necessarily in the right way. There remain various ways to carry out such work, and many pitfalls abound.

Below, we will define what a questionnaire actually is (including what separates it from surveys), and provide a guide to making one in the best possible way.

What are questionnaires?

Questionnaires are a set of written questions designed to gather standardized information about the opinions, preferences, experiences, intentions, and behavior of individuals, and can be devised for the purposes of a scientific study. Traditionally, they have been said to contrast with surveys in the sense that they do not collect mass data for further analysis, however the terms are largely used interchangeably these days (and many research studies also use them together).

While questionnaires provide a comparatively cheap, prompt, and efficient means of obtaining large amounts of information, questionnaire design is a multistage process that requires attention to a number of aspects at the same time to gather the information you seek. Why exactly is that?

Depending on the kind of information you aim to acquire, questions need to be asked in varying degrees of detail and in specific ways.

Given the same topic, it’s rather likely that different researchers will come up with different questionnaires that vary widely in their choice of questions, a line of questioning, use of open-ended questions, and length.

“experimental

Question everything – what makes a good questionnaire?

Basically, well-designed questionnaires are highly structured to allow the same types of information to be collected from a large number of respondents in the same way and for data to be analyzed quantitatively.

open and closed questionnaire formats

Now how should you go about it? Clearly, you wouldn’t want to just try a shot in the dark and start writing questions haphazardly. Despite the relative ease of conducting questionnaires, designing them is definitely no fishing expedition.

Before you know how to exactly phrase your questions, you need to wrap your head around a number of other things first in order to develop a reasonable set of questions suitable to deliver the information you need.

Six steps to good questionnaire design

#1: Identify your research aims and the goal of your questionnaire

What kind of information do you want to gather with your questionnaire? What is your main objective?

Ideally, there are already existing questionnaires with published results on the validity and statistical evaluation of all tested questions that you can use (or borrow a couple of ideas from).

While this is quite common in academic research, it might be rather hard to reuse existing questionnaires for commercial applications. In this case, you might have to scan papers and internal reports for key metrics of interest and create a questionnaire that specifically addresses these aspects.

#2: Define your target respondents

Clearly, you can’t test everyone – it’s rather plausible that there have to be certain restrictions with respect to the target audience of your questionnaire. The selection of groups is a key factor for maximizing the outcomes of your study.

To put it another way: You can run multiple questionnaire sessions over a longer period of time with a single group (longitudinal design), or you can present your questionnaire once to two or more groups (cross-sectional design).

While the former allows you to analyze how the questionnaire results of the group change over time, the latter delivers insights into differences among groups.

#3: Develop questions

Smart questions are the cornerstone of every questionnaire. To make them work, they have to be phrased in a way that prevents any misunderstandings or ambiguities.

Quite frankly, it’s a lost cause trying to analyze data from a questionnaire where people have mixed things up, selected incorrect answers or haven’t been able to read or understand the questions at all.

It makes a significant difference whether you want to hand a questionnaire to children, adults, or maybe even elderly respondents. It’s important to consider the cognitive, attentional, and sensory competencies of your target group – handing out long questionnaires with a huge amount of questions in small letter print and complicated phrasing might be information (and visual) overload for any respondent group.

Additionally, remember to avoid jargon or technical language – the text needs to be fully understood by anyone completing the questionnaire.

#4: Choose your question type

There’s a wide variety in how to phrase questions. In explorative questionnaires, you will find mainly open questions, where respondents can fill in any answer (this makes sense whenever you try to gain an understanding of the topics associated with your research question).

By contrast, quantitative questionnaires primarily include closed-questions, which have been predefined by the researcher either in form of multiple choice answers or rating scales (such as the Likert scale).

Here’s one example:

Open question:

“What did you like about the webinar?”

Closed question:

“The webinar was useful.”

[  ] Strongly agree

[  ] Agree

[  ] Cannot decide

[  ] Disagree

[  ] Strongly disagree

As is usually the case, both types of questions have benefits and drawbacks that are worth considering in order to come up with a solid questionnaire design that does the trick for you.

Besides open and closed-format questions, there are several other types of questions that you can use in your questionnaire.

#5:  Design question sequence and overall layout

After optimizing each question separately it is time to improve the overall flow and layout of the questionnaire.

Are there transitions from one question to the next? Are follow-up questions placed correctly? Are skip-rules implemented (if needed) so that respondents can skip questions that do not apply to them?

#6: Run a pilot

This stage is crucial for evaluation and optimization purposes. Any questionnaire should be handed to a representative sample of your target audience before you push it to the masses.

During piloting, you can identify issues in readability and understanding, in phrasing and overall arrangement. Pilot respondents should be monitored and interviewed closely.

You certainly want to avoid any inappropriate or problematic questions. Also, keep in mind to evaluate your pilot data statistically to make sure that the analytic procedures of interest truly can be applied to the data.

I hope this post helps you set out your questionnaire or survey design. If you’d like to learn more about the fundamentals of experimental design, then download our free guide below.

“experimental

References

[1] Gault, R. (1907). A History of the Questionnaire Method of Research in Psychology. The Pedagogical Seminary, 14(3), 366-383. doi: 10.1080/08919402.1907.10532551