7 Pitfalls That Can Make or Break Your Survey
The good news upfront: You do not need to work wonders in order to come up with well-constructed survey questions. Following some common principles, it’s surprisingly simple to create a survey that is able to extract exactly the information you are looking for.
On the flipside, there are a few common pitfalls that can make or break your survey. For the untrained eye, they can be hard to spot at times.
To help you stay on track, we have bundled up 7 simple fixes that will do the trick.
Easy fix 1: Use short sentences
Keep it simple. Long, drawn out questions increase confusion and raise the risk of misinterpretation. To guard against this pitfall, keep sentences short and crisp – stick to sentences with one or two clauses and rephrase sentences containing more than three clauses.
Easy fix 2: Focus on one piece of information at a time
Don’t miss the forest for the trees. To lessen confusion and cognitive overload, ask for one piece of information at a time. Two or more simple questions are always easier to answer than one giant question pieced together by a whole set of questions targeting different topics. This is particularly troublesome if respondents have different views on the individual parts of your survey question.
Easy fix 3: Avoid (double) negatives
Use negatives only sparingly.
“I don’t like the new website layout.” Assuming you do find the new website much less user-friendly and, frankly, pretty much a mess -which answer should you pick? Yes? No? See, here’s the trouble.
Answering negative questions/statements can easily bring forth misunderstandings and, ergo, lead to invalid results – respondents might just be puzzled and select the wrong answer. Better avoid any confusion right from the outset and positively rephrase negative statements.
Double negatives are even trickier. Limit the cognitive burden. Do not expect respondents to work hard thinking about a question’s intent – do the thinking for them to get the information you seek.
Easy fix 4: Ask precisely
Poetic effort won’t lead nowhere if your questions fail to be concise and specific in space and time. To receive the information you are looking for, make sure to specify the frame of reference in order to reduce any uncertainty on the respondent’s part.
Easy fix 5: Don’t ask unnecessary details
Make sure you ask for the exact level of details. Why?
- If you omit to ask essential details, you might not be able to fulfil the purposes of your survey and end up with only half the information you need.
- If you ask too many unnecessary details, respondents will be less inclined to complete your questionnaire. This is particularly important for sensitive information such as personal financial matters or marital relationship issues.
Easy fix 6: Be sensitive
Obtaining truthful answers to sensitive questions can be a balancing act. Chances are respondents will reply to blunt questions like “Do you have a history of mental illness?” most likely with either no response at all or a negative response (“no”). To get genuine answers, go for less direct approaches. Try the following:
Easy fix 7: Minimize bias
People tend to answer questions in a way they perceive to be socially desired or expected by the questioner; often, they look for clues in the questions. To avoid these tendencies, screen your questionnaire with respect to how suggestive your questions truly are. Similar to a court-martial, modify items whenever your “inner lawyer” objects.
If you would like to dig deeper into survey design, have a look at this helpful tip sheet by the Harvard University Program on Survey Research or reach out to our experts at iMotions. We’re happy to help!