Making the right choices between open-ended and closed survey questions, allows you to get only the data that best serves the goal of your customer, employee or market research survey.
You already know the difference between an open-ended question and a closed question, right?
Open-ended questions are the kind that don’t set specific options for response. For example, «What’s your reason for reading this blog post? Type your answer in the box below.”
Closed questions are the kind you can answer with one word or number, or by choosing from preset responses. For example, «How often do you visit this website? Once a day, once a week, once a month or once a year?”
The difference matters because when you’re designing a survey to collect feedback from customers or employees, the type of question you use influences the type of answers you get and the insights you can gain.
Let’s start by looking at when, how and why you’d want to use open-ended questions.
#1: Allow an infinite number of possible answers
The big plus of an open-ended question is that you’re not placing any limits on the response. That means your survey respondents can tell you anything they feel is relevant and anything they want you to know. Closed questions, on the other hand, drastically limit the possible responses.
#2: Collect more detail
Open-ended questions give your respondents the freedom and space to answer in as much detail as they like, too. Extra detail really helps to qualify and clarify their responses, yielding more accurate information and actionable insight for you.
#3: Learn something you didn’t expect
All this freedom to give any answer, of any length and with any level of detail, means that you’ll sometimes discover something completely unique and unexpected among your survey responses. Whether it’s a process innovation that’ll save the company money, or a marketing concept with the potential to boost your brand, these unanticipated answers can be extremely valuable.
#4: Get adequate answers to complex issues
Sometimes an open-ended question is the only way to collect the answers you need. In a situation that requires contextualisation, complex description and explanation, a simple Yes/No or multiple-choice answer just won’t cut it. When you’re asking someone to explain a decision or report a problem, for example, open-ended questions tend to work best.
#5: Encourage creative answers and self-expression
Given room to express themselves freely, some respondents will surprise you with their eloquence and creativity. An open-ended question frees respondents to convey their feedback and ideas to you in their own voices. You may also receive survey answers in unexpected formats, such as poetry, or a hyperlink to a blog post your respondent wrote on the topic in question.
#6: Understand how your respondents think
Free-form written answers reveal a great deal about the workings of the respondent’s mind. From the essential logic of their reasoning and the steps in their thinking process to their language choices and frame of reference, there’s a huge amount you can learn from reading their thoughts in their own words.
#7: Ask without knowing
The big problem with closed questions is that to design them into your survey, you’ll need to know roughly what answers you expect. If you’re testing a hypothesis, for example, it’s easy enough to come up with appropriate answer options that will support or refute it. But using open-ended questions lets you explore topics you don’t yet know enough about to form a hypothesis.
When to use closed questions?
So, open-ended questions offer a lot of advantages in the right situations. But not every situation is right for this type of question, and there will be times when you’re better off avoiding open-ended questions entirely. Here’s why.
#1: Keep analysis easy
Comparison, correlation and statistical analysis are much more difficult when every response is a freeform answer to an open-ended question. If you don’t have the time or the manpower to read every word of every answer, interpret them, and code them consistently for analysis, stick to closed questions unless there’s a good reason to go open-ended.
#2: Don’t torture your respondents
When faced with an open-ended question and a large blank space into which to insert their answer, respondents can feel intimidated or disheartened. Answering an open question takes more time, thought and effort than ticking a few radio buttons, and your survey respondents know it. Bear in mind that free-form written answers are easy for literate and articulate people, but some respondents may struggle more with this answer format than others.
#3: Get the answers you need
Open-ended questions hand over all control of the response to the respondent. If your question doesn’t make clear exactly what information you’re looking for, your respondents may give only vague answers, or misinterpret your question and focus their answer on a side topic you didn’t intend to explore. The ever present risk of open-ended questions is that the responses may be inadequate or irrelevant to your research needs — or the one sentence that matters may be buried in a long and meandering response.
#4: Balance your question types
If you’ve already hit your respondents with several open-ended questions in a survey, cut them some slack by keeping the survey short and making the remainder of your questions closed-ended. Respondents may enjoy the freedom to tell you exactly what they think, but they don’t want your survey to feel like an essay assignment.
#5: Open answers versus open questions
Sometimes it isn’t the question itself that matters so much as the answer options you provide. If you ask what sounds like a closed question, but allow free-form responses, you’ll still get additional information from some of your respondents. For example, a simple Yes/No answer can become «Yes, but…” or «No, and…” when the respondent has more to say on the topic.
Conclusion: Now what?
First, think. Every time you create a survey question, remember the advantages and disadvantages of making it open-ended. Know when you’ll need quantifiable answers for statistical analysis, and when you’ll benefit from giving your respondents greater freedom of self-expression.
Second, act. Send your surveys and analyze your responses. Share reports with everyone who needs them and plan your next steps.
Third, check. Compare your survey opening and completion rates to see if your new approach to open-ended questions has made any difference to your respondents’ behavior. Ask your data analysts if their job’s easier now that you use open and closed questions more appropriately. Evaluate the insight and competitive advantage your organization’s gained.