Blog | Survey Hacks: Are You Getting the Right Answers to All the Wrong Questions?
The value you get from your customer and employee surveys depends on the quality of the questions you ask.
You’ve worked hard on this research programme, but the insights you’re getting from it are distinctly underwhelming.
No big breakthroughs to report. No moment of revelation. No brilliant plan for competitive advantage.
You’re scratching your head to understand what went wrong. You followed a ton of "best practice” advice, you sent your questionnaires to as many people as possible to make sure your results were reliable and representative.
The problem is, you were asking the wrong questions the whole time. Not your fault; you didn’t know.
So what’s making your questions wrong? And how will you know when you’re asking the right questions?
Stop making assumptions
Here’s what ruins cut your insight more than anything else: untested assumptions about your survey and its recipients.
You’ve probably already heard that you should eliminate untested assumptions from your research method. Likewise for confounding variables. But let’s be pragmatic: in many research situations, you can’t eliminate them all completely. Instead of letting that hold you back, simply do your best to minimise their effect on your insight.
Evaluate the standard questions
The first assumption to be wary of is the idea that you should stick to expert-approved standardised questions, exactly like those every other business asks in surveys. It’s best practice, you’re told, but what does that really mean?
One standard customer experience research question that often causes confusion is, "How satisfied are you with the way your question or problem was resolved?” It’s confusing because it suggests the question or problem was resolved, while the customer may not feel that’s happened yet.
If a consumer contacts the support team about problems with broadband, for example, they may not feel their problem is fully resolved until an engineer visits to check for faults, or until their broadband performs as they expected. So they’ll answer the question neutrally, or abandon the survey, rather than give an answer they feel implies complete resolution.
From a business perspective, of course, what you want to know is whether the customer support staff are doing their job well and handling customer questions and problems correctly. You may have the greatest customer support in the universe, but if your customers are confused by that standard question, you’ll never know the truth.
The best practice is to do what works best for your organisation and your survey participants. Not to copy what someone on the outside says, but to analyse what the organisation needs and how you can achieve it.
That may mean creating your own questions to collect the right information in the right way. So, here’s how to write questions that lead to valuable insight.
Minimize question bias
Biased or leading questions create skewed data and inaccurate conclusions. Here’s a simple example.
- Wrong question: "How much did you enjoy this event?”
- Better question: "Did you enjoy this event?”
- Even better question: "How would you rate this event?”
It’s simple, and you’ve heard it before, but double-check the wording of your survey questions to make sure you’re not guilty of this basic error.
Emotive language is another cause of skewed responses. Take a look at these two questions:
- "Would you like to receive more information in printed formats?”
- "Would you like us to cut more trees and force you to wait longer to receive stuff that could’ve been delivered instantly via email?”
Both ask the same thing, but responses to the second question will be driven by the emotional power of words like "cut” and "force”.
However, the second question isn’t all bad — it also provides a reminder of the context and consequences, which helps you collect a more considered response. In this instance, your survey respondents may genuinely prefer to receive information via email once they’ve given the consequences some thought.
Minimize frustration and ambiguity
If you ask multiple choice questions, beware of leaving out a viable answer. When your survey respondents can’t find an answer that suits them, they have only two choices:
- Abandon the survey.
- Give an untrue or inaccurate answer.
To get the most detailed insight, include an answer option such as "Other”, or "None of the above”. Invite respondents who select this answer to leave a comment explaining what their "other” is, and if the results show a popular "other” then add it to the available answers in your survey. This may sound obvious to you, but you’d be amazed how often survey creators forget it!
You want specific answers from your respondents, but sometimes the honest answer is that they can’t remember or simply don’t want to tell you. Allowing them to answer "I don’t know” or "I prefer not to answer this question” gives you a truer picture of the situation.
The right questions for your organisation to ask are the questions that mean something for the future. To know if you're asking the right questions of your survey participants, ask yourself a few questions too:
- Do these questions test a hypothesis about your organisation, strategy or process? What does each question seek to measure or identify? What will you compare the answers to, and what idea will that allow you to confirm or refute?
- Are these questions linked to business outcomes? Start with organisational strategy and work backward to find the right questions. So if your desired outcome is increased profits, does it make sense to ask customers how loyal they are? If your organisation wants to improve sales metrics, do you need to ask employees about their work-life balance?
- Can you plan action based on the answers to these questions? What will you do if, for example, Net Promoter Score is low?
- What will you do if the response rate is low on an optional question? Will you make the question mandatory or will that only increase the survey abandonment rate? Will you test variations to see which one gets the highest response rate?
As you can see, you’ll need to interrogate your objectives before you interrogate your employees and customers.
Your results will speak clearly and persuasively for themselves.