Survey hacks: are you getting the right answers to all the wrong questions?
The value you get from your customer, employee or cpx research surveys depends on the quality of the questions you ask.
You’ve worked hard on this research program, but the insights you’re getting from it are distinctly underwhelming.
No significant breakthroughs to report. No moment of revelation. No brilliant plan for competitive advantage.
You’re scratching your head to understand what went wrong. You followed a ton of “best practice” advice; you sent your questionnaires to as many people as possible to ensure your results were reliable and representative.
The problem is, you were asking the wrong questions the whole time. Not your fault; you didn’t know.
So what’s making your questions wrong? And how will you know when you’re asking the right questions?
Try Questback 14 days for free.
Stop making assumptions
Here are what ruins your insight more than anything else: untested assumptions about your survey and its recipients.
You’ve probably already heard that you should eliminate untested assumptions from your research method, likewise for confounding variables. But let’s be pragmatic: in many research situations, you can’t eliminate them all completely. Instead of letting that hold you back, simply do your best to minimize their effect on your insight.
Evaluate the standard questions
The first assumption to be wary of is the idea that you should stick to expert-approved standardized questions, exactly like those every other business asks in surveys. It’s best practice, you’re told, but what does that mean?
One standard customer experience research question that often confuses is, “How satisfied are you with the way your question or problem was resolved?” It’s confusing because it suggests that the question or problem was resolved, while the customer may not feel it has happened yet.
If a consumer contacts the support team about problems with broadband, for example, they may not feel their situation is fully resolved until an engineer visits to check for faults or until their broadband performs as they expected. So they’ll answer the question
neutrally, or abandon the survey, rather than give an answer they feel implies complete resolution.
From a business perspective, of course, what you want to know is whether the customer support staff are doing their job well and handling customer questions and problems correctly. You may have the most outstanding customer support in the universe, but if your customers are confused by that standard question, you’ll never know the truth.
The best practice is to do what works best for your organization and your survey participants. Not to copy what someone on the outside says, but to analyze what the organization needs and how you can achieve it.
That may mean creating your own questions to collect the correct information in the right way. So, here’s how to write queries that lead to valuable insight.
Minimize question bias
Biased or leading questions create skewed data and inaccurate conclusions. Here’s a simple example.
- Wrong question: “How much did you enjoy this event?”
- Better question: “Did you enjoy this event?”
- Even better question: “How would you rate this event?”
It’s simple, and you’ve heard it before, but double-check the wording of your survey questions to make sure you’re not guilty of this basic error.
Emotive language is another cause of skewed responses. Take a look at these two questions:
- “Would you like to receive more information in printed formats?”
- “Would you like us to cut more trees and force you to wait longer to receive stuff that could’ve been delivered instantly via email?”
Both ask the same thing, but responses to the second question will be driven by the emotional power of words like “cut” and “force”.
However, the second question isn’t all wrong — it also provides a reminder of the context and consequences, which helps you collect a more considered response. In this instance, your survey respondents may genuinely prefer to receive information via email once they’ve given the consequences some thought.
Minimize frustration and ambiguity
If you ask multiple-choice questions, beware of leaving out a viable answer. When your survey respondents can’t find a solution that suits them, they have only two choices:
- Abandon the survey.
- Give an incorrect or inaccurate answer.
To get the most detailed insight, include an answer option such as “Other” or “None of the above”. Invite respondents who select this answer to leave a comment explaining what their “other” is, and if the results show a popular “other”, then add it to the available answers in your survey. This may sound obvious to you, but you’d be amazed how often survey creators forget it!
You want specific answers from your respondents, but sometimes the honest answer is that they can’t remember or simply don’t want to tell you. Allowing them to answer “I don’t know” or “I prefer not to answer this question” gives you a more accurate picture of the situation.
Anticipate results
The right questions for your organization to ask are the questions that mean something for the future. To know if you’re asking the right questions of your survey participants, ask yourself a few questions too:
- Do these questions test a hypothesis about your organization, strategy or process? What does each question seek to measure or identify? What will you compare the answers to, and what idea will that allow you to confirm or refute?
- Are these questions linked to business outcomes? Start with the organizational strategy and work backwards to find the right questions. So if your desired outcome is increased profits, does it make sense to ask customers how loyal they are? If your organization wants to improve sales metrics, do you need to ask employees about their work-life balance?
- Can you plan action based on the answers to these questions? What will you do if, for example, Net Promoter Score is low?
- What will you do if the response rate is low on an optional question? Will you make the question mandatory, or will that only increase the survey abandonment rate? Will you test variations to see which one gets the highest response rate?
As you can see, you’ll need to interrogate your objectives before you interrogate your employees and customers.
Your results will speak clearly and persuasively for themselves.
Customer Journey – Transform your CX
Do you want to create a superb customer journey? Gain a competitive advantage, enjoy greater profitability and deliver a superior customer experience? Download our new whitepaper now!