Do You Get Reliable Customer Feedback from Mystery Shoppers?

Market research trends are always changing, but the trend for mystery shopping – paying people to pretend to be customers and using their feedback, has especially fluctuated in recent years.

You want to know what customers think of your organization. That’s obvious — if you weren’t interested in customer opinion, you wouldn’t be reading this.

Market research trends are always changing, but the trend for mystery shopping has especially fluctuated in recent years.

Despite what detractors say, many companies still invest a lot of money, time and energy in mystery shopper programs, paying people to pretend to be customers and then using their feedback to guide business decisions.

Is your organization wasting precious resources on mystery shoppers?

Interested in trying customer satisfaction surveys?

Try Questback 14 days for free.

Send surveys and analyse reports

Get started in 5 minutes

No credit card required

Free 14-day trial

The origin of the mystery shopper

To be fair, mystery shopping grew from good intentions.

Before the internet existed, market researchers figured instead of spending vast amounts of time and money following up with real-life customers to collect their feedback; it would be easier and more effective to recruit, train and pay detail-oriented people to test a company’s customer services.

It was a cool idea. Researchers could instruct a mystery shopper to test precisely the services and processes the company wanted to investigate. And a mystery shopper, unlike a real customer, would bring a stopwatch with them to surreptitiously record how quickly the staff serving them responded. You could ask them for almost any detail you wanted.

But detailed data isn’t the only goal of customer experience research. There are some problems with mystery shopping.

#1 Very small sample size

is that by using mystery shoppers to get feedback about your business, you’re basing your decisions on data from a very small sample size. You can hire more mystery shoppers, of course, but that increases your costs without making much difference to the ratio of mystery shoppers to genuine customers.

#2 Unemotional questionnaires

A mystery shopper’s report is typically based on a very process-oriented questionnaire, often including several measurements of the time taken for individual steps in the process. So there are a lot of questions like:

Was your call answered within 3 rings?
Were you greeted within 30 seconds after entering the store?
Did your drink arrive within 3 minutes after you ordered at the bar?
Was your meal delivered to your home within 30 minutes of ordering online?
Did the person who served you offer at least 2 recommendations of other items you might like?

Less process-oriented questions are usually included, but these give you subjective measures, for example:

Was the area around the outside of the establishment clean and attractive?
Was it easy to find what you were looking for?
Did the staff make eye contact with you?
How knowledgeable was the person who served you?
How tasty and well-cooked was your meal?

These kinds of questions pay little if any attention to how the customer feels about the experience, which increasingly the deciding factor when it comes to your real customers and deepening engagement.

#3 They’re not real customers

Subjective scoring from someone who may not be an ideal match to your customer profile. You can select mystery shoppers based on demographics, psychographics and self-reported behaviors, but that doesn’t mean they’re truly representative of your real customers, and it doesn’t mean their opinions are your customers’ opinions.

Some market research agencies go so far as to instruct a mystery shopper to act out a role to suit your research needs, such as:

Pretend you don’t understand how the product or service works.
Pretend you’re worried about potentially harmful ingredients.
Pretend you’re angry with the company.

If you let that happen, well, then you’re getting your customer feedback from a person who doesn’t really exist, playing another person who isn’t really a customer.

That’s pretty absurdist:)

#4 They behave differently too

Even with careful mystery shopper selection and training, your mystery shopper won’t always behave like a typical customer.

For example, mystery shoppers usually have to provide an itemized receipt to claim a reimbursement of their expenses on a job. In an establishment where customers rarely ask for an itemized receipt for their transactions, the simple act of asking can make your mystery shopper stand out enough that the staff can guess the reason for their visit.

#5 Employees feel spied on

A lot of employees dislike mystery shopper initiatives because they feel they’re being “spied on” by the fake customer, or because they’re afraid a bad report or a negative assessment of their performance. Worrying about mystery shoppers can make employees stressed, mistrustful and less likely to perform well in their jobs. There are more employee-engaging ways to collect customer feedback and improve customer experience.

The solution: ask real shoppers!

The best people to tell you what it’s like to be your customer are your customers. Not someone who’s getting paid or reimbursed for being your customer, but someone who genuinely chose to spend their money with you.

New Market Research approaches build on cloud computing and mobile technology to make it easy to collect feedback from your customers. You can even invite them to share their opinions while they’re interacting with you, instead of waiting until they get home to fill out a questionnaire.

Mystery shoppers can help you check that processes are being followed, but inviting your real-life customers to complete a survey gives you a better way to measure customer satisfaction. This gives you a more holistic view of customer experience without the six problems of mystery shopping.

Your sample size can be as big as “every customer” or target a smaller customer segment.

Conclusion

Real customers may not have brought a stopwatch to time employees when they interact, but you can still ask how long they perceived the wait to be. And when you get their subjective opinion, it’s from a genuine customer whose opinions represent a real segment of your customer base. They behave like real customers because that’s what they are.

Speaking of feelings and behavior, collecting feedback from your customers makes everyone happier. Employees don’t dislike real customers the way they do mystery shoppers. And customers will be glad you care about their opinion enough to ask.

If you’d rather talk to real customers than fake ones, it’s time you made that move.