Blog | 5 reasons why mystery shopping isn’t enough to measure customer experience

Published February 01, 2016 by Alex Osbaldeston

Customer Experience
5 reasons why mystery shopping isn’t enough to measure customer experience

Times have changed - mystery shopping is no longer the most effective way of assessing employee integrity, customer experience and collecting 'real world' customer feedback, here is why.

Before the internet, collecting feedback directly from real-life customers was difficult. This meant it was simpler and less expensive for market researchers to recruit, train and pay detail-oriented people to work as mystery shoppers to test a company’s customer service levels. This has led mystery shopping to grow to be a billion dollar global industry, with well over a million assessments carried out every year in the UK alone. Demand surged during the recession, with 40% growth, as businesses tried to find ways to compete in the tough climate.

The advantages of mystery shopping

Where mystery shopping works particularly well is measuring adherence to process:

  • Was your call answered within 3 rings?
  • Were you greeted within 30 seconds after entering the store?
  • Did your drink arrive within 3 minutes after you ordered at the bar?

However process isn’t everything and doesn’t always impact the overall experience. For example, a large party ahead of you in a restaurant could mean you aren’t seated within the target time, but the manager makes up for it with complementary drinks, turning a negative into a positive. Additionally you have to rely on the mystery shopper’s subjective perceptions when rating areas such as ease of finding what you were looking for, the knowledge of the person who served you, or how tasty and well-cooked your meal was.

The move to real-time, continuous feedback

Today customers are increasingly vocal and keen to tell you what they think – and with social media and mobile devices, they have the channels and tools to deliver real-time, continuous feedback. So, many organisations today are more inclined to gather feedback from real customers to measure the customer experience. Here are five reasons why this makes perfect sense:

1. Get a more representative idea of what real customers think

While you can select mystery shoppers based on demographics, psychographics and self-reported behaviours, you can’t make them truly representative of your real customers. You will be relying on the mystery shoppers’ subjective feedback and you cannot be sure their opinions match those of your customers. Collecting feedback from real customers is obviously more likely to give you truly representative picture.

2. Customer feedback is more authentic

When you look at it, mystery shopping essentially means paying people to pretend to be your customers. They are often asked to act out a role to suit set research needs, created by the company itself. Real customers will rate you on the whole experience, from their point of view, rather than against criteria that you have had a hand in creating. Feedback will be deeper and more comprehensive and you can potentially pick up on issues that could be missed using mystery shoppers.

Additionally it’s difficult for a mystery shopper to always behave like a typical customer. For example, mystery shoppers usually have to provide an itemised receipt to claim a reimbursement of their expenses. If real customers rarely ask for an itemised receipt, then a mystery shopper will give him or herself away. Obviously this sort of bias is avoided if you decide to collect feedback from real customers.

3. Avoid staff negativity surrounding mystery shopper assessments

When staff find out that a mystery shopper will assess their performance, they often feel they are being secretly monitored and worry about the possibility of being punished if they receive a poor report. This can generate stress and mistrust among employees and affect job performance, either negatively or positively. In contrast, collecting real-time customer feedback through online surveys is more natural.

4. You can have a larger sample size when questioning real customers

If you are relying on mystery shoppers to get feedback, you’re probably basing decisions on data from a very small sample size because mystery shopping is expensive. If you are using feedback technology to quiz real customers instead your budget will stretch much further, giving a deeper insight from a wider pool of consumers.

5. Customer feedback can go beyond existing products and services

Customer feedback gives you more than a picture of how service levels are perceived around existing products or services. By creating an online consumer network, FMCG company General Mills is able to capture honest opinions from 170,000 influencers, including their thoughts on new products before they launch. This not only helps fine tune strategy, but also creates a buzz that drives demand when products hit the shelves.

Similarly, Warner Music has built a music community where 10,000 customers provide feedback on songs, videos and CD covers prior to release, as well as analysing the effectiveness of advertising measures and sales points. This gives the company a direct channel to tap into emerging music trends and has helped establish new artists while optimising return on investment.

The combination of technology and more vocal consumers can deliver incredibly powerful insights on your brand, current and future products from those that are closest to them – your existing customers. The real mystery in the 21st century is why any brand would spend their hard-pressed budgets paying strangers to pretend to buy from them, when they already have an engaged community that wants to deliver their opinions, within their existing customer base.

This blog was first published on MyCustomer on 11th September, 2014


Want to see how Questback can work for your business?