Want Better Customer Insight? Understand These 3 Scoring Models

For most people, the words “scoring model” might bring to mind credit scoring or sports scores. But for those of us working in customer experience and insight roles, scoring models affect almost everything we do.

You’ll see experts arguing about whether quantitative or qualitative data is better for insight. Let’s not get caught up in that argument today — both are important in different ways and for different reasons.

When you reduce responses to numbers, you can do a lot of useful mathematical analysis, so the power of quantified information is undeniable. But the way you quantify and model the data has a strong influence on the way you perceive your results. Each scoring model offers a different perspective.

Here’s a quick explanation of three commonly used but widely misunderstood scoring models along with their main advantages and drawbacks, to help you evaluate what your choice of the scoring model means for the quality of the customer insights you base on your scores.

Scoring non-numeric responses

Before we get into the common scoring models, it’s important to recognize that non-numeric responses can still be analyzed mathematically.

Whether you ask someone to give their feedback on a scale of 1 to 5, a scale of “strongly disagree” to “strongly agree” or a scale of “sad face” to “happy face”, the aim is typically to quantify the response — for example, by giving “sad face” a score of 1 and “happy face” a score of 5.

If you want to, you can reduce even purely qualitative data (for example, answers to open-ended questions) to a set of numbers by analyzing things like overall keyword frequency, or the frequency of words that follow key phrases such as “I like…”, “I want…” or “I hate…”.

Net scoring

Net scoring, also called difference scoring, involves subtracting one measurement from another to derive a single, easy-to-remember, easy-to-track number.

The best-known example is probably Net Promoter Score™, calculated via several steps:

1. Ask your customers how likely they are to recommend your company, product or service.
2. Ignore any “passive” ratings of 7 or 8 out of 10. (The creators of the Net Promoter Score thought the neutral point of customer sentiment should be higher than the mid-point of the scale. Does that reflect consumer subjectivity, or designer subjectivity?)
3. Calculate the percentage of remaining responses that fall above the omitted “passive” band and the percentage that fall below it.
4. Subtract the below-passive percentage from the above passive percentage.
5. This is your Net Promoter Score. If it’s higher than zero, more customers like you than not. If it’s below zero, you’re doing something wrong.

Something is wrong with this perspective on your customers. Setting aside the unusual definition and treatment of neutral ratings in the Net Promoter Score model, net scoring of any kind removes information from your results.

If you obtain a net score of +15, you might have arrived there by having 20% positive and 5% negative feedback. Or perhaps it was 50% positive and 35% negative. Or 15% positive and 0% negative. Wouldn’t you want to know whether you had 0% or 35% unhappy customers? Of course you would. But net scores won’t tell you.

Ratio scoring

Ratio scoring is the “sharing a pie” or “zero-sum game” approach to metrics. It lets you find the scale between two or more measurements where it’s assumed that increasing one of those measurements will decrease the other. So if you collect 120 positive responses and 60 negative responses in an opinion poll, the ratio of positive to negative is 2:1.

Rates, such as conversion rates or product return rates, are ratios expressed as a fraction or percentage. A 1-in-10 conversion rate is a 1:9 ratio of conversion to non-conversion; a 1% returns rate is a 1:99 ratio of returned to non-returned items.

Ratios and rates are handy metrics. They’re easy to grasp, like “9 out of 10 customers plan to buy from us again.” But the downside is, ratio-based scores can vary wildly from one day to the next and the score alone won’t show you which side of the ratio has changed.

For example, if your customer survey results tell you that today the rate of customers who plan to shop with you again has fallen to 5 out of 10, is that because today your customers included a busload of tourists who are unlikely to revisit your location? Or is it because today something happened to make your regular customers less likely to return?

Ratio metrics are helpful to identify areas you may want to analyze in greater detail, but they don’t directly convey the reality of the situation — you’ll need to explore the finer-grained source data to gain insight.

Ranked scoring

You’ve already heard that most of your revenue will come from only a small fraction of your customer base. Which of your customers are in the top 1% for loyalty? Who are your 100 highest-value buyers?

If you use percentiles, top 100s or any other kind of ranking, you’re taking advantage of this scoring model’s ability to instantly identify the extremes and mid-point of a spectrum. By choosing a single measurement and ordering everyone along that axis, you can pick out the top, bottom, or median customers (or products, or customer service workers).

The problem with ranking-based approaches is that they create arbitrary boundaries between, for example, your 100th and 101st highest-value customers. The difference in value may be only a penny, but if you treat the 101st customer differently you miss a huge opportunity. So think carefully about what data you rank and why.

Warning! Subjective scoring ahead

Bear in mind that no matter what scoring models you use, if you collect your data from human beings then the responses you get are subjective from the start. And interpreting those scores is equally subjective.

You might think a score of 5 out of ten is neutral. Your customer survey participants disagree. You might believe 70% conversion is excellent, but your board of directors wants to know what happened to the other 30%. No matter how much you focus on the numbers, there will always be people behind them — real, inherently biased, occasionally irrational people.