The Value of Repeat Responder
Feedback is Immense

Catherine is confused. She has just completed a deep study of the relationship between customer satisfaction and customer loyalty. There are surprises, some bad. Some customers whose NPS improved over the last 18 months did not renew their contracts. Yet other customers with sharp declines in scores remain loyal, even though the company did nothing special to retain them. What is going on?

Catherine leads customer experience for a large telecoms provider. It specializes in selling VoIP solutions to corporations. She has been with the company for two years and reports to the CEO. She now fears for her job. How has NPS lost predictive value? She has had years of successful financial forecasts with two other companies. Could they have been exceptions in some way?

Best practices

In this article we are looking at relationship research using survey-based NPS. Spectrum NPS will automate and greatly enhance NPS analytics. However, to use NPS you also need a feel for how it works at the basic data level.

To proceed with our story… You follow best practices in customer experience research. You have analyzed feedback from your relationship survey and you have a Net Promoter Score at brand level. You also have other satisfaction / agreement scores for the elements in your customer journey. From this data you find the top drivers of loyalty. You implement improvement activities to increase satisfaction and loyalty.

After a period of time, the next wave of the relationship survey comes in. The first thing everyone wants to know is “Has our score improved?” That bare question is easy to answer. But even if your score has improved, does the movement represent a true improvement in the experience you are delivering? If you have surveyed a representative sample of your customers in each wave, it is fair to assume that an increase in score is real.

Bear in mind that your customer base is changing. New customers in the honeymoon period are included in the sampling. Ones who have not renewed their contract have disappeared. Things change.

Comparing apples to apples

One really valuable analytic is repeat responder analysis. It applies in both B2B and B2C situations. Find customers / contacts who have responded in both periods. Their feedback will tell you at the individual client level whether you have improved (or not), and where. This data will provide a sub-plot to the headline story of your results.

Methodology

While it is possible to consider the actual scores given, it can be time-consuming to identify all the changes. Luckily, it is sufficient to look at movement by category, namely Promoters, Passives and Detractors. The procedure can differ somewhat depending on whether the data are B2B or B2C.

Recommended approach for B2C, or where B2B responses by account are limited

From the list of respondents from two survey waves, identify contacts who have responded twice. Do this using email address or, ideally, customer ID (which is unlikely to change even if the email address has). For each wave, note whether they were a Promoter, Passive and Detractor, then calculate:

• the number of Detractors in Wave 1 who are Detractors in Wave 2;
• the number of Detractors in Wave 1 who are Passive in Wave 2;
• and so on – see the example below.

The example table below shows NPS movement for these respondents in Wave 1 and Wave 2. The change is a truer trend of your NPS. So how did the change come about? Well, a change in the proportion of Detractors, Passives and Promoters (obviously!). What is interesting is what percent of respondents changed (or didn’t).

 Total number of repeat respondents = 928

Recommended approach for B2B

In B2B, the account rather than individuals is the ‘decision making’ or purchasing unit. The aggregate of the respondents in an account represents the likelihood of the account staying loyal, purchasing more, and so on.

In tracking repeat responders, the focus is only on determining whether the account is overall Promoter, Passive or Detractor. From the list of responses in Wave 1 and Wave 2, identify company accounts who have at least one respondent in both waves. (For a more robust analysis, you could select only those accounts with two or more responses, or even three or more).

Use this simple rule of thumb:

• if there are more Promoters than Detractors in the account, it’s Promoter;
• if there are more Detractors, it’s Detractor;
• if there is the same number of Promoters and Detractors, it’s Passive.

As above, calculate the number of Detractor accounts in Wave 1 and what category they are in Wave 2. Do the same for Passive and Promoter accounts.

Displaying the results

One way of visualizing the results is an Alluvial Diagram. It depicts the flow, or change of proportions, from one wave to the next. The thickness of the flow represents the volume (in this case the percentages). The chart below shows the data from the example above.

We can see that 72% of Promoters in Wave 1 were still Promoters in Wave 2. They are the most consistent group, so investment in creating Promoters pays off. It also shows that the danger with Passives (who are often ignored); they can become Promoters but also Detractors.

The data behind this chart (i.e., the actual contact names) can be used to contact the nine Promoters as well as the 52 Passives who all became Detractors. However, remember not to mention the NPS categories when speaking with clients!

Is it the same picture across all segments?

If you have sufficient repeat respondent data, segment your analysis to uncover variations in the trends. For example:

• Is there a lifecycle impact? Do customers stay loyal in their first one to two years of tenure and then decline?
• Are larger customer respondents more likely to remain the same than smaller?
• Does purchase of a specific product or service impact the likelihood to increase / decrease loyalty?
• We know there is a cultural / geographical impact on NPS. Is there a different pattern of repeat respondents in various geographies and do you have enough responses to determine that accurately?

What to do?

Where customer loyalty is declining, it is important to understand the root cause. It could be the result of one more poor service experiences, or it could be more fundamental. Ask yourself whether your product or service offering is still appropriate. Ask yourself whether an element of your service for these customers has reduced.

Similarly, looking at the customers who have become more loyal, identify whether you took any specific actions. Or did they change their relationship with you? Identify accounts or segments of customers who have the potential to increase their loyalty and focus efforts on them. If, as in the example above, Promoters are the most likely to remain Promoters, then your efforts will be rewarded.

Depending on the change (or not) in the NPS categories, the following table suggests a course of action.

With B2B customers, do not just look at the change in overall loyalty. Make sure that the account manager is tracking individual respondents, to understand the mood of the decision makers and influencers.

Understanding your ideal customer

Tracked over several years, we can use this analysis to understand customers who never increase their loyalty. Some people / customers will always be Detractors and you may determine a particular strategy for managing them (or maybe just let them go).

For Promoter customers who stay Promoters, do they share any common characteristics? Can you profile the ideal Promoter and target your acquisition efforts towards increasing your Promoter base? Acquiring customers is expensive, so targeting people and companies you know are likely to be or to quickly become Promoters is a wise investment.

Identifying the ideal Promoter and the likely Detractor has an added bonus when looking for new customers. For example, expending resources on customers who are likely to be Detractors could be very wasteful. Being able to sniff them out before signing them up is a potential big win.

After establishing the best type of customer, you should also be able to construct the best packages of product and service to offer them. Think too of creating the best package for Passive customers, to encourage them to become Promoters.

Back to Catherine

So, what had happened to our Catherine? Once she separated the repeat responders from the first-time responders, there were almost no surprises left.

There were situations where relatively positive answers from new respondents masked deteriorating views of people who had answered in Wave 1. And the new people turned out to have less influence on purchase decisions than those who had answered several times over the years. Fortunately, the same applied in the opposite direction, with positive repeat responders ensuring contracts got renewed.

All this will be quite easy to explain to her boss. The financial predictions based on repeat responders alone look pretty positive. Quite a relief.

Source: OCX Cognition.

ABOUT INSIGHTS FROM OCX COGNITION

OCX Cognition delivers the future of NPS. We ensure customer experience success by combining technology and data science with programmatic consulting. In our Insights section, we present a comprehensive and evolving collection of resources based on our research and expertise, collected for CX leaders committed to delivering business outcomes.