Weighting
Weighting allows companies to view scores that are accurately representative of customer experience by customer value. It might be needed if the distribution of completed surveys doesn't match the underlying distribution of customer value. This might be either because of varied customer engagement, difficulty in surveying certain customer segments regularly, or some other reason. Weighting allows companies to:
- Make better decisions
- Better reflect an accurate customer response mix or volume — For example, they might want to weight results to align to a more accurate demographic profile, regional distribution, and purchase types that reflect their customer base.
- Better reflect the value (the overall contribution to the business) of a customer group — For example, the US might represent 75% of the customer base, but UK customers might spend more. In that scenario, UK customers have a greater impact on revenue, and the company might want to give their surveys more weight.
- Achieve best of both worlds: maximum feedback with representative data — This allows companies to control both the feedback collection (through survey sampling), and the reporting side (through weighting). Companies can then pursue as much feedback as possible instead of setting quotas to mimic the population mix. They can also correct bias in a survey sample if the response profile is not reflective of the population it represents.
- Be transparent and fair — When companies link customer experience aggregate scores to compensation, it becomes important that the score is reflective of the underlying business, so employees remain engaged.
For detailed information about how Medallia has implemented the concept of weighting, see Weighting methodology.
For detailed information and a procedure for configuring weighting, see Configuring weighting.
Real world use cases
Examples of weighting use cases include:
- A retail (B2C) company weights the monthly scores of its stores by the store's transaction volume.
- An enterprise SaaS company (B2B) weights the quarterly account scores by the CRR of the account.
- An investment firm managing corporate pension funds (B2B) weights the annual account scores by the assets under management in the account.
- An advisory firm (B2B) would like to publish a firm-wide NPS in which each of their customers is equally weighted, regardless of response volumes.
- An hotel chain (B2C) would like to ensure the monthly scores across its properties reflect the number of rooms in each hotel.
PetMogul example
To illustrate the impact of weighting, consider a simple example. The fictional PetMogul pet store caters to owners of four types of pets: cats, dogs, hamsters, and turtles. PetMogul decides to run a post-sale survey, and after one month sees a total NPS of 25. While that seems good, management feels like most of their customers are not happy, and that the reported NPS is not a true representation of customer satisfaction. To investigate, PetMogul takes a closer look at their customer and response data:
All dog customers are unhappy, all hamster customers are ambivalent, and all cat and turtle customers are happy:
| Customer segment | NPS | Customers | Responses |
|---|---|---|---|
| Dog lovers | -100 | 100 | 20 |
| Cat lovers | +100 | 50 | 20 |
| Hamster lovers | 0 | 30 | 20 |
| Turtle lovers | +100 | 20 | 20 |
It seems pretty clear that there is a problem. The distribution of responses does not match the distribution of customers. A more accurate reading of the data might be achieved by weighting by the number of customers:
| Customer segment | NPS | Weight |
|---|---|---|
| Dog lovers | -100 | x (100 / 200) |
| Cat lovers | +100 | x (50 / 200) |
| Hamster lovers | 0 | x (30 / 200) |
| Turtle lovers | +100 | x (20 / 200) |
| Total NPS | -15 |
PetMogul is also interested in understanding the business value of each customer segment:
There might be a lot of unhappy dog lovers, but the cat lovers spend twice as much. Cat and dog lovers are equally important to PetMogul's success from a revenue perspective, so a more accurate reading of the data might be achieved by weighting according to revenue:
| Customer segment | NPS | Weight |
|---|---|---|
| Dog lovers | -100 | x (1000 / 2600) |
| Cat lovers | +100 | x (1000 / 2600) |
| Hamster lovers | 0 | x (300 / 2600) |
| Turtle lovers | +100 | x (300 / 2600) |
| Total weighted NPS | +12 |
Working with the company during implementation
Experience Cloud weighting is relatively straightforward to setup and maintain. The big challenge is communication. Before starting a weighting project, and at several points during the project, communicate with the company so everyone is aware of the effort, cost, and effect on performance. Additionally, there are potentially many ways to weight scores. Medallia chose a way that is most applicable for customer experience management. Ensure that you are able to explain the the purpose and functions of weighting, so you can help the company embrace the Experience Cloud weighting implementation.
Communicating the value of weighting
Help the company understand how to drive value from weighting. Value is not in the score itself. Value is in the underlying insights the scores provide, and in the related investigation that serves as a catalyst to action.
Help the company determine what they should weight:
- Simpler weights might allow companies to better understand their scores and derive insights more easily.
- Consider the business impact. For example, weighting by current revenue could lead a company to overlook the potential revenue from certain segments.
- Transaction volume is a common approach, since it represents the number of customer experiences that took place.
- Business value is another common approach used when not all customers are of equal financial value.
Help the company determine who should see weighted scores:
- A corporate audience is usually most appropriate.
- Frontline staff might not need access to weighting to avoid score-watching, confusion, and disengagement. By showing the frontline staff a weighted score, they might externalize that not all customers are equal. Even small deviations in scores can cause confusion and grow distrust in scores, which is cancerous for a customer experience program. However, ensure that you initiate this discussion to understand the company's specific needs.
- Continuing education of the company's users is necessary, even for Executives. For example, people need to understand how to read and compare weighted and unweighted scores.
Help the company understand when weights should change:
- Weights based on business value should not change frequently, so comparisons over time easier.
- If weights are based on transaction volume, more frequent updates might be appropriate to reflect what is happening currently.
- Changes in weights might need to coincide with changes to goal-setting.
Scoping
The following estimates might help you begin to scope the total effort. Work with Solutions Architecture or your Medallia Expert Services team to refine your estimate before communicating with the company.
Scope implementation at 92 hours + 28 per weighting scheme
- Project overhead: 20 hours
- Segment weight importers: 24 hours + 16 per weighting scheme
- Survey weight exporter: 8 hours
- Testing and calculation verification: 40 hours + 12 per weighting scheme
Scope services at 42 hours + 8 per weighting scheme
- Servicing overhead: 16 hours
- Additional data flows: 16 hours + 4 per weighting scheme
- Reporting maintenance: 10 hours + 4 per weighting scheme
