Impact score

The impact score displayed in Medallia Text Analytics reports shows the influence of each individual topic on overall customer satisfaction, and is indicative of the topics people discuss when they give high or low scores. Topics with higher impact scores have more impact on customer satisfaction than topics with lower impact scores. Comparing the impact scores of your topics can help you decide which areas you need to prioritize improving.

Note: Impact score is not an average or percentage score. Rather, it is a statistical figure based on a Medallia formula, expressing the difference between two scores while also considering the number of records used for each score.

Typically, an impact score is calculated for every topic with 100 or more comment matches.

The following example shows how to calculate the impact score for the topic "price" among a total of 10,000 surveys with a total NPS of 75. Since price was not mentioned in 9,000 of 10,000 surveys, the NPS of 78 (for "price" not mentioned) is subtracted from the NPS of 75 (for all surveys) for an impact score of "price" on NPS as -3. This negative score indicates that "price" is driving down overall customer satisfaction.

Impact Score example

For Medallia Alchemy Experience Reporting reports, impact score is determined for various score fields dynamically. For legacy reports, impact score relies on a main score field.

For information about setting up the impact score for legacy reports, see Configuring the impact score.

The importance of impact score

The impact score shows the relationship between a score (such as NPS) and survey comments. A positive impact score indicates that customers who comment about the topic are, in general, more satisfied than the average survey population—these customers are pulling up survey scores. A negative impact score indicates that these customers are less satisfied—these customers are pushing down survey scores.

In the following image, the Selection - Size topic has the lowest impact score, revealing that customers commenting about that topic are the least satisfied with their experience. Reading the comments related to Selection - Size and resolving the issues you discover could improve the impact score and, ultimately, increase your satisfaction scores.

Customers commenting about the Selection - Size topic are impacting NPS negatively

Although the impact score quantifies how much the surveys associated with a topic are raising or lowering the key metric score, it is intended to provide directional guidance only. Survey takers give a particular score for any number of reasons. For example, they might not explain all those reasons in the comment field. Alternatively, survey takers might comment about multiple topics that impacted their experience, rather than just one topic. Consequently, there is no guarantee that focusing on improving one particular topic will increase your overall score by the exact number indicated by the impact score.

Impact score calculation

When calculating the impact score, the system takes the following factors into account:

Average key metric score of all survey takers
Average key metric score (such as NPS or overall satisfaction) of survey takers who discuss the topic
The number of survey takers that commented about the topic in their survey
The total number of surveys

The formula for impact score is:

$\frac{[sample size of topic \times (key metric score of topic - key metric score overall)]}{(sample size overall - sample size of topic)}$

Sample Size of Topic: Number of survey respondents who discussed the topic.
Sample Size Overall: Number of survey respondents who answered the key metric question (all respondents).

Tip: Applying filters to the results reduces the overall sample size. Consequently, the denominator of the impact score formula changes and the impact score is different as a result.
For example, suppose you view the impact score with no filters, and then decide to filter to a specific language. After you apply the language filter, the overall sample size includes only the surveys in the specified language.
Key Metric Score of Topic: Average key metric score of the survey respondents who discussed the topic.
Key Metric Score Overall: Average key metric score of all survey respondents.

Note: When companies use a key metric that has a small rating scale (such as 0–5 or 1–10), the final impact score is often scaled up. The scaled impact score is the same as the impact score, except the result is multiplied by 100. For example, suppose an impact score is a small fraction, like 0.13. The scaled impact score would return 13 for the same data.