Text Analytics metrics

To extract meaningful, practical insights from textual customer feedback, Medallia Text Analytics uses several scores and calculations (collectively called metrics) for display in reports. Sentiment and impact require configuration, while other metrics are calculated automatically by Medallia Experience Cloud.

Sentiment

Text Analytics adds an additional layer of context to unstructured feedback by identifying the sentiment associated with individual phrases. Phrases are sorted into one of six sentiment classes, which can be displayed in reports: strongly positive, positive, mixed opinion, negative, strongly negative, and neutral. For detailed information, see Sentiment analysis.

Impact

The impact score displayed in reports shows the influence of each individual topic on overall customer satisfaction, and is indicative of the topics people discuss when they give high or low scores. Topics with higher impact scores have more impact on customer satisfaction than topics with lower impact scores. Comparing the impact scores of your topics can help you decide which areas you need to prioritize improving. For detailed information, see Impact score.

Percentage of responses

Text Analytics includes the following metrics for displaying the percentage of responses:

% of Total Responses
% of Responses

The % of Total Responses metric shows the percentage of responses including the selected topic or theme out of all responses in the dataset.

The % of Responses metric is primarily used across TA reporting. This metric shows the percentage of responses including two topics or themes out of responses containing the main insight.

Note: If you do not filter for a topic or theme in a report displaying % of Responses, that metric uses the full set of response data, just like the % of Total Responses metric. After you filter by a topic or theme, the % of Responses metric displays data calculated from the limited set of data, as described below.

For example, the following table shows four different responses with comments, and the topics associated with those comments:


Response	Comments	Topics
1	The associates at your store barely know anything. But I come because your prices are awesome.	Staff - Knowledge in phrase 1, and Prices in phrase 2
2	The associates at your store don’t know anything about the prices of individual items. I am so frustrated.	Staff - Knowledge in phrase 1, and Prices in phrase 1
3	The associates at your store don’t know anything.	Staff - Knowledge
4	The associates at your store are really nice.	Staff - Attitude

Assume that you want to learn when people talk about whether your staff is knowledgeable, how often is it about knowledge of prices? This would be a co-occurrence search of the Staff - Knowledge topic (the contextual topic) and the Prices topic (the co-occurring topic).

You can get this information in reporting either:

By having the Topic Deep Dive filtered by Staff - Knowledge topic and in the Related Topics module, looking at the row for Prices topic.
Or by looking at the row for Staff - Knowledge and Prices topics in the Co-occurrence module.

When using phrase-level filtering:

The % of Total Responses metric is 25%, since only one of the four responses (response 2) includes the Staff - Knowledge and Prices topics in the same phrase.
The % of Responses metric is 33%, since there are three responses with the Staff - Knowledge topic, and of those responses only one response (response 2) includes the Prices topic in the same phrase.

When using record-level filtering:

The % of Total Responses metric is 50%, since two of the four responses (response 1 and response 2) include the Staff - Knowledge and Prices topics in the same record.
The % of Responses metric is 67%, since there are three responses with the Staff - Knowledge topic, and of those responses two responses include the Prices topic in the same record.

Reporting metrics and calculations

In most Medallia Alchemy Experience Reporting modules for Text Analytics, you can select one or more of the following metrics to display. For more information about configuring Text Analytics report modules, see Reporting.


Metric	Description
Score	The Main Score of the record, such as Likelihood to Recommend. The score (or question) is denoted in the module's sub-header and is displayed on the module with the Calculation that was applied. The score's source is defined by the Calculation filter in the report's Control panel.
# of Responses	The total number of filtered records tagged to the topic or theme result. Use this metric to understand the total size of a topic relative to other topics within a filtered dataset.
% of Responses	The percentage of filtered records tagged to the topic or theme result compared to the total number of filtered records. Use this metric to understand the relative proportion of tagged records.
% of Total Responses	The percentage of responses with the selected topic or theme out of all responses. Important: This metric has been deprecated.
Impact Score	Indicates how much the topic or theme increased or decreased the overall score. It takes into account the topic’s volume as well as its score. Use this metric as a prioritization tool to understand the relative ranking of a topic against other topics. The formula for impact score is: $\frac{[sample size of topic \times (key metric score of topic - key metric score overall)]}{(sample size overall - sample size of topic)}$ Medallia Alchemy Experience Reporting reports can calculate the impact score for various score fields, as configured by the report administrator and the report user. Legacy reports require a main score field to be configured. For more information, see Configuring the impact score.
Impact Score (Scaled)	This is an alternative metric to Impact Score, to be used when the key metric at your company uses a scaled range (such as 1-5 or 0-11). Best practice is to use either Impact Score or Impact Score (Scaled), but not both. A topic’s scaled impact score indicates how much that topic increased or decreased the overall score, multiplied by 100. It takes into account the topic’s volume as well as its score. Use this metric as a prioritization tool to understand the relative ranking of a topic against other topics.
# Positive	The number of records processed by Text Analytics containing Positive or Strongly Positive sentiment. Sentiment is identified at the phrase level; while a topic may be mentioned multiple times with positive sentiment in the same record, the record only counts towards this metric once. Use this metric to understand positive sentiment when mentioning the topic or theme result.
% Positive	The percentage of records processed by Text Analytics containing Positive or Strongly Positive sentiment. This is helpful to understand the relative proportion of customers favorably mentioning the topic or theme result.
# Negative	The number of records processed by Text Analytics containing Negative or Strongly Negative sentiment. Sentiment is identified at the phrase level; while a topic may be mentioned multiple times with negative sentiment in the same record, the record only counts towards this metric once. Use this metric to understand negative sentiment when mentioning the topic or theme result.
% Negative	The percentage of records processed by Text Analytics containing Negative or Strongly Negative sentiment. Use this metric to understand the relative proportion of respondents unfavorably mentioning the topic or theme result.
Net Sentiment	% Positive sentiment minus % Negative sentiment for records processed by Text Analytics. Use this metric to understand sentiment about a particular topic or theme.
Impact on Net Sentiment	The impact of a specific topic or theme on the net sentiment score, calculated as: $\frac{[sample size of topic \times (net sentiment score of topic - net sentiment score overall)]}{(sample size overall - sample size of topic)}$ Use this score to understand which topics or themes should be prioritized as areas for improvement.

Co-occurrence reporting metrics

When co-occurrence mode is enabled for a reporting module, metrics that measure topic-theme or topic-topic results instead measure co-occurrence results. This means that instead of comparing an individual topic or theme result to the set of filtered records, the topics and themes that tended to occur together ("co-occurrence results") are compared as a single unit to the set of filtered records.

For example, the # of Responses metric counts the total number of filtered records tagged to the topic or theme result. This is helpful to understand the total size of a topic relative to other topics within a filtered dataset. But when co-occurrence mode is enabled, the # of Responses metric counts the total number of filtered records tagged to the topic/theme and the the co-occurring topics/theme result. This is helpful to understand the total size of this result relative to other co-occurring topics/themes within a filtered dataset.

The table below defines Text Analytics metrics when using a reporting module with co-occurrence mode enabled.


Metric	Description
Score	The Main Score of the record, such as Likelihood to Recommend. The score (or question) is denoted in the module's sub-header and is displayed on the module with the Calculation that was applied. The score's source is defined by the Calculation filter in the report's Control panel.
# of Responses	The number of filtered records tagged to the topic or theme and the co-occurring topics or theme result. Use this metric to understand the total size of this result relative to other co-occurring topics or themes within a filtered dataset.
% of Responses	The percentage of filtered records tagged to the co-occurrence result compared to the total number of filtered records. Use this metric to understand the relative proportion of tagged records.
% of Total Responses	The percentage of responses with the selected co-occurrence result out of all responses. Important: This metric has been deprecated.
Impact Score	Indicates how much the co-occurring topics increased or decreased the overall score. It takes into account volume as well as score. Use this metric as a prioritization tool to understand the relative ranking of topics against other topics. The formula for impact score is: $\frac{[sample size of topic \times (key metric score of topic - key metric score overall)]}{(sample size overall - sample size of topic)}$ Medallia Alchemy Experience Reporting reports can calculate the impact score for various score fields, as configured by the report administrator and the report user. Legacy reports require a main score field to be configured For more information, see Configuring the impact score.
Impact Score (Scaled)	An alternative metric to Impact Score, used when the key metric at your company uses a scaled range (such as 1-5 or 0-11). Best practice is to use either Impact Score or Impact Score (Scaled), but not both. A co-occurrence result's scaled impact score indicates how much the co-occurring topics increased or decreased the overall score, multiplied by 100. It takes into account volume as well as score. Use this metric as a prioritization tool to understand the relative ranking of topics against other topics.
# Positive	The number of records tagged to co-occurrence results containing Positive or Strongly Positive sentiment. Sentiment is identified at the phrase level; while a topic may be mentioned multiple times with positive sentiment in the same record, the record only counts towards this metric once. Use this metric to understand positive sentiment when mentioning the co-occurring topics or themes.
% Positive	The percentage of records tagged to co-occurrence results containing Positive or Strongly Positive sentiment. Use this metric to understand the relative proportion of respondents favorably mentioning the co-occurring topics or themes.
# Negative	The number of records tagged to co-occurrence results containing Negative or Strongly Negative sentiment. Sentiment is identified at the phrase level; while a topic may be mentioned multiple times with negative sentiment in the same record, the record only counts towards this metric once. Use this metric to understand negative sentiment when mentioning the co-occurring topics or themes.
% Negative	The percentage of records tagged to co-occurrence results containing Negative or Strongly Negative sentiment. Use this metric to understand the relative proportion of respondents unfavorably mentioning the co-occurring topics or themes.
Net Sentiment	% Positive sentiment minus % Negative sentiment for records tagged to co-occurrence results. Use this metric to understand sentiment about co-occurring topics or themes.
Impact on Net Sentiment	The impact of a specific co-occurrence result on the net sentiment score, calculated as: $\frac{[sample size of topic \times (net sentiment score of topic - net sentiment score overall)]}{(sample size overall - sample size of topic)}$ Use this score to understand which topics or themes should be prioritized as areas for improvement.