Automated score calculation components

Each of the automated score components below is assigned a weight indicating its value relative to other components. For example, a call experience automated score might weigh overtalk higher than hold time because the organization considers talking over a customer to be worse than having them on hold.

1. Present topics — These items are desirable and increase the automated score.

   For example, agent greeting is a present topic in the default Agent QM score.

2. Absent topics — These items are undesirable and decrease the automated score.

   For example, agent profanity is an absent topic in the default Agent QM score.

3. Metadata — Metadata is structured data about the interaction, such as hold time or overtalk.

   Metadata availability may vary based on the types used by default automated scores. If an automated score uses metadata that is not available, that value's weight should be redistributed to other score components during customization.

   Metadata availability and priority tend to vary by customer, so automated scores can be customized and new automated scores can be created to suit specific implementation scenarios.

Implementation

These steps are the most significant development stages for automated scores. Score development for specific experience programs may vary from these milestones.

1. Requirements gathering. Evaluation of reporting metrics and brand-specific experience priorities and rubrics to identify opportunities for deeper analysis from an automated score program.
2. Score recommendation. Recommendation of specific automated scores and potential customizations based on the requirements identified.
3. Score review. Post-recommendation score element review, along with additional customization as needed.
4. Rubric finalization. Final review of score components and rubric.
5. Initial setup. Scores are tested, edited, and customized as needed based on feedback and testing. End users are taught to use and understand the automated scores.
6. Go-live. The score is live and can be used for reporting.
7. Calibration period. Recommended period of 2 to 3 months for fine-tuning new automated scores.
8. Maintenance. Recommended annual updates to score based on new product features, changes to reporting requirements and rubrics, and feedback.

Interpreting automated scores

Although automated scores range from 0 to 100, the threshold for a so-called good score is typically around 65 to 70. Score averages and quality thresholds vary because they depend on the customer's data, topic accuracy, rubrics, and quality priorities. Each individual score is different, and so is its analysis.

Medallia recommends using score bands for automated scores instead of analyzing the raw score out of 100. Score bands group individual scores based on historical averages to show how a score compares to other scores, rather than to 100. This keeps the focus on the individual score when evaluating scores.

For example, an individual record may have a score of 50 out of 100 for Chat Experience, and a score of 65 out of 100 for Agent Quality Management. Taken individually at face value, these scores may appear good or bad depending on the observer's experience. And the experience score might appear better or worse than the quality management score. But evaluating these scores in bands could show them both to be high relative to the same type of score in other records.

Score bands are generated manually based on averages from a single type of automated score. Bands generated for one type of score will most likely not apply in the same way to other types. The example score bands below provide immediate insight about an individual score while obfuscating the final numeric value.

Preconfigured score examples

Experience Cloud includes preconfigured automated scores that may be customized to use different topics, value weights, or metadata.

1. Call/Chat Experience Score — Describes customer experience during a chat or call center interaction

   Present topics

   • agent greeting
   • verification of information
   • closing
   • ownership
   • empathy

   Absent topics

   • agent unprofessional language
   • uncertainty
   • repeat; customer confusion
   • frustration
   • either party asking the other for repetition

   Metadata

   • silence
   • hold time
   • call duration
   • overtalk

2. Call/Chat Agent QM Score — Evaluates an agent's adherence to quality protocols during a chat or call center interaction with a customer

   Present topics

   • agent greeting
   • introduction
   • customer name request
   • additional assistance
   • pleasant close
   • use of caller's name
   • empathy
   • ownership
   • courtesy
   • acknowledgment of customer
   • bridging
   • self-service promotion
   • summary of customer's request

   Absent topics

   • agent tragic phrasing
   • unprofessional language
   • asking for customer repetition
   • not yielding to customer
   • uncertainty

   Metadata

   • overtalk

3. Call/Chat Interaction Effort Score — Quantifies the overall effort required from the customer during a chat or call center interaction

   The interaction effort score uses two components. The first component uses topics and metadata that indicate customer difficulty. The second uses topics and metadata that indicate customer ease. These components have equal weight in the combined Interaction Effort Score.

   • Interaction Effort Score (Hard)

     Present topics

     • channel switching
     • self-service
     • repetition of information
     • generic treatment of customer
     • transfers
     • escalation
     • length of time
     • negative emotion
     • language describing perceived difficulty

     Metadata

     • transfers
     • overtalk

   • Interaction Effort Score (Easy)

     Present topics

     • ease of use
     • positive ending
     • agent confirmation of ease
     • positive emotion
     • first call resolution

     Metadata

     • overtalk
     • silence

4. Call/Chat Multiple Contact Flag — Uses yes/no flags to indicate multiple contacts to enable analysis of records for first contact resolution

   Topic flags

   • customer made multiple attempts to call
   • customer made contact via other channels like chat or email
   • agent offer of additional help

5. CX Risk Score — Uses present topics and metadata to detect customer anger, irritation, and potential for churn

   Present topics

   • multiple contact attempts
   • contact attempts via multiple channels
   • wait time
   • not receiving a callback
   • discrimination
   • frustration
   • churn
   • negative loyalty and trust
   • threats of legal action
   • filing a complaint
   • requesting a callback
   • requesting escalation
   • customer use of profanity or harassment

   Metadata

   • first contact resolution

Score customization

Preconfigured automated scores are designed to offer a strong starting point, but they may be customized. New automated scores may also be created.

Whether using preconfigured, customized, or new scores, components and design principles are the same: automated score components and weights depend on the organization's quality rubrics, experience priorities, performance indicators, and other objectives.

There are four ways to customize an automated score.

1. Add new present or absent topics to the calculation
2. Edit the present and absent topics already included in the calculation
3. Add metadata to the calculation
4. Redistribute the weights of calculation components

Preconfigured flag examples

Experience Cloud includes these preconfigured automated flags.

Churn Indicator

Customer indicates they plan to churn or leave

Legal Risk

Customer threatens legal action

Customer Confusion

Customer expresses confusion

Suggestion

Customer offers a suggestion or recommendation for improvement

Transfers

Customer is transferred to another department or agent

Multiple Call Attempts

Customer has to contact multiple times