Sentiment overrides (opt-in feature)

Sentiment overrides allow users to edit the sentiment of individual phrases. Users can edit sentiment from any dashboard with the Medallia Alchemy Experience Reporting Comment Stream or Responses Feed dashboard modules, or the legacy Verbatims Tag Set module.

Features and capabilities

Sentiment overrides take effect immediately after the company confirms the change and impact the entire system. Consequently, changing the sentiment for a significant number of comments could change the percentage of positive sentiment associated with a certain topic. Users can change the sentiment of a comment multiple times, but only the most recent change is saved.

For example, if the sentence Great company--strong principles, good people, first-class research, and excellent products, is incorrectly identified as negative by the sentiment model, a company can correct the error by changing the sentiment to positive. After the user confirms the change, when other users view the same sentence, it will have a positive sentiment tag.

Important: Sentiment overrides are local to each company. They do not impact the sentiment model in real time. However, when Medallia updates a model, mis-classified sentences identified by users are incorporated into the training data.

Training data includes sentences previously mis-classified by the model, in addition to the manually annotated sentences used to create the model.

To override sentiment in a report, click the smiley next to a highlighted phrase. In the dialog box, select the correct sentiment and then click Update.

Sentiment override dialog in a report

Tip: Reports that support sentiment overrides require you to filter to a specific topic. When there are multiple phrases in a comment, only the phrases that apply to the topic specified in the filter are highlighted.

Medallia Text Analytics reports can display up to six sentiments (strongly positive, positive, mixed opinion, negative, strongly negative, and neutral). Users can choose from those same options when overriding sentiment.

Strongly Positive

Feedback that expresses intensely positive and little or no negative sentiment. For example: The food was awesome, the best flavor of my life!

Positive

Feedback that expresses a positive feeling, but the positivity is not intense. For example:

I checked in very late, but was greeted by pleasant staff.
Check-in and check-out were easy and quick.

Mixed opinion

Feedback that discusses more than one topic and conveys contradictory sentiments towards different topics. For example:

Overall stay very good; electronic key cards very poor, constantly had to have reactivated, delayed us getting into the room following check in.
The bright side was the counter staff (except for check-in) & housekeeping staff (very nice & hard-working).

Negative

Feedback that expresses a negative feeling, but the negativity is not intense. Suggestions for improvements are implicitly expressed negative sentiments. For example:

I would think twice about staying here again, probably the new Serenity Garden Inn.
Not up to the standard that I expected for the Serenity Garden Inn.
...but would prefer passes did not have so many exclusions.
Sales associates should always greet customers.
You can give the guest a call to let them know when housekeeping will be coming to clean.

Strongly Negative

Feedback that expresses intensely negative and little or no positive sentiment. For example: The food was awful and the room wasn't nice.

Neutral

Feedback that expresses a neutral opinion or an objective fact. For example:

Very late when we checked in.
I travel 30-40 weeks per year for business and try to stay at the same locations for convenience.
The weather is hot.

After users begin to override sentiment, you cannot revert a subset of sentences back to their original sentiments. By request, Medallia can revert all sentences back to the sentiments originally assigned by the sentiment model, but you should only consider this option as a last resort.

Enabling sentiment overrides

The authority to override sentiment is assigned at the role level. Discuss which roles should be permitted to override sentiment with your company.

Important: Medallia recommends that users familiarize themselves with the definition of sentiment categories before modifying the sentiment labels. The quality of a sentiment model relies on the quality of labeled data, so it is important to have consistent sentiment labels. In addition, Medallia recommends that only corporate users or a dedicated team be permitted to edit sentiment. In order to maintain quality and prevent potential cheating, do not allow front-end users to modify sentiment.

To enable sentiment overrides for a role, assign the Edit Sentiment reporting permission to the role. For more information about assigning permissions, see Roles.

Exporting sentiment overrides

You can monitor the sentiment overrides users have made by sending yourself a list of them in CSV format. The exported file includes the following columns, with a separate row for each override:

Company: Name of the company.
User: Name of the user who made the sentiment override.
User Role: Role of the user.
Time of change: Date and time the override was issued.
Processed Sentence: Sentence to which the sentiment override was applied.
Language: Language the survey taker wrote the comment in originally.
Topic: The topic assigned to the sentence in the Processed Sentence column.
Comment Field: Name of the comment Q-field.
Comment: Complete comment, in the original language.
Translated Comment: Translated version of the comment. If the comment was not translated, this field is blank.
Original Sentiment: Sentiment assigned by the sentiment model to the sentence in the Processed Sentence column.
Edited Sentiment: Sentiment assigned by the user to the sentence in the Processed Sentence column.
SurveyID: Survey ID associated with the comment.
Response Date: Date and time the survey was submitted.

To export sentiment overrides:

Open the Reporting > Text Processing > Topic & Theme Tagging > Sentiment screen.
Add a date to the ExportOverridesSinceDate property.
Select the Confirm box, and then click Send Email Override Exports.
You will receive the file by email.