Themes with generative AI
Functionality, reporting module support, administration, and processing configuration for GenAI themes in Medallia Experience Cloud
Themes are summarized groups of similar or related phrases from survey respondents' comments. Themes can be used to expose new and emerging feedback insights, to detect new trends that may turn into issues, and to discover the root cause of those insights and issues.
Themes with generative AI use artificial intelligence for feedback theme discovery, clustering, and description. These GenAI themes are less redundant, more specific, and easier to understand than themes generated without AI.
New themes are generated from feedback phrases with 2 or more words, and all matching phrases (including single-word phrases) are tagged with discovered themes. Similar phrases are clustered together, and similar clusters are combined to avoid generating duplicate themes.
GenAI themes use CS, EX, and Social data from Experience Cloud in native English, and can be configured to use comments that have been translated into English. For more information about language support with genAI features, see Language support for generative AI.
Each new theme has a title exposed to the end user and several internal components. All GenAI themes are considered level 1, and additional sub-theme levels are not generated.
GenAI themes do not replace human review of individual, high-priority verbatim comments, nor do they fully automate feedback taxonomy decisions; human confirmation is required for theme publishing. Fine-tuning and calibration may be required for highly specialized or technical domains.
Reports and GenAI themes
Although the discovery engine has changed, most existing Text Analytics and reporting functionality remains unchanged. GenAI themes support all of the same Text Analytics reporting modules, metrics, and filters as themes generated without AI. When enabled for a tag pool, GenAI themes data replaces existing theme data throughout reporting for that tag pool.
Reporting modules with GenAI themes data support the Topic Filter module and the single- and multi-theme data shapers. Excel, PDF, and image export functionality is unchanged. When searching for themes in the Control Panel or TA Explorer module, only theme titles are included in the search.
These reporting modules can use GenAI themes:
- Text Analytics modules
- Other modules
Administration and processing for Themes with GenAI
Users must have the Manage Themes administrative permission to configure Themes with GenAI.
Theme processing is configured by tag pool. To enable GenAI themes processing for a tag pool, use Setup to navigate to Reporting > Topic & Theme Tagging > Auto Tagging Pools and enable the Process using Themes with GenAI option for each tag pool to use GenAI themes.
Up to 3 tag pools can be configured to use GenAI themes on one instance, and this maximum is configurable. In general, GenAI themes processing works best with fewer, larger tag pools. The following image shows the Auto Tagging Pools screen in Setup with the Process using Themes with GenAI option enabled.
To configure and initiate theme discovery and the frequency of automated reprocessing, use Setupto navigate to Reporting > Topic & Theme Tagging > Themes with GenAI Configuration.
By default, theme processing is limited to native English data. To include translated comments, enable the Include translated comments for Themes discovery and processing global configuration option on the Themes with GenAI Configuration screen.
The following image shows the Themes with GenAI Configuration screen with processing history and theme discovery options.
- Initial discovery
-
Initial discovery generates themes from a tag pool, and removes any previous theme data for that tag pool. Previous theme data is deleted at the end of the discovery process, and if discovery fails, the deletion is not performed.
The following historical data time periods may be used for initial theme discovery. All time periods listed are calculated on a rolling basis from the current date.
- Last year (the preceding 365 days)
- Last six months
- Last three months
- Last month
If the time period selected for initial theme discovery contains more than 500,000 phrases, themes are generated with a randomized sample of a maximum 500,000 phrases.
- Automatic discovery
-
Automatic processing cadence tends to be a function of how quickly responses come in for a given business. For example, a very large retailer might want daily theme discovery due to high feedback volume. A smaller company with fewer responses coming in more slowly might not need daily reprocessing.
Automatic reprocessing frequency options include
- None
- Daily
- Weekly
- Every two weeks
- Monthly
If the time period selected for automatic theme discovery contains more than 100,000 phrases, themes are generated with a randomized sample of a maximum 100,000 phrases.
Records are selected for processing based on the Responsedate system field (e_responsedate).
Processing for GenAI themes has two main components: model training and historical processing. While executing, the processing job is in one of these states:
- Running — Execution is in progress.
- Completed — Model training and historical processing were completed successfully.
- Canceled — Dataset contains insufficient phrases for model training, or historical processing was aborted.
- Failed — There was an unexpected error during model training or historical processing. When there is a processing error, an Admin Event Ticket is generated, and themes processing must be restarted manually.
To start a new GenAI themes processing, any existing processing execution must be completed or canceled. While themes are processing, a table in the Processing History section of the Themes with GenAI Configuration screen lists the processing job's current progress in the Description column.
To pause processing, use Setup to navigate to Reporting > Global Text Analytics Settings > Processings. The Current executions section includes an Actionable executions section. Use the Actionable executions dropdown to select the processing job, then click the Save button at the top or bottom of the screen. When the page reloads, the Actionable executions dropdown shows a Pause button with a Confirm checkbox. Select the Confirm checkbox and use the Pause button to pause processing.
When processing is paused, the Actionable executions dropdown shows Resume and Abort buttons with Confirm option. To resume or abort processing, select the checkbox by Confirm and use the Resume or Abort button.
While paused, the job's description in the Processing History table on the Themes with GenAI Configuration screen indicates that the job is paused, but this state is not reflected in the execution status, which remains Running. Processing jobs may take up to 15 minutes to pause or resume.
To stop the discovery process, use Setup to navigate to Reporting > Topic & Theme Tagging > Themes with GenAI Configuration, select the relevant tag pool, and set the Automatic discovery frequency to None.
Tag pools with existing theme data are eligible for initial discovery processing, whether that is legacy theme data or an older iteration of GenAI themes data. All existing theme data is deleted when initiating GenAI themes discovery for a tag pool.
To view and rename themes, use Setup to navigate to Reporting > Topic & Theme Tagging > Themes with GenAI. Themes shown may be filtered by tag pool using the Filter by Tag Pool dropdown. To change a theme's name, select it in the list and revise the Name attribute. The following image shows the Themes with GenAI screen with a searchable list of themes; one theme is selected and its attributes are displayed.
Themes can be hidden using the Excluded Themes page. To hide a theme's visibility in reporting:
-
Use Setup to navigate to Reporting > Topic & Theme Tagging > Excluded Themes.
-
Select the tag pool with the theme to be hidden.
-
With a tag pool selected, use the first list under MemberTextanalyticsExcludedThemes to find the theme to be hidden, and use the arrows to move it to the second list. Themes in the second list are hidden throughout reporting.
-
To show a hidden theme, move it from the second to the first list.
The following image shows the Excluded Themes screen with all themes enabled.
Theme data is retained for 3 years in alignment with Medallia's standard data retention policy. To remove GenAI themes data from a tag pool, use Setup to navigate to Reporting > Topic & Theme Tagging > Auto Tagging Pools, select the tag pool, and disable the Process using Themes with GenAI option.
AI-generated data transparency and feedback
When AI-generated content or data is displayed in Experience Cloud, the generative AI icon displays with the module title or data, and a text disclaimer displays in the module's footer or after the relevant content.
When all fields in a module are populated with AI-generated data, the generative AI icon displays with the module title.
When a text field contains AI-generated data, the generative AI icon displays next to that text field's label, and the disclaimer displays after the relevant content. When only some text fields in a module are populated with AI-generated data, the icon is displayed on each text field but is not displayed with the module title.
When the TA Topic/Theme Summary module is configured to use GenAI themes, a summary describing the main theme displays. This summary displays irrespective of the Display data summary in subtitle module property. Summaries include a feedback prompt for the user asking whether the main theme was helpful. The user is able to answer "yes" or "no." On successful submission, a notification displays. If there is an error, the user is prompted to try again. Reloading the module displays another feedback prompt.
Summary feedback provides Medallia with examples of accurate and inaccurate records for future model training. Ratings and feedback affect neither the summary itself nor the system that generated it.
