How to configure speech recognition

Speech-recognition features allow you to accept spoken input from customers and agents during callbacks. With this feature, customers can provide spoken input rather than DTMF, and agents can use speech input to confirm they are ready to accept a callback. This article provides step-by-step instructions to configure speech recognition for both customers and agents.

Configuring Customer Speech Recognition (beta)

Note: Government Users - IVR speech recognition is not currently available. Please skip to Configuring Agent Speech Recognition.

If speech recognition is enabled for your Organization, you can configure Call Targets to accept spoken input instead of DTMF responses. When enabled, customers can still provide DTMF input if they choose, but they will also have the option to respond via voice to register an ASAP callback.

Note:
  • Customer speech recognition is in beta testing. Email us at beta@getmindful.com to request access.
  • This section covers the customer side of a call. Instructions for agent speech recognition are in the next section.

Quick access: Callback > Call Targets > General tab > Generalscreenshot of the caller input section of the call target page

The Caller Input setting determines whether customers can optionally respond via speech in certain scenarios, or only DTMF.

  • DTMF Only: Select this to disable customer speech recognition and require DTMF responses.
  • Speech with DTMF Fallback: Select the checkbox to enable customer speech recognition.
    • When selected, customers will still have the option of responding via DTMF, if they choose.

When Speech with DTMF Fallback is enabled, customers will be able to interact with certain parts of the Callback IVR experience with their voice, rather than DTMF input. In this case, the audio experience is also different, with alternative prompts used to guide customers through the IVR. To view the prompts used, see the Speech Prompts section of the Media Sets article.

Configuring Agent Speech Recognition

When Mindful Callback needs to prompt an agent to indicate they are ready to accept a callback, the default behavior is to instruct agents to press 1 when ready. As an alternative, you can configure your Call Targets to detect and recognize agent speech instead.

Quick access: Callback > Call Targets > General tab > Callback Strategy

On the Call Target screen, the Agent Answer Recognition radio button determines what is required from agents to indicate they are ready to accept a callback.

  • When set to DTMF, agents will be prompted to press 1 to accept a callback.
  • When set to Voice, agents will hear a customizable audio prompt instructing them to say specific words or phrases to accept.

Screen capture of a Call Target's details with the Agent Answer Recognition section highlighted

When Agent Answer Recognition is set to Voice, additional fields appear on the screen to customize the speech recognition experience. You can define keywords, set a required confidence level, or optionally choose to accept any speech that is recognized rather than listening for keywords.

Overview

This section covers the process of enabling and configuring speech recognition for your Organization and Call Targets:

  • Step 1: Enable speech recognition for your Organization
  • Step 2: Configure a Call Target to use speech recognition
  • Step 3: Configure keywords or phrases to detect
  • Step 4: Set a required confidence level
  • Optional: Allow any speech recognition
  • Recommended: Update the agent voice prompt for added keywords
Note:
  • The Agent Answer Recognition feature must be enabled by Mindful staff prior to use. Otherwise, the toggle switch and its related settings will not be accessible.
  • This setting is only available when First Party Called is set to Customer and Wait for live Agent is selected. Voice detection is not available for Agent First Call Targets.
  • This feature only detects speech in the English language at this time.
  • When thePriority Queue Timeout is set to Infinite, Agent Answer Recognition cannot be set to Voice. An error message will appear if you attempt to enable voice recognition with an unlimited timeout. This ensures that a timeout is always in place if a technical issue prevents agent voice recognition.
Important: It is possible to automate the process of agents accepting calls by automatically playing an audio prompt (containing voice or DTMF) from the agent side when an agent connects to a call. We recommend the following best practices to ensure the audio prompt is recognized by Mindful (especially in Avaya environments):
  • Record the audio file in a soundproof environment, such as a sound booth, to avoid background chatter.
  • Include one second or more of leading silence in the audio file.
  • If you are using DTMF (Agent Answer Recognition = DTMF), note that the audio quality is more important than the volume of the tone. Mindful has a DTMF tone audio file available (tone.wav) to assist with DTMF recognition.

Step 1: Enable Speech Recognition for Your Organization

The Agent Answer Recognition feature must be enabled by Mindful staff for your Organization prior to use. Otherwise, the radio button and its related settings on the Call Target screen will not be accessible. If you do not see the Agent Answer Recognition radio button on the Call Target screen (Quick access: Call Targets > Your Call Target > General tab > Callback Strategy), contact the Mindful Support team to enable it.

Step 2: Configure a Call Target to Use Speech Recognition

Quick access: Call Targets > Your Call Target > General tab > Callback StrategyScreen capture of the Agent Answer Recognition section with the Voice option selected

  1. On the Call Target screen, set the Agent Answer Recognition radio button to Voice.
  2. If you do not see the Agent Answer Recognition radio button, see Step 1 to enable the feature for your Organization.

To learn more about this setting, see the Agent Answer Recognition section of the Callback Strategy reference article.

Step 3: Configure Keywords or Phrases to Detect

Note: If you plan to allow any speech recognition rather than configuring a list of keywords, you can skip this step.Screen capture of the Agent Answer Recognition section with Recognition Keywords set to connect and hello
  • To add new keywords or phrases, type a word or phrase into the text field. Press Enter when finished to add the word or phrase to the list. Only one word or phrase can be added at a time, but you can continue to add words and phrases in the text field to expand the list.
  • To remove keywords, click the X icon next to any keyword in the list.

Step 4: Set a Required Confidence Level

Note: If you plan to allow any speech recognition rather than configuring a list of keywords, you can skip this step.

With the Confidence Level setting, you can set a threshold (as a percentage) for positively identifying that an agent has spoken a keyword. A default value of 80 is used for the Confidence Level unless you specify otherwise. This means that the speech recognition engine must reach 80% confidence that a keyword has been spoken before it identifies a match.Screen capture of the Agent Answer Recognition section with the Confidence Level set to default value of 80

  • To loosen the confidence requirement and positively identify agent speech more often, enter a value lower than the default of 80.
  • To tighten the confidence requirement for more precision, enter a value higher than the default of 80.

(Optional) Allow Any Speech Recognition

On the Call Target screen, you can use the Allow any Speech Recognition check box to instruct the speech recognition engine to ignore the configured Recognition Keywords and simply recognize any speech from agents. When selected, agents can respond with any speech when prompted to connect to a caller.Screen capture of the Agent Answer Recognition section with the Allow any Speech Recognition option un-selected

Tip: If you enable the Allow any Speech Recognition setting, consider updating the Prompt Agent to Connect (Voice) audio prompt to advise agents that they can respond with any speech to connect to a customer. By default, the audio file plays "To connect to the caller, say 'Connect'."

(Recommended) Update the Agent Voice Prompt for Added Keywords

When Agent Answer Detection is set to Voice, the Prompt Agent to Connect (Voice) audio prompt will be played to agents who answer a callback. This prompt plays "To connect to the caller, say 'Connect'" by default.

If you add or remove Recognition Keywords from the list, we recommend updating the Prompt Agent to Connect (Voice) audio prompt to advise agents of the complete list of recognizable words or phrases.

For example, if you added the word Ready to the list of Recognition Keywords, you might update the audio prompt to say "To connect to the caller, say 'Connect' or 'Ready'."Screen capture of a Media Set's details, focusing on the Connecting Agent section

For complete instructions on updating audio prompts, see How to manage audio files.