Assignments
Assign QA Reviews automatically based on defined criteria.
QA assignments introduce automation to the QA Review process by automatically distributing and assigning QA Reviews to a set of reviewers. Assignments automatically identify appropriate interactions for review and distribute the workload among reviewers based on the configured criteria. This saves time for reviewers and their managers and eliminates selection bias.
In this guide, we will cover how to create and manage assignments.
Only users with roles that allow for the management of QA assignments will be able to create, edit, or archive assignments. Only users in roles that grant access to perform a QA Review can be selected as reviewers.
The QA assignments feature is only available to clients sending Agent Connect survey requests in an integration with a CRM system or other application. The feature is not available if you only send surveys manually. For help integrating your system, contact the Agent Connect Support team.
Create a new assignment
An effective assignment is built with five considerations:
Which types of interactions should be reviewed?
Which specific interactions should be reviewed?
Who will conduct the reviews?
How many reviews should be completed?
How frequently should the assignment reset the review goal?
To create a new QA assignment:
Open the QA menu in the top navigation bar, then select "Assignments".
On the QA Assignments page, click New Assignment.
Enter a unique Assignment Title.
The title will help to identify the assignment on the Assignments page. You might choose to categorize your assignments by team name, interaction type, or some other meaningful category.
Select reviewees
In the Reviewees section, determine whose interactions should be reviewed.
Select "Groups", "Teams", or "Employees" to determine the scope of the reviews in the assignment.
Click the Add... button beneath the selection to add groups, teams, or employees to be reviewed.
You can select more than one group, team, or employee, but only one of these types may be selected.
Note:If you select a parent group, all children of that group will be included, as well.
Data for each user in a group only goes back as far as the moment the user was added to that group. The user's data will still be available in other contexts (team and individual stats, for example). For group reporting, only data that was generated while the user was associated with the group will be included.
When a user is removed from a group, that user's data will no longer be included.
Reviewees who are a part of multiple groups will be counted only once towards the total, as a single reviewee participant. For example, if John is in the New Hire group and the Vendors group, and both groups are selected, John will appear once in the count of eligible employees.
Beneath the selected reviewees, select the user roles that should be included.
This step allows you to deselect administrator roles, managers, and other roles whose interactions should not be reviewed.
If needed, click Exclude Individual Employees to select individuals within the selected reviewee group whose interactions should not be reviewed.
Select reviewers
In the Reviewers section, determine who will perform the reviews.
To automatically assign reviews to the appropriate team leader, select "Respective Team Leaders". Note that this can result in reviews being assigned unevenly among reviewers, based on the size of each reviewer's team.
To manually edit the list of reviewers, select "Designated Reviewers", then click Add a Reviewer to add reviewers to the list. In this case, reviews will be distributed as evenly as possible among the reviewers. The appropriate team leaders will be pre-selected, but you can remove them if needed. If you do not see a user in the list, that user's role may not contain the appropriate permissions (see the note below).
If a selected reviewer does not meeting the following criteria, a warning message will be displayed:
Reviewers must have a user role with the Perform QA Review and Can Manage Teams permissions.
The Data Visible setting in the QA section of the reviewers' user roles must be wide enough to cover all reviewees in the assignment.
(Optional) Enable additional preferences
The Preferences section contains two optional settings:
Blind Reviews — Select this option to hide the team member name and photo from the interaction details during the review.
If the reviewee's name is included in the transcript or audio recording, it will still be accessible to the reviewer.
Assign Reporting Group Ownership — Select this option to hide the assignment details from all users except those in a selected reporting group.
Specify a cadence
You can select a weekly, monthly, or custom cadence. Weekly cadences begin on Monday and end on Sunday. Monthly cadences begin on the first day of the month and end on the last day of the month.
Assignments start at 12:00:00 UTC at the start of your cadence and end at 23:59:59 UTC on the last day of your cadence. For example, a weekly assignment cadence will start on Monday at 12:00:00 UTC and end on Sunday at 23:59:59 UTC. A monthly assignment cadence will start at 12:00:00 UTC on the 1st of the month and end at 23:59:59 UTC on the last day of the month.
If you select a custom cadence, additional fields will appear to allow you to select the start date/time and custom cadence, in days. A custom cadence can begin in the past, but the entirety of the cadence cannot be in the past. For example, you can configure a weekly cadence in the middle of the week, even though that cadence begins on a Monday.
Custom cadences start on the configured date and time, and repeat based on the number of days configured. You can set up your custom cadence to start at a certain time in your local timezone, if needed. Agent Connect displays help text to convert that local timezone into UTC so that globally distributed teams can understand the timing.
Select a default scorecard
The scorecard selected for the assignment will be applied to reviews in the assignment by default, but reviewers can change it as needed. If you would like the reviewers to manually select a scorecard for every review, leave this field empty.
Set the assignment goal
The assignment goal section allows you to specify exactly what kinds of interactions will be included:
Any interaction — Enter the number of interactions of any type to include.
Star rating — Enter the number of 1-3 star interactions and 4-5 star interactions to include.
Integrated channels — Enter the number of interactions to include for each available channel (phone, etc.).
Number of interactions with this tag — Enter the number of interactions with a specific tag to include.
The example image above would include one phone interaction, one email interaction, and two more random interactions of any type.
(Optional) Determine what happens when the assignment runs out of interactions
If the Review Similar toggle switch is enabled on the QA settings page, the Out of Interactions section will appear beneath the Assignment Goal section. You can select Allow Partial Match to instruct the assignment to pull in similar interactions that do not strictly meet the assignment criteria.
The narrower the criteria for assigning reviews, the more likely it is that Agent Connect will not have any reviews to distribute. If your reviewers do not have enough eligible interactions, you might consider the following:
Expand your assignment criteria.
Use the Allow Partial Match setting described above.
Just wait! Because we distribute reviews on a rolling time period, it is possible that more eligible interactions will occur and can be distributed to your reviewers during upcoming days.
Even if your criteria is wide open, it is still possible that Agent Connect does not have any interactions available to assign. As your team members handle more support interactions, more work will become available for your reviewers.
For more information see the Allow partial matches section later in this guide.
Configure filters
The final section allows you to refine the interaction criteria for the assignment even further. Excluding interactions can make the pool of interactions more meaningful by avoiding simple interactions with canned replies or other interactions that should not be reviewed.
In this section, you can choose to include or exclude interactions with the following attributes:
Interaction age (seven days by default)
Setting the interaction age ensures that only recent interactions are included, which makes reviews more timely and actionable.
Specific tags
Any custom interaction fields configured on the Integrations page (Settings > Integrations)
You can use any field being sent by your source system to generate custom filters. Common interaction fields may include contact reason, handle time, or topics (if integrated with Medallia Speech or Text Analytics).
To add an interaction field, first use the Select field dropdown menu to select an attribute being sent from your source system, then click Add. Next, configure the required settings for the new field in the list.
How are assignments distributed among reviewers?
Agent Connect has robust logic in place to distribute QA Reviews fairly when the total assignment goal is not evenly distributed among reviewers.
Reviews can be distributed unevenly in the following scenarios:
When the Respective Team Leaders checkbox is selected in the Reviewers section, reviewers with different numbers of team members can be assigned different numbers of reviews.
In this case, QA Reviews will be distributed unevenly, based on the size of each team.
When the total assignment goal cannot be evenly distributed among the number of reviewers (for example, three reviewers with a total assignment goal of 10 reviews), one of the reviewers must be assigned one more review than the others.
In this case, the one remaining QA Review will be assigned to a different reviewer in each Assignment period, ensuring that each reviewer takes a turn completing an extra review.
When the reviewers have different Data Scope in their user roles, some reviewers will have more access to reviews than others.
In this case, Agent Connect will assign the maximum number of reviews possible to reviewers with the most limited Data Scope. They will probably still have fewer reviews than other reviewers with wider Data Scopes, but the distribution will be as fair as possible.
In all cases, you can see the exact number of reviews that will be assigned to each reviewer by clicking the Reviewers link near the right side of the page when creating an Assignment:
Publish the assignment
When finished, click Create Assignment. It will then become available for reviewers.
Interactions that fit your parameters are selected randomly in order to eliminate bias in the review process. The assignment will continue to run automatically according to the configured cadence, and the progress for each reviewer will continue to reset according to the cadence until the assignment is archived. If edits are made to the participants during the assignment period, the assignment goal will update accordingly.
Assignment summary
To the right of the configurable settings, you will see a summary of what will be included in the assignment, including:
The number of employees and reviewers included
The number of reviews per employee
The exact number of reviews per reviewer
The total number of reviews for each round of the assignment (determined by the cadence)
Complete assignments as a reviewer
If you are a QA reviewer and you would like to review your assignments, click the QA entry in the top navigation bar, then select "Assignments" from the dropdown menu.
In the My Assignments section, you will see a list of assignments for which you are a selected reviewer. You will also see details about the assignment, the status, progress, a list of team members to be reviewed, and a list of reviewers.
From here, you can take additional actions, such as reviewing the assignment details or starting your reviews.
Click Review to get started. Agent Connect will automatically select an interaction that meets the assignment criteria for you to review. New reviews will be selected for you until you have met your review goal. If no more reviews are available, check back later to see if more interactions that meet the criteria have arrived.
If you need to skip a particular review, click Skip Interaction near the top right of the page.
You will have an opportunity to select a reason for skipping the interaction before moving on. If an interaction is skipped, it will be marked as ineligible for review and will no longer be included in the assignment for any reviewers.
Manage current assignments
On the QA Assignments page, click the ellipsis (...) to view additional options.
From the dropdown menu that appears, you can quickly view assignment details and history, duplicate the assignment, or archive it to make it inactive. Click Build Criteria to view the complete configuration of the assignment.
Edit participants
To update the reviewees and reviewers in the assignment, first click Details in any assignment's row, or select "Details and History" from the dropdown menu to open the details view for that assignment. Next, click Edit Participants. You will then be able to update the Reviewees and Reviewers sections of the assignment configuration, and changes will take effect immediately.
If you remove or deactivate reviewers, those reviewers' share of the remaining reviews within the assignment will be evenly distributed across the other reviewers.
If you deactivate a reviewee, the review target will update automatically. For example, if your assignment requires reviewing interactions for 10 reviewees across two teams, where the target is to complete two reviews per reviewee (a total of 20 reviews), and one reviewee on either team is deactivated, the new target for the assignment would be 18 reviews.
As you update participants, the Summary section will show the expected impacts of the change.
Participants will automatically update based on changes to a group or team included in the assignment.
If a reviewee is removed from an active assignment after reviews have already been completed, those completed reviews are retained. However, no new reviews will be generated for that reviewee.
If a reviewer is removed from an active assignment after completing reviews, their completed reviews remain part of the assignment. The reviewer will not be expected to complete additional reviews and the assignment will be removed from their assignment list.
Updates to assignment participants appear at the top of the assignment details page. Select the linked employee or reviewer count to open a side panel showing who was added or removed during the assignment period.
Edit assignment criteria
Other than the participants, existing assignments cannot be edited. If you need to make changes to other configuration of an assignment, you will need to duplicate the existing assignment, then make changes to the new copy.
How group updates will affect assignments
As you add or remove members from groups, Agent Connect will update affected assignments to reflect the current state of the group's membership.
For assignments based on groups or teams, any changes in the group/team membership will automatically change the number of reviews assigned to each reviewer. If a reviewer's assignment goal has changed, a yellow indicator will be displayed near the "Assignments" option in the QA menu for that reviewer.
After loading the Assignments page, the reviewer will see a note on the affected assignment detailing the changes. Hovering over either of the two indicators on this page will reveal what changed and how the change impacted the number of assigned reviews:
We recommend that reviewers check the Assignments page any time they see the indicator in the navigation menu to verify whether the change has increased their number of assigned reviews.
For more information about what changed in the underlying group or team, click in the Members column to view a slide-out panel showing any changes:
Archive assignments
You can archive an assignment by clicking the ellipsis icon (...) and selecting "Archive" from the dropdown menu. Archiving will disable the assignment for all of the selected reviewers. Past reviews completed as part of an archived assignment will remain active and accessible.
Track assignment progress
Progress for each active assignment can be reviewed in the assignment overview section, where you will see a count of completed reviews, remaining reviews needed to reach the target, and a progress bar.
Assignments will be shown in the My Assignments section if you are assigned as a reviewer. The total goal and progress of the assignment is based on the distribution of reviews within the assignment and the number of reviews you have completed for that assignment.
The All Assignments section may include a combination of your assignments and others for which you have visibility, based on the data scope of your user role.
If there are changes to the participants within an assignment (which can occur automatically by changes in group or team membership, participants, or user accounts), the goal for the assignment may change within the assignment period. We immediately process updates that apply to an assignment when there are participant changes to update the assignment goal, reviewees, and reviewers.
Assignment details
Within the assignment details, you can view high-level statistics such as the current progress, assignment duration, number of reviewers and reviewees, and the assignment goal. Additionally, you can identify team members who are champions (high achievers compared to others in that assignment) or those who may need additional coaching.
Skip reason reporting
When a reviewer skips an interaction, the assignment details will include the reason entered by the reviewer.
With this information, you can identify any trends in the reasons that interactions are being skipped and review the transcripts that were originally skipped, if needed.
If a skipped interaction actually does need to be reviewed, it cannot be reviewed within the same assignment. Instead, the reviewer must use the New Review button in the QA navigation menu or open the review on the Completed Reviews tab.
Tracking participant changes
Assignment details include reporting on past periods by listing the total number of reviewees and reviewers that were included in the assignment. You can hover over the number of employees or reviewers to view additional context and identify which users were added or removed during the selected period.
Assignment periods which have passed will include the total number of reviewees and reviewers at the conclusion of that period. The current period will reflect the most up-to-date totals of participants and will continue to update if there are further changes until the finalized counts are captured at the end of the period.
Allow partial matches
As mentioned previously, you can configure an assignment to relax the interaction criteria when no more matches can be found.
Why does it matter?
When an assignment has no more interactions to review, the reviewers will not be able to meet the 100% completion mark. Providing some flexibility for the assignment can help reduce frustration for reviewers and allow teams the opportunity to get closer to completing an assignment.
Allow Partial Match setting
To allow partial matches, a Company Admin must first enable the Review Similar toggle switch on the QA settings page (Settings > QA):
This will expose the Out of Interactions section on the assignment configuration page, where you can select the Allow Partial Match checkbox. With this done, the assignment will be able to relax the criteria and pull in interactions that do not strictly meet the requirements, but only after all of the full matches have been reviewed.
How does it work?
When an assignment runs out of eligible interactions based on the original criteria, a reviewer will see the following message:
When a user clicks Review Similar, they will be served an interaction that best matches the original criteria without being a perfect match.
Alternatively, a reviewer can return to main assignments page and wait for an interaction that matches the original criteria before continuing.
How does Agent Connect determine what interactions to include?
Agent Connect follows a two-step process to prioritize the next best interactions to serve:
Consult the age filter.
If "Exclude interactions for more than 7 days" was set, Agent Connect will pull interactions to serve from the seven days prior to that, as long as it is within six months ago.
Sort the new batch of interactions by the highest matching criteria from the assignment.
Example
Considering the following assignment criteria:
Star rating — 1-3 stars = 5
Integrated channels — 3 phone
Number of interactions with Tag — 3 with "shipping"
If an interaction fulfilled the following criteria:
Star rating — 1 star
Integrated channels — phone
And another interaction fulfilled the following criteria:
Star rating — 2 stars
In this scenario, the first interaction would be served for before the second, because the first matches two criteria from the assignment whereas the second only matches one.
