Reports and alerts

Medallia Experience Cloud reports for ACE visually show the locations (if any), spread, frequency, and validity of the output from Polygraph, and they are part of the ACE app.

Reports and dashboards

ACE reports are powered by fields created from the deployment of the Anti-Cheating Engine add-on App.

HealthWatch: Anti-Cheating Engine report

The HealthWatch: Anti-Cheating Engine report provides a detailed view of suspicious survey records received by your company, along with additional indications of suspicious behavior. For more information, see Health Check.

The Is cheating an issue? section provides summary information about suspicious activity over time. It includes at-a-glance modules that shows the percentage of recent suspicious records and the recent average suspiciousness scores. This section also includes a line chart to help you see the trend of these metrics over a longer span of time.

Is cheating an issue? displaying several modules.

The Where is there suspicious activity? section includes a plot graph, with each dot representing a unit (which, in term, represents a location). Each unit is plotted by its average suspiciousness score and the number of suspicious records submitted for that unit. Hover over a dot to see more detailed data for that unit.

This section also includes a table with unit-based suspiciousness data. Click a unit in the left column of the table to open the Responses Feed module, filtered for that unit.

Where is the suspicious activity? includes two modules: Suspicious activity by unit and Suspiciousness Scores by unit.

The What rules are triggered? section displays which Rules and the number of times they were triggered in a set period of time.

This section also includes a table with unit-based rule data. Click a unit in the left column of the table to open the Responses Feed module, filtered for that unit.

What rules are triggered? includes two modules: Rules triggered over time and Rules triggered by unit.

Supplemental Responses Form

The Anti-Cheating Engine Add-on app includes a module you can append to the Responses Form report, providing a look at potentially suspicious survey response data. For more information about configuring Responses reports, see Responses reports.

In Admin Suite, open the Responses Form report you want to supplement.
Click Add Module, click Add Existing Module, and then select ANTI-CHEATING RESULTS.
Verify that the module displays the appropriate field groups and fields. The module should require no additional configuration.
To add a new Field group, click Add Field Group.
To change the fields in the Field group, click Select Fields to add fields, or click Close next to a field to remove it from the group.
Click Save to save the module, and then click Close icon. to close the edit panel for that module.
Click Save to save the parent Responses Form report.
When you are ready to make the modified report available to users, publish the report and module.

Fields

The following Event fields are created from the deployment of the Anti-Cheating Engine add-on App:

# of IP addresses: The number of IP addresses associated with the survey taker.
# of Reporting application cookies: The number of unique Medallia Web reporting cookies associated with the survey taker. A Web reporting cookie is stored by the Web reporting when a survey taker logs in. Taking a survey for a unit from the same browser used by an account that has access to that unit in Web reporting is a strong indication of cheating. The cookie is not set if the browser is in a registered Medallia IP address (such as through VPN), the site being accessed is not a production system (such as sandbox instances), or if the person logging in has internal-admin access to the system.
# of Survey cookies: The number of unique Survey engine cookies associated with the survey taker.
# of Survey requests: The number of times a survey taker loaded any page in the survey
# of Survey related by IP address: The number of times an IP address associated with the record has been used to take a survey, within 2 days before and 2 days after the record. A high number is an indicator of suspicious activity as it suggests one survey taker is submitting multiple responses.
# of Survey related by survey cookie: The number of survey cookie matches, which are for surveys taken from same browser from which the user opened Experience Cloud. This is an indicator of suspicious activity, and can be tracked to a specific user.
Anonymous Proxy: Indicates whether the survey taker is using an anonymous proxy or not.
Continent in which survey was completed: Continent in which survey was completed, for example North America.
Country in which survey was completed: Country in which survey was completed, for example United States.
Likely region in which survey was completed: Likely Region (Level 1) in which survey was completed, for example California.
Likely sub-Region in which survey was completed: Likely Region (Level 2) in which survey was completed, for example Croydon.
Has unique Reporting application cookie: Whether or not a survey taker has a unique Web reporting cookie relative to other surveys on the instance in the past 180 days.
Has unique Survey cookie: Whether or not a survey taker has a unique Survey engine cookie relative to other surveys on the instance in the past 180 days.
Mean request interval (sec): The mean interval between requests (in seconds).
Median Request Interval (sec): The median interval between requests (in seconds).
Primary IP Address: The IP address appearing most frequently among those associated with the survey record. If the device accessed the Survey engine through a proxy or from behind a corporate firewall, this is the IP address of the proxy or firewall. If survey taker starts a survey on one device and completes it on another one, only the first IP address to touch the survey is considered as Primary.
Related survey IDs: A pipe-separated list of the most recent 10 survey IDs that are related by either cookie or IP address.
Reporting application cookie - Decoded: The Medallia Web reporting user ID associated with the Web reporting cookie for the survey.
Reporting application name & identifier: The Medallia Web reporting user name or email address associated with the Web reporting cookie for the survey.
Survey Response date: Date the survey was responded based on the Feedback protection report.
Survey Start date: Date the survey was started based on the Feedback protection report.
Suspicious flag date: Date a record was flagged as suspicious
Suspicious record flag: Whether or not a record was flagged as suspicious.
Suspiciousness score: The Overall suspiciousness score for each record. The suspiciousness score is a weighted sum of the scaled scores for each rule. The greater the number of rules that returned values considered suspicious (values greater than 0), the higher the suspiciousness score.
Timezone in which survey was completed: Time zone in which survey was completed, for example America/Los_Angeles.
Rule 001 - Survey cookie match: Field corresponding to Rule 01 - Survey cookie match.
Rule 002 - IP address match: Field corresponding to Rule 02 - Shared IP address
Rule 003 - Speedy response: Field corresponding to Rule 03 - Speedy response
Rule 004 - Free email provider: Field corresponding to Rule 04 - Free email domain.
Rule 005 - Survey completed close to Unit (based on Latitude/Longitude): Field corresponding to Rule 05 - IP Address near unit (Lat/Lng).
Rule 006 - Time between surveys: Field corresponding to Rule 06 - Feedback burst about unit.
Rule 007 - High overall score: Field corresponding to Rule 07 - Field sum outlier.
Rule 008 - RA cookie match: Field corresponding to Rule 08 - Reporting application cookie.
Rule 009 - Survey completed close to Unit (based on Zip code): Field corresponding to Rule 09 - IP address near unit (Postal).
Rule 010 - Suspicious value entered in survey: Field corresponding to Rule 10 - Bad Q-field value.
Rule 011 - Proxy server: Field corresponding to Rule 11 - Anonymous proxy.
Rule 012 - Disposable email domain: Field corresponding to Rule 12 - Disposable e-mail domain.
Rule 013 - Survey completed far from Unit (based on Latitude/Longitude): Field corresponding to Rule 13 - IP Address far from unit (lat/lng).
Rule 014 - Survey completed far from Unit (based on Zip Code): Field corresponding to Rule 14 - IP address far from unit (Postal).
Rule 015 - Government & educational institution email domains: Field corresponding to Rule 15 - Public institution e-mail domain.

Alerts

You may create Alerts to follow up on records being flagged as suspicious by ACE. Alerts are generated in near real-time as surveys are completed while they are in Status (e_status) 0 ("COMPLETION_PENDING").

Consider the following when setting up Alerts:

ACE can hide records from end-users days after the surveys are completed.
Alert Alert workflows may be interrupted if ACE removes or hides records while alerts are Open. For example, if a respondent replies to a Feedback response email after a survey has been hidden by ACE.
You may add condition the ACE Importer to prevent records with alerts from being removed, which can also be applied selectively to specific alerts. For example, detractor but not promoter alerts.