ACE overview
ACE manages the integrity of the survey process by identifying and flagging data anomalies and survey responses that might not represent actual feedback. To do so, it generates reports of suspicious records, and highlights units and any user associated with such activity. Managers can review each flagged survey response to decide whether it should be quarantined and removed from reports
ACE applies a set of up to sixteen rules to each record analyzed. If a specific rule deems a record suspicious, then that rule's relative importance is considered when calculating the record's overall suspiciousness score. If the overall score passes the threshold defined for your company, the record is considered suspicious and ACE generates files of records daily. If the overall threshold is not crossed, the record is not considered suspicious, even if one or more individual rules were violated.
This image shows the overall ACE workflow:
Polygraph
When the survey response enters Polygraph, it is analyzed against the enabled rules to obtain its Overall suspiciousness score, which is the sum of the Multipliers (weight) for each rule that a given record hits. If the survey's score is equal or greater than the Total score threshold, then the survey response is included in the daily report. The Threshold is configurable and can be set to 1 to include all records that hit at least one rule. The following image shows a the Configuration screen of a Polygraph instance, each rule with a set Multiplier and Threshold:
Polygraph runs every day at 12:00 am in the data center's timezone. The timezones for each data center are as follows.
| Data center | Timezone |
|---|---|
| sc4 | America (Los Angeles) |
| sea1 | UTC |
| can | UTC |
| fra1 | Europe (Berlin) |
| lon | UTC |
| sin1 | Asia (Singapore) |
| syd1 | Australia (Sydney) |
For more information, see Configure Polygraph.
SFTP
When a record passes the Total score threshold, Polygraph deems the record suspicious. Additionally, ACE generates a file of records daily, called Feedback protection report, which Polygraphs pipes into Experience Cloud for reports via a flat file in an SFTP folder. For more information, see Reporting screen.
Importer
To retrieve the daily Feedback protection report generated by ACE, create a Feed Pull that retrieves that report from the SFTP and tie it to the ACE Importer.
Importer
The ACE App comes with a default an ACE importer to update records called Survey update: Anti-Cheating Engine import.
The following code snippet represents the Importer definition used to update existing records. Information from ACE is appended to the completed survey records that ACE deemed suspicious.
<!--
NOTES:
The following is the Auto Importer which imports the Feedback Protection report generated by the Polygraph engine. It updates existing survey records to append the Feedback Protection data.
TODOS:
-
UPDATE LOG: [yyyy-mm-dd] <name>: <message>
-->
<import-spec inputType="CSV" columnsCheck="ALLOW_EXTRA_OR_MISSING_COLUMNS" name="SURVEY UPDATE: Anti-Cheating Engine Import" description="Import of suspicious records from Polygraph">
<input-column heading="survey_id" />
<input-column heading="first_started_at" />
<input-column heading="responded_at" />
<input-column heading="score" />
<input-column heading="r001" />
<input-column heading="r002" />
<input-column heading="r003" />
<input-column heading="r004" />
<input-column heading="r005" />
<input-column heading="r006" />
<input-column heading="r007" />
<input-column heading="r008" />
<input-column heading="r009" />
<input-column heading="r010" />
<input-column heading="r011" />
<input-column heading="r012" />
<input-column heading="r013" />
<input-column heading="r014" />
<input-column heading="r015" />
<input-column heading="num_related_by_cookie" />
<input-column heading="num_related_by_ip_address" />
<input-column heading="related_by_cookie_or_ip_address" />
<input-column heading="num_request" />
<input-column heading="mean_request_interval_seconds" />
<input-column heading="median_request_interval_seconds" />
<input-column heading="num_se_cookie" />
<input-column heading="has_own_se_cookie" />
<input-column heading="num_ra_cookie" />
<input-column heading="has_own_ra_cookie" />
<input-column heading="num_ip_address" />
<input-column heading="primary_ip_address" />
<input-column heading="is_anonymous_proxy" />
<input-column heading="is_satellite_provider" />
<input-column heading="primary_ra_cookie_decoded" />
<input-column heading="primary_ra_cookie_lookup" />
<input-column heading="continent" />
<input-column heading="country" />
<input-column heading="subdivision_1" />
<input-column heading="subdivision_2" />
<input-column heading="time_zone" />
<javascript-library><![CDATA[
function numberCheck(value) {
if(parseInt(value)>0){return '1';}
{return '2'};
}
]]></javascript-library>
<output-column-group pluginName="Survey" recordUpdateMode="UPDATE">
<output-column>
<input-column heading="survey_id" />
<target-field fieldId="surveyid" fieldName="Survey ID" requiredness="REQUIRED_USED_FOR_DUPLICATE_CHECK" type="LONG_INTEGER" />
</output-column>
<output-column>
<input-column heading="score" />
<target-field fieldId="e_bp_fp_suspiciousness_score_frc" fieldName="Suspiciousness Score" requiredness="OPTIONAL" type="FRACTIONAL" />
</output-column>
<output-column>
<input-column heading="r001" />
<target-field fieldId="e_bp_fp_r001_yn" fieldName="Rule 001 - Survey Cookie Match" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="r002" />
<target-field fieldId="e_bp_fp_r002_yn" fieldName="Rule 002 - IP Address Match" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="r003" />
<target-field fieldId="e_bp_fp_r003_yn" fieldName="Rule 003 - Speedy Response" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="r004" />
<target-field fieldId="e_bp_fp_r004_yn" fieldName="Rule 004 - Free Email Provider" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="r005" />
<target-field fieldId="e_bp_fp_r005_yn" fieldName="Rule 005 - Survey Completed Close to Unit (based on Latitude/Longitude)" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="r006" />
<target-field fieldId="e_bp_fp_r006_yn" fieldName="Rule 006 - Time Between Surveys" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="r007" />
<target-field fieldId="e_bp_fp_r007_yn" fieldName="Rule 007 - High Overall Score" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="r008" />
<target-field fieldId="e_bp_fp_r008_yn" fieldName="Rule 008 - RA Cookie Match" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="r009" />
<target-field fieldId="e_bp_fp_r009_yn" fieldName="Rule 009 - Survey Completed Close to Unit (based on Zip Code)" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="r010" />
<target-field fieldId="e_bp_fp_r010_yn" fieldName="Rule 010 - Suspicious Value Entered in Survey" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="r011" />
<target-field fieldId="e_bp_fp_r011_yn" fieldName="Rule 011 - Proxy Server" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="r012" />
<target-field fieldId="e_bp_fp_r012_yn" fieldName="Rule 012 - Disposable Email Domain" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="r013" />
<target-field fieldId="e_bp_fp_r013_yn" fieldName="Rule 013 - Survey Completed Far from Unit (based on Latitude/Longitude)" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="r014" />
<target-field fieldId="e_bp_fp_r014_yn" fieldName="Rule 014 - Survey Completed Far from Unit (based on Zip Code)" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="r015" />
<target-field fieldId="e_bp_fp_r015_yn" fieldName="Rule 015 - Government & Educational Institution Email Domains" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<javascript-transform><![CDATA[
return numberCheck(value);
]]></javascript-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="num_related_by_cookie" />
<target-field fieldId="e_bp_fp_num_related_by_cookie_int" fieldName="# of Surveys Related by Survey Cookie" requiredness="OPTIONAL" type="INTEGER" />
</output-column>
<output-column>
<input-column heading="num_related_by_ip_address" />
<target-field fieldId="e_bp_fp_num_related_by_ip_address_int" fieldName="# of Surveys Related by IP Address" requiredness="OPTIONAL" type="INTEGER" />
</output-column>
<output-column>
<input-column heading="related_by_cookie_or_ip_address" />
<target-field fieldId="e_bp_fp_related_survey_id_txt" fieldName="Related Survey IDs" requiredness="OPTIONAL" type="STRING" />
</output-column>
<output-column>
<input-column heading="num_request" />
<target-field fieldId="e_bp_fp_num_se_request_int" fieldName="# of Survey Requests" requiredness="OPTIONAL" type="INTEGER" />
</output-column>
<output-column>
<input-column heading="mean_request_interval_seconds" />
<target-field fieldId="e_bp_fp_mean_request_interval_sec_frc" fieldName="Mean Request Interval (sec)" requiredness="OPTIONAL" type="FRACTIONAL" />
</output-column>
<output-column>
<input-column heading="median_request_interval_seconds" />
<target-field fieldId="e_bp_fp_median_request_interval_sec_frc" fieldName="Median Request Interval (sec)" requiredness="OPTIONAL" type="FRACTIONAL" />
</output-column>
<output-column>
<input-column heading="num_se_cookie" />
<target-field fieldId="e_bp_fp_num_se_cookie_int" fieldName="# of Survey Cookies" requiredness="OPTIONAL" type="INTEGER" />
</output-column>
<output-column>
<input-column heading="has_own_se_cookie" />
<target-field fieldId="e_bp_fp_has_own_se_cookie_yn" fieldName="Has Unique Survey Cookie" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<mapping-transform fallbackMode="USE_DEFAULT" default="2">
<mapping-transform-pair from="TRUE" to="1" />
<mapping-transform-pair from="true" to="1" />
</mapping-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="num_ra_cookie" />
<target-field fieldId="e_bp_fp_num_ra_cookie_int" fieldName="# of Reporting Application Cookies" requiredness="OPTIONAL" type="INTEGER" />
</output-column>
<output-column>
<input-column heading="has_own_ra_cookie" />
<target-field fieldId="e_bp_fp_has_own_ra_cookie_yn" fieldName="Has Unique Reporting Application Cookie" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<mapping-transform fallbackMode="USE_DEFAULT" default="2">
<mapping-transform-pair from="TRUE" to="1" />
<mapping-transform-pair from="true" to="1" />
</mapping-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="num_ip_address" />
<target-field fieldId="e_bp_fp_num_ip_address_int" fieldName="# of IP Addresses" requiredness="OPTIONAL" type="INTEGER" />
</output-column>
<output-column>
<input-column heading="primary_ip_address" />
<target-field fieldId="e_bp_fp_primary_ip_address_txt" fieldName="Primary IP Address" requiredness="OPTIONAL" type="STRING" />
</output-column>
<output-column>
<input-column heading="is_anonymous_proxy" />
<target-field fieldId="e_bp_fp_is_anonymous_proxy_yn" fieldName="Anonymous Proxy" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<mapping-transform fallbackMode="USE_DEFAULT" default="2">
<mapping-transform-pair from="TRUE" to="1" />
<mapping-transform-pair from="true" to="1" />
</mapping-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="is_satellite_provider" />
<target-field fieldId="e_bp_fp_is_satellite_provider_yn" fieldName="Satellite Provider" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
<mapping-transform fallbackMode="USE_DEFAULT" default="2">
<mapping-transform-pair from="TRUE" to="1" />
<mapping-transform-pair from="true" to="1" />
</mapping-transform>
</target-field>
</output-column>
<output-column>
<input-column heading="primary_ra_cookie_decoded" />
<target-field fieldId="e_bp_fp_primary_ra_cookie_decoded_auto" fieldName="Reporting Application Cookie - Decoded" requiredness="OPTIONAL" type="ENUMERATED_AUTOINDEX" />
</output-column>
<output-column>
<input-column heading="primary_ra_cookie_lookup" />
<target-field fieldId="e_bp_fp_primary_ra_cookie_lookup_auto" fieldName="Reporting Application Name & Identifier" requiredness="OPTIONAL" type="ENUMERATED_AUTOINDEX" />
</output-column>
<output-column>
<input-column heading="continent" />
<target-field fieldId="e_bp_fp_continent_auto" fieldName="Continent in which survey was completed" requiredness="OPTIONAL" type="ENUMERATED_AUTOINDEX" />
</output-column>
<output-column>
<input-column heading="country" />
<target-field fieldId="e_bp_fp_country_auto" fieldName="Country in which survey was completed" requiredness="OPTIONAL" type="ENUMERATED_AUTOINDEX" />
</output-column>
<output-column>
<input-column heading="subdivision_1" />
<target-field fieldId="e_bp_fp_subdivision_1_auto" fieldName="Likely Region in which survey was completed" requiredness="OPTIONAL" type="ENUMERATED_AUTOINDEX" />
</output-column>
<output-column>
<input-column heading="time_zone" />
<target-field fieldId="e_bp_fp_time_zone_auto" fieldName="Time Zone in which survey was completed" requiredness="OPTIONAL" type="ENUMERATED_AUTOINDEX" />
</output-column>
<!-- Set flag that indicates record was present in Polygraph report -->
<output-column>
<constant value="Yes" />
<target-field fieldId="e_bp_fp_suspicious_record_flag_yn" fieldName="Suspicious Record Flag" requiredness="OPTIONAL" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="NAME" />
</target-field>
</output-column>
<output-column>
<input-column heading="first_started_at" />
<target-field fieldId="e_bp_fp_first_started_at_time" fieldName="Survey Start Date" requiredness="OPTIONAL" type="TIMESTAMP">
<timestamp-field-parse-options format="yyyy-MM-dd'T'HH:mm:ssZ" />
</target-field>
</output-column>
<output-column>
<input-column heading="responded_at" />
<target-field fieldId="e_bp_fp_responded_at_time" fieldName="Survey Response Date" requiredness="OPTIONAL" type="TIMESTAMP">
<timestamp-field-parse-options format="yyyy-MM-dd'T'HH:mm:ssZ" />
</target-field>
</output-column>
<output-column>
<now />
<target-field fieldId="e_bp_fp_suspicious_flag_time" fieldName="Suspicious Flag Date" requiredness="REQUIRED" type="TIMESTAMP">
<timestamp-field-parse-options format="yyyy-MM-dd HH:mm:ss" timeZone="America/Los_Angeles" />
</target-field>
</output-column>
<!--
REMOVE COMMENT TO HAVE RECORDS INCLUDED IN THE REPORT AUTO-EXCLUDED - update to correct auto exclude status
<output-column>
<constant value="4" />
<target-field fieldId="e_status" fieldName="Status" requiredness="REQUIRED" type="ENUMERATED">
<enumerated-field-parse-options mappingKey="SEQUENCE_NUMBER" />
</target-field>
</output-column>
-->
</output-column-group>
</import-spec>
Feed Pull
Configure the Feed Pull choosing the:
- Name — A distinctive name.
- Processing mode — Upload with pretend processing.
- Schedule — Daily.
- Pull method — SFTP.
- Auto importer processor — Survey update: Anti-Cheating engine import.
ACE set up
After Request ACE, follow these steps to set up ACE for your company:
-
Set up a Feed Pull to retrieve daily files created by Polygraph as described in Importer, above.
-
Work with your Medallia expert to customize reporting from ACE App, which includes some simple reports on ACE data, to meet your specific requirements. Be careful about which role(s) you provide access to as this information can be used to circumvent ACE.
-
Test and iterate on configured rules:
-
Pull sample reports using the initial ruleset and review the results to see if they align with expected results.
-
Create additional rulesets as needed and download ad-hoc reports from polygraph to compare the results.
-
Identify the ruleset that best matches the expectations and update the configuration for the ACTIVE ruleset to match.
-
-
Launch:
- Update the Feed Pull to process on ACTIVE.
- Monitor to ensure files process as expected.
- Review results and validate records being flagged are in line with client expectations.
Note that ACE rules only works for data coming after the date it was activated. For example, if ACE was activated on March 1, and you may want to test the rules from January 1 to May 31 (current date), the rules do not apply to data from January 1 to February 28.
