Rules

Rules define specific conditions for records to be deemed suspicious. Each rule has a relative importance (weight) which is considered when calculating the record's overall suspiciousness score (known as Total score threshold). If the overall score matches or exceeds the Total Score threshold defined for your company, the record is considered suspicious and ACE generates files of records daily. If the Total Score threshold is less than the threshold, the record is not considered suspicious, even if one or more individual rules were violated.

Consider these concepts connected with rules:

  • Multiplier — Number of points a rule adds (or subtracts) if it is triggered for a particular survey. A multiplier of 0 effectively excludes the rule. Click the number to enter a new number for the Multiplier and click Save.

  • Threshold — Number of surveys that share the same cookie, IP, etc. (depending on each rule), whose meaning may vary significantly depending on the rule. Click the number to enter a new number for the Threshold and click Save.

  • Stability and accuracy models — Rules 01, 02, and 06 support two model versions:

    • Stability model — Default option. Flags records at or above the configured threshold. By looking only at past data (relative to each record), it creates stable scores at the cost of higher accuracy. Use this option when the company wants to minimize fluctuations in scores and when these rules are not strong indicators for this customer.

    • Accuracy model — Recommended option. Flags records at or above the configured threshold and all records that contributed to reaching threshold. By also considering future data (relative to each record), it creates more accurate scores, at the cost of higher stability. Use this option when taking a strict stance and would like to remove any record that appears involved in gaming, and when these rules are very strong indicators of cheating for this company.

  • Rules Q-fields — Some rules, such as r007 and r010, require specifying the Feedback fields to use in the calculation.

  • Traffic analysis window (days) — Number of past days Polygraph checks to generate the Feedback protection report. The minimum is one day and the maximum is 30 days. This historical data considered for rules is relative to the inspected survey.

Rule 01 - Survey cookie match

Survey engine stores a randomly generated and unique cookie (named sEc) for the respondent when visiting browsers. Note that:

  • If the same respondent takes subsequent surveys using the same browser, the Survey engine reuses the same cookie value.

  • If there are different survey takers on the same Survey engine instance/domain, each survey taker has their unique cookie.

  • If a survey taker responds to surveys for different companies in the same Survey engine instance/domain, each survey uses a separate cookie.

Models

  • Stability model (default) — Other surveys with the same cookie value are looked for both in the preceding and subsequent dataset (until the end of the reported window). This ensures that results are stable, since they are based only on past information relative to the inspected survey, but as a result the first Threshold surveys are not flagged. For example, if Threshold is 3, the first three surveys with a matching cookie value are not flagged. The fourth survey (preceded by at least other three surveys) and the subsequent ones with a matching cookie value are flagged.

  • Accuracy model — Other surveys with the same cookie value are looked for both in the preceding and subsequent dataset (until the end of the reported window). This means that future information relative to the inspected survey is also considered, making results more accurate. For example, if the Threshold is 3 and there are four surveys with a matching cookie value, all of them are flagged. However, for this reason, results are unstable, since evaluating the same record multiple times with newer information available may yield different results.

Rule 02 - Shared IP address

Similar to Rule 01, the Survey engine uses the main IP address (IPv4) of the survey taker instead of the cookie. If more than one IP address is detected, the main IP address is the one that appears the most. This rule also has a threshold (how many other surveys with the same IP address need to be found for the inspected survey to be flagged), a multiplier (how many points does this rule add to the overall score if it triggers), and uses the same Traffic Analysis window.

Models

  • Stability model (default) — Other surveys with the same IP address value are looked for both in the preceding and subsequent dataset (until the end of the reported window). This ensures that results are stable, since they are based only on past information relative to the inspected survey, but as a result the first Threshold surveys are not flagged. However, the default stability model for IP addresses has a blind spot where other previous surveys happening in the same hour are not detected. This behavior is addressed in Stability model (v2.0).

  • Stability model (v2.0) — Same as Stability model (default), but without the blind spot.

  • Accuracy model — Other surveys with the same IP address are looked for both in the preceding and subsequent dataset (until the end of the Traffic Analysis window (days)). This means that future information relative to the inspected survey is also considered, making results more accurate. For example, if the Threshold is 3 and there are four surveys with a matching IP address value, all of them are flagged. However, for this reason, results are unstable, since evaluating the same record multiple times with newer information available may yield different results.

Rule 03 - Speedy response

This rule flags the inspected survey as suspicious when the survey taking duration is equal or less than the Threshold percentile of the responses of the same survey taken during the preceding Traffic Analysis window (days)). The survey taking duration results from the first and last interaction of the respondent with the survey (page load and clicking the Finished button).

Important: This rule always flags a fixed percentage of surveys; combine it with other rules to avoid potentially flagging a fixed percentage of non-fraudulent feedback. Additionally, this rule is only considered after 100 occurrences to avoid jittering surveys.

Note that the Threshold ranges from 0 to 1; for example, 45% displays as 0.45.

Rule 04 - Free email domain

Only for personalized survey. This rules flags the inspected survey as suspicious when the domain part of the email address stored in the Email field (e_email) matches any known free email domains, including email providers such as gmail.com and hotmail.com.

Rule 05 - IP Address near unit (Lat/Lng)

The Survey engine looks up the survey taker's main IP address (IPv4) in the GeoIP database to obtain an estimated latitude/longitude, which compares it against the latitude/longitude information for the unit (Unit ID) associated with the survey. This requires to previously set up the Unit geographic information in Medallia Experience Cloud. This rule flags an inspected survey as suspicious when the estimated distance between the survey taker and unit is less than the Threshold value (in kilometers).

Rule 06 - Feedback burst about unit

This rules flags the inspected survey as suspicious when it is completed in less than Threshold (in seconds) after another survey, for the same Unit ID (e_unitid).

Models

  • Stability model (default) — Other surveys with the same unit are looked for only in the preceding dataset (from the survey being inspected up to the Traffic Analysis window). This ensures that results are stable, since they are based only on past information relative to the inspected survey, but as a result the first survey is not flagged, only the subsequent ones. Additionally, since surveys are processed in chronological order and there could potentially be more than one survey being taken in the same second, running the report a second time may yield different results, when the surveys are evaluated in a different order.

  • Accuracy model — Other surveys with the same unit are looked for both in the preceding and subsequent dataset (until the end of the Traffic Analysis window). This means that future information relative to the inspected survey is also considered, making results more accurate. However, for this reason, results are unstable, since evaluating the same record multiple times with newer information available may yield different results.

Rule 07 - Field sum outlier

For each survey, the Survey engine sums the list of Feedback fields in the r007: q-fields box. This overall sum is compared against that of the preceding surveys until the Traffic analysis window (days) preceding days, for the same Spec ID. This rules flags the inspected survey as suspicious when the sum is in the top or bottom percentile of the population.

Percentiles are expressed with values between 0 and 1.0. For example, a threshold of 0.1 flags the top and bottom 10% of sum values. However, any value equal or greater than 0.5 flags all surveys.

Rule 07: Q-fields box with three Feedback fields.

Tip: Use this rule with values with normal distributions (known as bell curves). Similar to Rule 03, this rule always flags a fixed percentage of surveys; combine it with other rules to avoid potentially flagging a fixed percentage of non-fraudulent feedback. Additionally, this rule is only considered after 100 occurrences to avoid jittering surveys.

Rule 08 - Reporting application cookie

Similar to Rule 01, the Survey engine stores a cookie (named rAc) that is unique for the Experience Cloud users onto the visiting browser. This rules flags the inspected survey as suspicious when the requests also contain the Experience Cloud reporting application cookie for the company.

This rule applies both to Medallia Web reporting and Medallia Mobile. A survey taken on a mobile device may trigger this rule when:

  1. The cookie's expiration is set for the maximum integer, even if a user logs in once to Web reporting on a mobile device.

  2. Using Medallia Mobile, as the browser captures the cookie via Medallia Mobile login page, as it is the same as Web reporting.

Note: Medallia employees and partners can trigger this rule if they have the corresponding cookie in their browser when completing feedback. Voice of Employee programs may also trigger this rule (if those employees have access to the Experience Cloud instance).
Warning: For this rule to work, both the instance and the Survey engine must reside in the same domain, such as medallia.com. If they are located in different domains, the Survey engine requests only contain Survey engine cookies, not the Reporting application ones.

Rule 09 - IP address near unit (Postal)

The survey's main IP (IPv4) is looked up in the GeoIP database to resolve to an estimated zip/postal code and compared against the zip/postal information for the unit (Unit ID) associated with the survey. This requires to previously set up geographic information for the unit in Medallia Experience Cloud. If the estimated distance between the survey taker and unit is less than the Threshold value (in kilometers), this rule flags the inspected survey.

Note: This rule is based on USA zip and postal codes. For areas outside of the USA, use Rule 05.

Rule 10 - Bad Q-field value

If the Feedback field contains a specific value, this rule flags the inspected survey. Note that only numerical choice set values are accepted.

Rule 10: Q-field box with a Feedback field.

Rule 11 - Anonymous proxy

If the IP address of the survey taker is associated with a known proxy service (such as VPN providers or freely available services like Tor), this rule flags the inspected survey.

Rule 12 - Disposable e-mail domain

A disposable email address is a dynamically generated email address commonly used to avoid emails from the recipient of the address. Similar to Rule 04, if the domain of the email address in the Email field (e_email) matches a known provider of disposable email addresses, this rule flags the inspected survey.

Rule 13 - IP Address far from unit (lat/lng)

The survey's main IP (IPv4) is looked up in the GeoIP database to resolve to an estimated latitude/longitude and compared against the latitude/longitude information for the unit (Unit ID field) associated with the survey. This requires to previously set up geographic information for the unit in Experience Cloud. If the estimated distance between the survey taker and unit is more than the Threshold value (in kilometers), this rule flags the inspected survey.

Rule 14 - IP address far from unit (Postal)

The survey's main IP (IPv4) is looked up in the GeoIP database to resolve to an estimated zip/postal code and compared against the zip/postal information for the unit associated with the survey (Unit ID field). This requires to previously set up geographic information for the unit in Experience Cloud. If the estimated distance between the survey taker and unit is more than the Threshold value (in kilometers), this rule flags the inspected survey.

Note: This rule is based on USA zip and postal codes. For areas outside of the USA, use Rule 05.

Rule 15 - Public institution e-mail domain

This rule operates under the assumption that public institutions (usually owned, funded or controlled, at least partially, by governments) are a more reliable source of feedback. If the domain of the email address in the Email field is from a known public institution, this rule flags the inspected survey (reducing the score instead of increasing it, by using the Multiplier value).

Rule 16 - IP deny-list rule

This rule flags any survey that has its main IP in the list.

Blacklisted IP includes 69.89.31.226.