Sampling guidelines

Sampling is the process of filtering raw Invitation files to identify invitees to be sent personalized invitation surveys. Sampling occurs after an Importer collects the feedback invitation data, and before an Invitation email is sent. Each sampling process has up to four stages:
  • Opt-out — Excludes invitees who have expressed intent to no longer receive survey emails.
  • Filters — Identifies the emails to include or exclude, usually based on business rules.
  • Quarantine — Avoids sending emails too frequently to the same invitee.
  • Quota — Attempts to achieve a target count of responses for various segments of the company.

    Raw invitations pass through Opt Out, Filter, Quarantine, then Quota before identifying the final survey emails

How sampling works

At any stage, if an Invitation email does not pass a filter, the record is not sampled and never considered again. By default, Medallia Experience Cloud automatically tests all raw invitations against the permanent opt-out list. No other sampling rules happen by default; all other rules are custom to the instance.

Sampling occurs in batches of 5,000 records, which are sampled against both the other records in that same batch and the ones in the database. For example, if in Batch A there two records (R1 and R2) that have the same IDs and share the same conditions, R2 gets sampled out as it is part of the same batch, without being checked against the database. However, if R1 and R2 have different IDs and conditions, they are checked against the database.

See Creating a custom sampler below for details about the steps for implementing a sampler.

Warning: Automatic sampling after import happens when there is a Trigger whose Feed Event is FILE_FEED. By default, Experience Cloud has at least one Trigger whose event is MANUAL, and that trigger is for testing. In production, there needs to be at least one for FILE_FEED.

Additionally, the Auto Importer specification must have a declared Sampling operation, otherwise, no sampling happens to the imported file.

For detailed information, see Sampling.

Tips and best practices

For surveys about a one-time event (such as a purchase)

  • Make sure the email is sent soon enough for the invitees to remember everything about the event, but not too soon, as there may be missed opportunities for feedback on problems that haven't cropped up yet.

Surveys about ongoing events

  • Avoid surveying large amounts of invitees in waves, because too much feedback coming in at once makes it difficult for account managers to close the loop and act on the data. Instead, a best practice is to flatten out these waves and send emails continually throughout the year.

  • An account should be surveyed 90 to 180 days prior to their anniversary/renewal. This allows for enough time to take action based on the survey feedback.

  • Typically, a relationship survey is sent once a year. This allows for enough time to close the loop and make changes that are visible to the invitees. If no improvements have been made between surveys, invitees do not see the value in responding again.

Relationship surveys (B2B)

  • It is important to include contacts at all buying influence levels of the company: decision makers, influencers, and end-users.

  • As a best practice, survey all invitees for a single account at the same time. Benefits of this approach include:

    • 360-degree view of an account at a particular point in time.

    • Ability to leverage the client's program team to drive response rates (following up with non-respondents).

    • Manageable closed loop process since focus is on holistic account.

  • Survey each account once every six months to one year, distributing different accounts across that period. This is known as Continuous Feedback Management (CFM), and it prevents peaks and valleys in survey volume, allowing frontline account managers to follow up with each of their accounts in a timely manner. Under CFM, sampling is most often distributed either based on contract renewal date or using dates flagged in campaigns in the company's CRM.

    When a company’s IT or organizational infrastructure is not conducive to CFM sampling, many companies turn to a quarterly wave-based approach where the accounts are divided into quarterly waves of survey emails.