Track invitation open rates

Track email invitation open rates is a useful metric for determining respondent engagement. When used in A/B testing of two different invitation emails, you can get a rough idea of which email design is more successful getting a user to open it. Open rates are just one metric for evaluating the success of personalized surveys. Other metrics include:

Sampling (percent sampled, opted-out, and quarantined) rates — Indicate how many raw invitations get converted to survey emails. This can be an indication of the quality of the information collected about the respondents.
Delivery (delivered, bounced, undeliverable) — Indicate the quality of the email addresses.
Reading (open rate) — Indicate respondents are opening the HTML-format emails. A change in subject line might affect the engagement rate. A change in content might affect spam filters.
Surveying (started and completed) — Show how many survey takers start and complete the survey. Changes to the survey can improve or decrease this engagement rate.

Reading rates are the third step in the collection of survey program metrics.

To determine if a respondent opened an email,Iinvitation emails can include a small 1x1 pixel image reference at the bottom of the email. When opened, the email client retrieves the image from the Survey engine server, which records the open-event for the invitation email (only the first event is recorded for each email).

Invitation-open data in reports

Invitations reports and Health Check reports provide detailed information about the health of your invitations, including customer engagement. For more information, see Invitations reports and Health Check.

To supplement these reports, build Custom modules to analyze the open rates. For example, the following custom report shows the number of invitations sent, emails opened, and surveys opened each month. For more information, see Custom Module.

Sample custom report using open-invitation data

This sample report uses this code in the Custom module:

text
<!-- ON EXISTING COMPANIES, YOU WILL NEED TO CREATE ALL R-FIELDS REFERENCED IN THIS REPORT BEFORE IT WILL RENDER. PLEASE REFERENCE THE R-FIELD TEMPLATE LIBRARY WHEN DOING SO! -->
<analytics-report hide-empty-rows="true" hide-empty-columns="true"> 
	<chart type="COLUMN" data-labels-enabled="true" allow-decimals="false" decimals="0"> 
		<secondary type="LINE" data-labels-enabled="true" allow-decimals="false" decimals="0"/> 
	</chart>  
	<axis axis=""> 
		<optionbox-filter plugins="UNIT,TIMEPERIOD" time-field="e_creationdate"/>  
		<split field="e_company_survey_type_enum" values="4,6" filter="true"/>  
		<split field="k_company_global_invitation_tracking_enabled_yn" values="1" filter="true"/>  
		<timeperiod-dropdown-split/> 
	</axis>  
	<axis axis=""> 
		<field-split fields="r_company__invitations_sent" text="Invitations Sent"/>  
		<chart-axis-separator/>  
		<field-split fields="r_company__invitations_sent" text="Invitations Opened"> 
			<split field="a_invite_opened_yn" values="1" text="Invites Opened" filter="true"/> 
		</field-split>  
		<field-split fields="r_company_clckthrough_number" text="Surveys Opened"/> 
	</axis> 
</analytics-report>

Limitations

The email open rate is not completely reliable because it only works for a subset of the emails actually opened (and read) due to these limitations:

The feature only works for HTML-format templates. It does not work for respondents who opted to see text-only emails, or for email clients that do not support HTML.
The respondent's email client must be configured to allow the message content to show images retrieved from the Survey engine when the message is read. Some respondents choose to disable this feature, and some email clients completely block this action. This can also affect people who read their email on a mobile device, depending on the features of the email client.
Certain devices, browsers, and/or operation systems may prevent or block tracking the email open rate due to their privacy policies. However, this does not prevent Medallia Experience Cloud ability to report on survey open rates, which is the primary measure of how effective an email invitation is.

Interpret results

Before using the open rate to evaluate changes (such for A/B testing), it is important to establish a baseline rate, and then later to compare the change results as relative to the baseline. To do so:

At least 10,000 unique emails need to have been successfully delivered.
The measurements should be made over a period of two to three months.
Aggregate the response rates over a period of at least a week or longer. Fast blips in the data can be misleading due to inherent measurement errors. Ignore day-level variances and focus on longer calculations for better accuracy.

Important: Do not trust the rate as an absolute: this gives a rough idea of what is going on, but should never be held as an absolute metric. Use it to inform decisions, along with the metrics from other sources.

Implement open-rate tracking

To implement open-rate tracking of HTML invitation emails, include the $TRACK_INVITE_EMAIL_OPEN$ variable at the bottom of Distributions.

...
<tr>
<!--===================  Start email footer  ===================-->
  <td>
    <table ... >
      <tr>
        <td ...>
          <table ...>
            <tr>
              <td ...>
                Please do not "Reply" to this invitation.<br><br>
                ...

                $TRACK_INVITE_EMAIL_OPEN$

              </td>
            </tr>
          </table>
        </td>
      </tr>
    </table>
  </td><
/tr><
!--/////////////////////  End email footer  /////////////////////-->
...

When the Survey engine generates the Invitation email, it replaces this variable with an HTML <img> tag that references the tracking graphic and identifies the respondent, similar to this:

<img width="1" height="1" src="https://....medallia.com/?surveyid=0123456789abcdef" />

Restriction: There is another tracking variable ( $INVITE_OPENED_TRACKER_SECURE_URL$ ), but that one is for advanced, special-cases, and should never be used without approval from Medallia experts.

Screen capture showing the two "track" tags witth callouts emphasizing which to use and not use

Open-event metrics

When an email is opened and the tracking image is requested, the Survey engine records several System fields statistics about the event.

a_email_client_user_agent_brand: Brand name of the email client.
a_email_client_user_agent_browser: Browser name of the email client.
a_email_client_user_agent_browser_and_ver: Browser/version of the email client.
a_email_client_user_agent_device_name: Device name running the email client.
a_email_client_user_agent_os: Operating system running the email client. This is set when the invitation is opened and rendered.
a_email_client_user_agent_os_and_ver: Operating system and version running the email client.
a_email_user_agent_raw: Complete User-Agent request header sent by the email client.
a_email_invite_first_opened_date: Timestamp when the invitation was first opened. Supersedes a_invite_first_opened_date.
a_invite_first_opened_date: Deprecated. See a_email_invite_first_opened_date.
a_invite_first_opened_ip_address: IP address of the email client that opened the invitation, when the address is provided by the device.
a_invite_opened_yn: Whether or not the invitation email was opened. This fields gets updated to Yes when a survey is clicked, even if Tracking invitation open rates is not implemented.