Reporting

Reporting provide valuable insights into employee performance, engagement, and satisfaction. By analyzing these reports, organizations can identify areas of improvement and implement necessary changes. Furthermore, reporting allows for the tracking of these changes over time, ensuring their effectiveness.

EX has unique requirements, the most important of which is maintaining the confidentiality and security of employee data. For confidential EX programs, in addition to suppressing personal identifiable information, the following are used to protect the employees’ identity:

  • Blackout windows — Survey responses are not displayed in real-time but only once the survey expires. This prevents accidental identification of respondents based on the time the survey was taken. Additionally, the response date e is suppressed from the report.

  • Aggregate reports — All parts of a survey response, such multiple comments from the same response, are not displayed together to prevent accidental identification. Instead, scores and comments are only presented in an aggregate manner in reports.

  • Minimum sample size — Scores and comments are only displayed when a user has access to certain number survey responses (typically 5 responses) to help protect the employees’ identity. Medallia Experience Cloud does not show any data if there are less than those responses for a given score or segment/demographic group.

Blackout windows

The following are common reporting roll-out strategies to enable blackout windows.

Note: Although not typical, some companies may choose not to display participation rate information during the survey wave. Work with your company to determine the best roll-out strategy.

Roll-out strategy 1

This is the most common strategy for reporting roll-out.

Layer 112

  1. All users have access to participation rate (response rate) only.
  2. Responses are available to all users once the survey is closed.

Roll-out strategy 2:

To configure these Phase 1 and Phase 2 in reporting, set 2 different Event fields in the Invitation importer: Admin reporting date and Manager reporting date. Choose the appropriate reporting date for each role.

Layer 1123

  1. All users have access to participation rate (Response rate) only.
  2. HRBPs and Admins only have access to responses.
    Tip: Some companies may require that HRBPs and Admins get access to response information before managers. We recommend providing access to managers as soon as possible so that they can start actioning on the feedback.
  3. Responses are available to managers once the survey is closed.

Roll-out strategy 3

Layer 112

  1. HRBPs and Admins have access to response reporting in real-time. Managers have access to participation rate (Response rate) only.
  2. Responses are available to managers once the survey is closed.

Reporting date

The Reporting date field for EX is a K-field that applies to the following roles:

  • Admin — Admin's typically want to see data flow in immediately. To do so, change the Admin's Time field > Data Field Default Override to Response date, also shuffling over Reporting Date EX.

  • Insights

  • Executive

  • Onboarding

  • Employee — This should be removed if companies do not want employee to see data since there could be a viewable benchmark when launching the employee role during fielding.

Depending on the EX program, Reporting date may be set to different dates:

  • Employee engagement — Set to the survey Expiration date. Unless overridden at the dashboard or module level, this shows data whenever the survey Expiration date falls within the selected Time period.

    All Employee engagement time periods should be set to automatically increment each day (setting End date to +0/+0/+0). When the survey expires, the Reporting Date EX (expiration) date falls within the time period and roles are automatically launched. Change the time period after all roles launched to be set as a few days after the last role has launched.

    Participation dashboards are locked to use Creation date so all roles see participation immediately upon launch if they have access to the time period.

  • Employee journeys and moments — For Onboarding, set to the employee Hire date. Data is based on when employees were hired, not when they completed the survey. For example, an employee hired in January may take a survey in February, but this field allows to see the January hires together.

    For Exit, set to the employee Expiration date. Data is based on when the employee terminated, not when they completed the survey. For example, an employee terminated in January may take a survey in February, but this field allows to see the January terminations together.

  • Employee services and data — Set to Response date.

Engagement survey reporting release date and Response date

The Engagement survey reporting release date field for EX is a K-field that applies to the following roles:

  • Manager of managers
  • Frontline managers

Depending on the EX program, Engagement survey reporting release date may be set to different dates:

  • Employee engagement — Set to launch 14 days after survey closes, which may be adjusted for all waves or can be modified to be adjusted year over year. Unless overridden at the dashboard or module level, this shows data 14 days after the engagement survey's expiration date. For example, if the survey closes on February 1, the date should be set to February 15 (14 days after expiration).

    All Employee engagement time periods should be set to automatically increment each day (setting End date to +0/+0/+0). When the survey expires, the Reporting Date EX (expiration) date falls within the time period and roles are automatically launched. Change the time period after all roles launched to be set as a few days after the last role has launched.

    Participation dashboards are locked to use Creation date so all roles see participation immediately upon launch if they have access to the time period.

  • Employee journeys and moments — For Onboarding and Exit, set same as Reporting Date EX.

  • Employee services and data — Set to Response date.

The Response date field for EX is an Event field that applies to all other roles, which see the data almost immediately.

Tips and best practices

  • If the company is not sure when to launch a program or wants to stop existing launching, change all the Employee engagement Time periods from +0/+0/+0 to an exact date before the Reporting release date (Expiration date) or the Engagement survey reporting release date (14 days after expiration). This stops data showing on the dashboards until readjusted.

  • To launch a program for Executive role at the same time as the Manager role, change the role's Time field > Data Field Default Override to match the roles you want to launch with.

  • To modify the launch date, for example to be 7 days instead of 14, adjust the Engagement survey reporting release date code to be 7 days. Note the field is in milliseconds.

    Warning: if you have launched previous surveys already and you are increasing this value, proceed to the following point.

  • To modify the release date for different years, for example to be 10 days this year and 14 days last year, add an If statement in the code to set the date per program or wave:

    • Use an Event or K-field for the wave: 
      if (e_client_voe_wave_number === 1) {
  //Remove this object if e_client_voe_wave_number is a Date object. 
  var date = new Date(e_expirationdate);
  date.setDate(date.getDate() + 10);
  e_expirationdate = date;
}
    • Check the year on the expiration date and then set the dates for each year accordingly. 
      var year = getYear(e_expirationdate);
//Remove this object if e_expirationdate is a Date object. 
var date = new Date(e_expirationdate);

if (year === 2022) {
  date.setDate(date.getDate() + 10);
} else if (year === 2023) {
  date.setDate(date.getDate() + 14);
}

e_expirationdate = date;

Aggregate reports

Aggregate reports help prevent accidental employee identification. Presenting all parts of a response together provides hints that can unintentionally identify the respondent. For example, Jane, the only remote employee in Mark's team, responded these engagement survey questions:

  • Gave a score of 3 for “[Company] is a great place to work.”

  • Gave a of 1 for “My manager gives timely feedback that helps me improve my performance.”

  • Her first comment: “My manager does not take the time to support my personal development plan.”

  • Her second comment: "Benefits for remote employees are unsatisfactory.”

If Mark sees Jane's scores and comments together, he would know they come from her since she is the only remote employee.

Configuring aggregate reports

To configure aggregate reports:

  • Display aggregated scores and comments separately reports.

  • Use a separate Responses Feed or Comment Stream module for each comment field (even in Medallia Text Analytics modules). Enable randomization to ensure the same response is not the first or last comment in each module.

  • Do not display scores next to comments unless the score contextualizes the comment. For example, a Likelihood to recommend score and comment Engagement classification + comment.

  • Do not use standard Responses reports for roles that should only view aggregate reports.

Note that this does not mitigate the risk of using Filters or Data views in order to make the same identification.

Minimum sample sizes

Minimum sample sizes in reports ensure that scores and comments are only displayed when there are enough responses to obscure the identity of the respondents. The recommendation for minimum sample size is 5 responses. Users who have access to fewer responses do not see any response information in reports.

Configuring minimum sample sizes

To optionally configure sample sizes:

  • Minimum sample size can be set for a Data views. In Apps, there are specific data views that are used to apply a minimum sample size of 5 in reports. For example, select the Employee confidential access Data view for each module and each dashboard in reports to respect minimum sample size.

    Note: Do not use the minimum sample size setting on individual modules to achieve confidentiality. This does not display the ** marker and footer to contextualize why data is unavailable for the user.

  • To report on metrics without applying minimum sample size, such as Number of responses, Participation rate, etc., create a separate Data view with no minimum sample size setting and apply it to the modules.

Tips and best practices

  • Filters do not respect minimum sample size in any way. A specific combination of filters could result in a user being able to infer or see segment data about an individual response. This is typically referred to as backing out of data to identify an individual response.

  • If using Calculated Reporting fields (R-fields) with custom JavaScript, the App uses specific modules to maintain the minimum sample size. If any other module uses these fields, the minimum sample size must be applied in the field code itself.

  • For Data views:

    • Roles that should respect minimum sample size in reports should not use the default Data view (typically the company name) in reports since is typically shared across all roles (CX and EX). Consequently, sample size should not be applied on the default Data view. Instead, use the Employee confidential access Data view.

    • To prevent accidental exposure of feedback without minimum sample size applied, set the default Data view in the following way:

      • The Unit type should be Employee (or the relevant hierarchy setting to display the correct Topic set in Text Analytics.

      • Set Block data access to Yes. Create the Event field Block data access and do not set any value to it. This ensures that the default data access does not return any employee engagement feedback.

    • If the company requires to display certain metrics, such as Number of responses, without minimum sample size setting, create a separate Data view with no minimum sample size setting. Do not use the default Data view as this could result in accidental exposure of feedback without minimum sample size applied if the Data view does not change on a new module.

    • If your company requires multiple minimum sample size settings (for example, 5 for score-based reports and 10 for comment-based reports), create separate Data views with different minimum sample size settings. Apply the appropriate Data view for each module in reports.

    • Do not provide data access using All units. Organization hierarchy filtering is disabled for users who are assigned to all units in their data access permissions. Instead, use Unit type or a similar unit group for data access.

Bypassing Minimum sample size

Report configuration may allow users to bypass minimum sample size and infer scores and/or comments for a group of employees with insufficient sample. Note that Experience Cloud never explicitly shows personable identifying data or results below the minimum sample size. However, companies that are extremely sensitive to this type of issue may need to severely limit the filters and data views available to various roles.

Consider the following example:

There is a filter called Salary Band with the values: 6, 7, 8, 9, TE available to EX roles. When applying the filter Salary Band = 8, users do not see any results due to the minimum sample size setting of 5. However, they can infer the score provided by the employee with the Salary Band = 8 by:

Layer 1123

  1. Selecting all data:

    Sample Size: 9. % Favorable (4-5): 66.7

  2. Selecting only data for all salary bands except 8:

    Sample Size: 8. % Favorable (4-5): 77.5

  3. Finding the delta between the 2 results:

    Sample Size: 1. % Favorable (4-5): 0

To avoid bypassing minimum sample size settings:

  • Do not allow roles that adhere to minimum sample size to create Custom time periods. To do this, select the permission Disable Custom Timeperiods Option in Filters for the relevant roles.

  • Limit or completely remove filters based on roll-ups in the hierarchy and/or demographic segmentations to prevent users from bypassing the minimum sample size and inferring scores and/or comments.

  • For sensitive clients, use only My Directs and Indirects dashboards and remove the My Directs dashboard.