Benchmarking

Benchmarks provide a point of reference in your reporting data, allowing you to assess the differences between that benchmark and another set of data. For example, you could compare the performance of a unit to that of another unit, or to its own performance from a different time period.

Warning: Implementing benchmarking can be very complex. When implemented poorly, users might be confused with the options they see in the Benchmark dropdown. For example, if you implement several types of benchmarking, users might see benchmarks with similar names that provide benchmarking for very similar use cases, and be unable to determine which they should use. Make sure you understand your company's needs, and then plan the simplest benchmarking structure you can to meet those needs. Above all else, ensure that you test your implementation thoroughly before putting it into production.

Availability in reports

When you add a benchmark filter to a report, the Benchmark dropdown is available to users in the Control Panel.

To add benchmarks to dashboards, add the Benchmark filter to the Filters area of the dashboard in Medallia Admin Suite. For information about configuring dashboards, see Dashboards.

In custom modules, include the BENCHMARK plugin in the optionbox-filter element of the AA2 XML, as described in Benchmarks in custom reports, below. For more information about configuring custom modules in Admin Suite, see Custom Module.

Additionally, the Satisfaction Table, Satisfaction Compare, and Satisfaction Benchmark reports include the Benchmark filter automatically. For more information about configuring Satisfaction reports, see Satisfaction reports.

Time period benchmarks

Time period benchmarks allow you to benchmark a unit or a unit group against itself across time periods. For programs that are implemented less than a year ago, the lack of historical data can make time period benchmarks less useful initially. Over time, however, these benchmarks can be very handy when analyzing yearly trends. For example, a retail company might want to know at-a-glance how a store performed in the current holiday season compared to the previous year. All time period benchmarks appear in the PAST PERFORMANCE section of the Benchmark dropdown, after any unit benchmarks you define.

Your instance includes several time period benchmarks, which you can modify or supplement on the Timeperiods screen. To make a time period available in the Benchmark dropdown, in the MemberAdditionalKinds property choose Benchmark by moving it to the right-side box.

Assigning Benchmark in the MemberAdditionalKinds property

Note: Timeperiod benchmarks are not available for configuration in Rank Snapshot dashboard modules, which support benchmark intersections and unit groups. For more information about configuring Rank Snapshot modules, see Rank Snapshot.

In custom reports you might want to display a benchmark column next to the column for time period. For example, when looking at Current Year data you might want a benchmark column for Previous Year. Select that benchmark time period in the BenchmarkTimeperiod property on the Timeperiods screen. To display the selected benchmark time period in custom reports, when defining a timeperiod-split or timeperiod-dropdown-split element, set the benchmark-timeperiod attribute to true, as shown in the following example XML:

<analytics-report hide-empty-rows="false">
	<axis axis="">
		<optionbox-filter plugins="UNIT,TIMEPERIOD_WITH_REPORTING_DATE,QUESTION,BENCHMARK,CALCULATION" no-filter-plugins="TIMEPERIOD_WITH_REPORTING_DATE"/>
		<timeperiod-dropdown-split single-column="true" />
		<benchmark-split />
	</axis>
	<axis axis="Overall Experience">
		<calculation-dropdown />
		<question-dropdown-split text="My Property"/>
	</axis>
</analytics-report>

In addition to the benchmarks defined on the Timeperiods screen, Medallia Experience Cloud includes the Same Period Year Ago benchmark by default. If you do not want this benchmark available to users, check the Benchmark dropdown - Hide 'Same Period Last Year' parameter on the Optionbox Settings screen.

Benchmark Dropdown - Hide 'Same Period Last year' property

Unit group benchmarks

Unit group benchmarks allow users to benchmark against the unit groups to which they have access. Use this type of benchmarking for simple, almost static, location-based hierarchies (such as stores, hotels, car dealerships, and so on) where each unit group rolls up into another unit group in the organizational hierarchy.

Specific unit groups appear in the Benchmark dropdown for a user only if the following criteria are met:

  • The user has access to all of the active units that compose the unit group.
  • The unit group is available to the user. For more information, see Users.
  • Segment filter requirement — The Control Panel must contain a Segments filter (with the unit group field as a Primary Filter) to support hard-coded selections in the Benchmark list.

  • Timeperiod filter requirement — The Control Panel must contain a Timeperiod filter to support hard-coded selections in the Benchmark list for Timeperiod Benchmarks.

    For example, a custom module with the following optionbox-filter element in its AA2 will not see unit groups in the Benchmark dropdown because it lacks the required filter attribute value:

    <optionbox-filter plugins="COLUMNS,BENCHMARK,CALCULATION"/>

For example, If UnitGroupA contains Unit1, Unit2 and Unit3, but the user has access only to Unit1 and Unit 2, then UnitGroupA is not available in the Benchmark dropdown for that user. If the unit group contains no units, it is unavailable in the Benchmark dropdown to all users. Because of this data-access restriction, unit group benchmarks are not useful to users whose data access is limited to a single unit. Conversely, it can be useful for a Regional Manager who has access to all units that belong to his or her region, and who needs to benchmark units against the entire region.

Warning: Org hierarchy filtering (OHF) does not affect permission to use unit group benchmarks. Medallia Experience Cloud checks which units the user can access in the current hierarchy, and which units belong to which unit groups in the that hierarchy.

The All Units option in the Benchmark dropdown is available to all users, and selects all of the units to which the user has access. Medallia recommends hiding the All Units option from the Benchmark dropdown unless it is specifically needed for your company's roles, and user data access.

  1. Open the Company > Units > Groups Admin screen.
  2. Select a unit group.
  3. Turn on the HideFromBenchmark property, and then click Save.
    Note: Changing this property for a parent unit group does not affect child unit groups automatically. You must configure each unit group separately.
    HideFromBenchmark property

The Benchmark dropdown lists unit groups in the same order they appear on the Company > Units > Groups Admin screen. Medallia recommends listing unit groups in alphabetical order, but if you need a different order, consider adjusting the Priority of your unit groups on the Groups Admin screen.

Unit fields added as as unit reporting fields (in the MemberUnitReportingField property on the Company Settings screen) are not included in unit-group-benchmark report data filtered by unit groups or units. These fields are, however, included in report data filtered by non-unit-group filter selections (such as segments, time periods, searches, custom queries, and so on).

Extending data access for benchmarking

Occasionally, users need access to data outside their usual unit and unit groups. For example, the property manager of a San Francisco Orion hotel might need to benchmark that unit against all of his or her company's hotels in North America. Experience Cloud allows you to extend users' data access for benchmarking through benchmark intersections and extended benchmarking. While they perform similar functions, they must be configured and maintained differently.

Medallia recommends that you configure benchmark intersections instead of extended benchmarking wherever possible. Benchmark intersections provide the following benefits:

  • More flexible reporting — You can reference a benchmark intersection directly in benchmark-intersection-split elements in custom reports.
  • Automatic updates — After you configure a benchmark intersection, you do not need to update it as new unit groups are added to the hierarchy. Updating extended benchmarking is a manual effort. For this reason, Medallia recommends benchmark intersections for organization hierarchies that need frequent updates, such as agent or employee hierarchies. Extended benchmarking is more suitable for programs with simple, almost static, location-based hierarchies.
  • Flexibility in naming — The name of a benchmark intersection must be generic, since it will be shared by multiple users with different data access levels. For example, a My Region benchmark intersection is more flexible than region-specific extensions available through extended benchmarking. For example, assume the Property1 unit is a member of the BrandY unit group, and the Property2 unit is part of the BrandZ unit group. A regional manager has access to multiple units, and therefore multiple brands. With a benchmark intersection called Brand, when the manager selects Property1, the Brand benchmark calculates the aggregate score for BrandY. When the manager selects PropertyB, the Brand benchmark calculates the aggregate score for BrandZ. If the manager selects both Property1 and Property2, the Brand benchmark calculates the aggregate score for BrandY and BrandZ. If you were to implement extended benchmarking for this use case, the manager would have BrandY and BrandZ as two separate options in the Benchmark dropdown.
Warning: While the main use case for benchmark intersections is to calculate aggregate scores for the intersection of 2 or more unit groups, you can create a benchmark intersection with only one unit group. In that case, the benchmark intersection behaves similar to extended benchmarking.

Benchmark intersections

For any user, a benchmark intersection allows the user to benchmark his or her unit against the intersection of 2 or more unit groups to which that unit belongs. If you create a benchmark intersection with only a single unit group, users can benchmark against that unit group if they have access to any unit in that unit group.

Note: To be used in benchmark intersections, fields with an AlternativeSet value of Unit must be selected in the MemberUnitReportingField property on the Company > Profile > Company Settings screen.

These unit reporting fields are not included in benchmark-intersection report data filtered by unit groups or units. The fields are, however, included in report data filtered by non-unit-group filter selections (such as segments, time periods, searches, custom queries, and so on).

Because K-fields cannot be selected in the MemberUnitReportingFields property, you cannot use K-fields in benchmark intersections.

Restriction: The Control Panel must contain a Segments filter with the unit group field added as a Primary Filter. The Benchmark filter requires this Segments filter to support hard-coded selections.

Benchmark intersections behave differently, depending on whether OHF is enabled:

  • With OHF enabled (the default behavior), the groups associated with the units for benchmarking are determined by the groups mapped to records in the org hierarchy filtering a-field (a_org_hierarchy).
  • With OHF disabled, the groups associated with the units are determined by the intersection between the units and a group in the current hierarchy. If there is any intersection between a unit and a unit group, the user is given full access to that group.

To create a benchmark intersection:

  1. Open the Reporting > Report Helpers > Benchmark Intersections screen.

  2. Click New.
  3. Enter a Name that will appear in the Benchmark dropdown in reports.
  4. In the MemberUnitGroups property, choose the unit groups you want to be in this benchmark intersection by moving them to the right-side box.

    Consider using unit groups that are parents of other unit groups. For example, choose the Brand and Region unit group parents, and not the individual child unit groups, such as the North America region or the Orion brand. In that example, Experience Cloud automatically calculates the intersection for all Brand and Region permutations.

    Warning: Do not create a benchmark intersection using the root (All Units) unit group. That configuration can degrade system performance, and in some cases can cause your instance to stop working. If you need to benchmark against every unit, enable that on the Groups Admin screen by turning off the HideFromBenchmark property for the root unit. Doing this adds the All Units option when benchmarking, and does not harm system performance.
  5. Enter a Label, which you can reference in custom reports. For example, company_bench_int_myregionbrand.
  6. Click Save.
  7. In the MemberRoles property, select the roles you want to have access to use this benchmark intersection by moving them to the right-side box.

Just as with unit group benchmarks, data displayed by benchmark intersections are not filtered by any unit group or unit filter selections, but can be filtered by non-unit-group filter selections (such as segments, time periods, searches, custom queries, and so on).

Extended benchmarking

If benchmark intersections do not fit your needs (such as for a company with a mostly-static, location-based org hierarchy), consider using extended benchmarking.

Unit fields added as as unit reporting fields (in the MemberUnitReportingField property on the Company Settings screen) are not included in extended-benchmark report data filtered by unit groups or units. These fields are, however, included in report data filtered by non-unit-group filter selections (such as segments, time periods, searches, custom queries, and so on).

Note: To reduce the complexity of your implementation, Medallia recommends that you avoid extended benchmarking if you implement unit group benchmarking.

The following role-based capabilities provide access to extended benchmarking. For any role, assign no more than one of these capabilities.

  • Extended Benchmarking if access >=1 unit (recommended) — Allows users to benchmark their units against only the parts of the org hierarchy to which their units belong. For example, the HotelA unit is part of the BrandX unit group. The property manager of the HotelA unit wants to benchmark that unit against all other units in the BrandX unit group.
  • Extended Benchmarking (not recommended) — Allows users to benchmark their units against parts of the org hierarchy to which their units do not belong. For example, the property manager of the HotelA unit wants to benchmark that unit against all units in the BrandY unit group.

To configure extended benchmarking:

  1. Open the Company > Units > Bulk Groups screen, and then turn on the Include in extended benchmark property for each unit group that should have extended benchmarking enabled.
    1. You can do this manually for each unit group, but if you have multiple unit groups to update consider using the following bulk method: 
    2. In the Bulk dropdown, select Bulk Download (tabs) to export a spreadsheet containing the property settings for your groups.

      Selecting a Bulk Download of unit groups

    3. In your spreadsheet, change the Include in extended benchmark column from false to true for any unit groups that should have extended benchmarking enabled.
    4. Copy the cells on your spreadsheet. Ensure that you copy only the cells spanning the columns and rows of your data, including headers.
    5. In the Bulk dropdown of the Bulk Groups screen, select Bulk (tabs), and then paste your copied spreadsheet cells over the content in the text box.
    6. Check only the Update and Add properties, and then click Preview.

      Preview of a bulk-groups change

    7. Verify that your changes are in place, and then click Save.
  2. On the Company > Units > Groups Admin screen, ensure that the HideFromBenchmark property is turned off for any unit group that has extended benchmarking enabled.
  3. Enable roles to use extended benchmarking:
    1. Open the Company > Users > Roles > Roles screen.
    2. For any role that needs extended benchmarking, add one of the following capabilities by moving it to the right-side box of the MemberCaps property:
      • Extended benchmarking if access >=1 unit
      • Extended benchmarking
    3. Click Save.
  4. Test your implementation by using Switch Account to view reports as users with the affected roles.

    Because the InternalAdmin role has access to all units, you cannot test and troubleshoot benchmarking effectively using the InternalAdmin view.

Benchmarks in custom reports

For detailed information about building custom reports, see Custom reports.

To add the Benchmark dropdown to the Control Panel in custom reports, add BENCHMARK to the plugins attribute of the optionbox-filter element. For example:

<optionbox-filter plugins="UNIT,TIMEPERIOD,CALCULATION,BENCHMARK">

Additionally, you can use the following benchmarking XML elements to your custom reports. For detail, see the Analytics XML documentation available through the custom report editor in your Experience Cloud instance.

  • benchmark-split
  • benchmark-intersection-split
  • benchmark-previous-columns

The benchmark-split element adds Benchmark and Difference columns to your report, as shown in the following image. If you want to hide the Difference column, add hide-difference="true" to your split.

Benchmark and Difference columns in a custom report

The benchmark-intersection-split element allows you to reference a benchmark intersection, creating a table or a chart where you can compare units against that benchmark intersection. For example:

A benchmark-intersection-split element comparing a dealership against a region average

The benchmark-previous-columns element add a single column that computes a benchmark based on the 2 previous columns. For example:

A benchmark-previous-columns element computing a benchmark based on the 2 previous columns.