Creating an import specification

When you create a specification, it is best to provide a sample file matching the data-feed file to let the system create an initial mapping of the fields, and to see how the data are transformed. The Auto Importer analyzes the file, creates the initial specification, and attempts to map the data to fields in the database. Alternatively, you can provide the XML that is the source of the specification.

To begin creating a specification, click New at the bottom of the specifications list.

There are two options for creating a specification after clicking the New button:

• Use the XML editor to define the specification in XML. This is not recommended unless you are using an existing specification as the starting point. In that case, copy the existing XML and paste it into this editor.
• Analyze a feed-file to identify the input fields and create the initial specification. To analyze a file, click the

Sample feed-file

The sample file must match the structure of the eventual data-feed file that will be provided by the company. The most common file formats are supported, including CSV, pipe separated values, Excel (.xls, .xlsx), JSON, IBM SPSS (.sav) and various XML formats.

The file format is normally identified in the source. For example, in web feeds it is sent as the Content-Type header.

The file should have a representative sample of records and data to allow the analyzer to map the data, and for you to see samples of the results.

Restriction: The maximum size of the sample file is 2 MB (2,097,152 bytes).

Warning: Avoid using Excel files for production imports because large Excel files can cause large performance problems. Better is to convert the file to CSV first; CSV does not have the same size issues as Excel.

Surveys

The file contains raw invitations or survey results. The importer attempts to automatically map each file field to an existing A-Field or Q-Field. When a field cannot be mapped, Auto Importer suggests creating a new Q-Field based on the data type as detected in the file.

Restriction: Avoid using this processor for junking or unjunking records, instead use the JunkSurvey and UnjunkSurvey processors. For more details, see Junk survey records.

Warning: In pretend mode, some units created with the Org hierarchy processor are not available when the Surveys processor makes a reference to them, and the feed-file-processing history report shows an Auto Importer processing error. Process the file in active mode to avoid this error.

Episodes

The file contains episodes (or stays). The importer attempts to automatically map each file field to an existing E-Field. When a field cannot be mapped, Auto Importer suggests creating a new Event field based on the data type as detected in the file.

Org hierarchy

The file contains Units and their Unit groups.

The importer attempts to automatically map each file field to basic Unit fields (such as city, postal code, name, and identifier), Unit group fields (such as country/state, or other custom hierarchies), and Unit Data fields. When a field cannot be mapped, Auto Importer suggests creating a new Unit Data Field based on the data type as detected in the file.

Important: Before using this option, create the parent Unit Group nodes of the hierarchy using the Unit groups admin page.

Accounts

The file contains (User) accounts. The importer attempts to automatically map each file field to basic account fields such as username, e-mail, and password, and to permission fields.

When updating accounts, one, and only one of these two fields must be flagged for REQUIRED_USED_FOR_DUPLICATE_CHECK: either Username or Company Account ID. Note that neither of these fields are updatable, and attempting to update one when the other is the required duplicate check just ignores the new value, the rest of the matched record is updated.

Lookup table

A file containing a lookup table. The importer attempts to automatically identify the key field based on the uniqueness of the values it contains. If you need more than one key, mark them as REQUIRED_USED_FOR_DUPLICATE_CHECK as needed.

After clicking Save, the screen changes to allow you to edit the specification details.

JunkSurvey

Warning: Junked records may not be recoverable; assume data will be lost forever once junked.

Administrators should first consider other solutions, such as excluding records, for ignoring records to avoid the risk of irrecoverable data. For more information, see Exclude surveys.

The file contains one column — "Survey ID" — with the list of all survey IDs of the records that need to be junked. The importer updates existing records and changes their e_status field value to 7 JUNKED. This processor takes individual Survey IDs as inputs. Junked records are not visible in Admin Suite reporting tools. For information about this processor, see Junk survey records.

Note: Experience Cloud shows an error message when you try junking records that are already junked.

UnjunkSurvey

The file contains one column — "Survey ID" — with the list of all survey IDs of the records that need to be unjunked. The importer updates existing records and changes their e_status field value to 1 COMPLETED. Unjunked records are visible in Admin Suite reporting tools. For information about this processor, see Junk survey records.

ActionPlans

The file contains action plans that need to be imported, updated or deleted. The importer attempts to automatically map each file field to basic action plan fields such as action, created by, plan name, owner id, status, and id fields.

The file contains mandatory fields depending on the action. The available fields are.

Field name	Field ID	Type
Action	action	Enumerated: [ADD, UPDATE, DELETE]
Baseline Begins	baselinebegins	Date
Control Group Name	controlgroupname	String
Creator Id	creatorid	Integer
Description	description	String
Drivers	drivers	String
Enable Impact Analysis	enableimpactanalysis	Enumerated: [true, false]
End Test	endtest	Date
id	id	Integer
Impact Group Name	impactgroupname	String
Metrics Ids	metricsids	Integer
Name	name	String
Owner Id	ownerid	Integer
Setup Begins	setupbegins	Date
Start Test	starttest	Date
Status	status	Enumerated: [APPROVED, IN_PROGRESS, COMPLETED]
Suggested Actions	suggestedactions	String
Survey Ids	surveyids	Long_integer
Themes	themes	Integer
Topics	topics	Integer
Unit	unit	Unit

When updating action plans, id must be flagged for REQUIRED_USED_FOR_DUPLICATE_CHECK. Note that if id is empty, Auto Importer generates an id value. When an id value is provided, Auto Importer uses it for duplicate check, but ignores when it needs to assign it and generates a new id value.

The file can have have optional fields, like description, unit, surveyids, themes, and topics. It can also include fields related to impact analysis.

ActionPlansTasks

The file contains action plans tasks that need to be imported, updated or deleted. The importer attempts to automatically map each file field to action plan task fields.

The file contains 7 mandatory fields, and an optional description field:

Field name	Field ID	Type
Action	action	Enumerated: [ADD, UPDATE, COMPLETE]
Action Plan Id	actionplanid	Integer
Assignee	ownerid	Integer
Description	description	String
Due Date	duedate	Date
Name	name	String
Status	status	Enumerated: [NOT_STARTED, IN_PROGRESS, COMPLETED]
Task Id	taskid	Integer

When updating accounts, these two fields must be flagged for REQUIRED_USED_FOR_DUPLICATE_CHECK: either taskid and actionplanid. The importer attempts to automatically map each file field to basic action plan fields.

Properties

These properties appear after the initial specification has been saved.

Name

Name of the specification. This name matches the name in the XML specification, such as "My new import specification" in this specification element:

<import-spec inputType="JSON" name="My new import specification" description="An auto-generated spec to deal with Account

Category (optional)

Groups all definitions with the same Category value together, which can make them easier to find on the list. Definitions are ordered by Priority inside each category. Definitions with no Category value are listed underneath. Use the filter above the list to see just the definitions of the selected category.

The act of assigning a definition to a category that doesn't exist creates a new category. Note that category names are case sensitive.

Note: To delete or change the name of a Category, simply change the Category value in each of the definitions that belongs to it.

Open in Auto Importer UI link

Switches the editor to the Auto Importer UI, which is a more friendly way to edit the specification. Most of the features on this page are available in the UI editor. For features not available, use the Definition property to alter the XML description.

Definition

XML description of the specification. Use this editor to perform a detailed change to the specification. You can also use this to paste a specification copied from another importer.

XML documentation link

Opens documentation explaining the XML format used in the Definition property. It includes examples and explanations for each syntactic element, including descriptions of the JavaScript helper functions.

Issues

If there any problems with the analysis or with saving the specification, issue messages appear below the Definition box.

• Errors indicate a significant problem and they must be fixed before the specification can be used for importing.
• Warnings identify issues that should be fixed, but which will not stop an import process.  

When there are no issues, there are no messages.

Fields to be created on sync

When the import specification refers to fields that do not exist (such as after analyzing the sample file), each field appears in this list.

• Change a name, ID, or type in the XML Definition.
• Click Save to save the specification and update this list.
• Click Sync fields to create the fields.

This section only appears when there are fields to be created.

Enumerated values to be created on sync

When the import specification needs to create a new enumerated field or extend a pre-existing one with new enumerated values, each new value appears in this list.

• Change a name, ID, or type in the XML Definition.
• Each value has a sequence number, the value of which matches the order the value appears in the XML specification.
• Save to save the specification and update this list.
• Sync fields to create the fields.

This section only appears when there are enumerations to be created.

Sync fields

This button updates the database schema (Alternative Sets) to include the fields listed in Fields to be created on sync, and to add the enumerated values listed on Enumerated values to be created on sync.

Warning: Use care when choosing to click this button. This activity is hard to undo. You must manually delete each field and the respective Alt Sets.

This button only appears when there are fields or enumerations to be created.

JavaScript Library

JavaScript editor showing the contains of a <javascript-library> element when it exists in the XML specification.

Use the <javascript-library> node to define any functions that are used multiple times within the Auto Importer; this greatly enhances serviceability.

Important: Medallia Experience Cloud uses a JavaScript processing engine called Mozilla Rhino — also known as the Rhino Engine — to provide flexibility for scripting. Access to the Rhino Engine is restricted by a ClassShutter mechanism, which implements arbitrary logic to determine whether Rhino can access the class. When access is denied, Experience Cloud shows an error message. To access a class that is not currently allowed, contact your Experience Cloud expert. For more information, see JavaScript allow-listing.

Transforms

JavaScript editor for each a <javascript-transform> element in the XML specification.

Available fields

Lists the available fields for the <output-column-group> (or all of them, if more than one is used) referenced in the spec. Unsynced fields will not be shown here until Sync fields is clicked. The columns format is the same as Fields to be created on sync.

Clicking on the field's name opens the field definition for e-fields, q-fields, unit group fields and unit data fields. This is not available for permission fields, basic unit fields, basic account fields or other special fields (such as surveyid or episodeid).

Process a file on pretend

Uploads and processes a file in pretend mode: the file is processed but the records are not added to the database.

Click Choose File to upload and process a file.

The filename shown, if any, is either the file that was used to create the New import specification (loaded via one of the Analyze buttons), or it is the last file to have been pretend-processed.

Restriction: The maximum size of the pretend file is 2 MB (2,097,152 bytes). For testing purposes, you can upload a larger file with a Feed Pull, using a dummy Feed Pull with dummy login credentials so that you can test on pretend, overriding the 2-MB-file-size limitation. See the "Testing" topic in Feed Pulls for details.

The two basic use cases for pretend mode are:

• Check auto importer processing is working as intended. To validate import specification is correct.

• Validate the input file has the right contents.

Important: There are a few limited use cases where pretend processing needs to be run in production instances, we recommend that you avoid running pretend processing in production: carry out checks and validations in QA or sandbox instances.

Warning: In pretend mode, some units created with the Org hierarchy processor are not available when the Surveys processor makes a reference to them, and the feed-file-processing history report shows an Auto Importer processing error. Process the file in active mode to avoid this error.

Reprocess on active

Reprocesses the uploaded file and adds the records to the database. This option is only available after a file processed in Pretend mode was processed without error.

Last uploaded file

Name of the last uploaded file. Note that a file cannot be uploaded if the spec has errors.

Associated data file

The filename shown, if any, is either the file that was used to create the New import specification (loaded via one of the Analyze buttons), or it is the last file to have been pretend-processed.

Click the filename link to download the file. The name is a link only when the file is available to download.

Notes

Sorting:

• Don't rely on the sorting order of the lookup table to test sorting.

Mapping:

• Requiredness: OPTIONAL_USED_FOR_DUPLICATE_CHECK is valid only for Goal import, Lookup Table, and Weight imports.

JavaScript:

• Use the <javascript-library> node to define any functions that are used multiple times within the Auto Importer; this greatly enhances serviceability!

• Do not use /g (global) in regex expressions in the global <javascript-library> section of an import specification; in this case the /g is redundant and can cause performance issues.

• Use the discardRecord(message) function to reject records while logging clear and meaningful messages for each reason that a record gets discarded, e.g., discardRecord('Invalid unit type - expecting \'agent\' or \'store\' ');  Best practice is to do this within a <record-validation> node to reject records before they undergo further processing; this can significantly reduce import time.
• Be sure to consider null values carefully. To test for empty fields, use the hasContent() function which checks for three types of null: null, undefined, and empty string. For example, to test for a value in an "email" field":

if !hasContent (value)
   return ‘MISSING EMAIL’ ;

Limitations:

• Each instance can simultaneously run up to 5 processors in parallel. However, only one instance of a processor can run at a time. As such, if an instance has two processors, only two processors may run in parallel on that instance.

• Each processor can process only one file at a time. When a processor is fed multiple files, it processes them in order based on the file's last-modified timestamp.

For more information, see Importer tips.