JSON Pull Web Service Data Adapter

The JSON Pull Web Service Data Adapter enables you to share data held in your existing systems with MXO. Typically this data is used in Eligibility Rules.

Important: MXO supports connecting to web services that implement the HTTPS (not HTTP) protocol with a valid certificate, listening on a port in the range 1025-65535.

Example

A mobile phone company stores data about customers on an in-house CRM. This includes data about products a customer already owns. The company want to expose this data to MXO to ensure that whenever a customer visits their website they are presented with personalized content based on products they already own.

For example, a customer who owns an iPhone 12 is presented with accessories specific to that phone rather than accessories for all available phones.

Set up your Web Service

MXO interacts with external sources using a client web service that collects data from, for example, your CRM and provides it to the data adapter. Direct access to your data repository is not required.

General Requirements

This web service must be built on RESTful concepts, and conform to the following requirements:

  • Must accept JSON requests and responses. The response type is determined by the ACCEPT header in the response.
  • Must use the GET HTTP verb.
  • Must have a valid SSL certificate installed. This ensures communication occurs over a secure channel using HTTPS. Note that the certificate cannot be self-signed.
  • Must be exposed to the web. For example, by adding the service to your external DNS.
  • Must add details of the MXO server's outgoing IP address range to the service's whitelist, if communication with the service is to be limited to a specific range of IP addresses. We also advise whitelisting the IP address of the main company office, to assist with debugging.
  • Must support either OAuth 2.0 Client Credentials or Basic authentication.

Multiple Key Requirements

If you are using multiple keys in your environment, the web service must be configured to:

  • Accept the total number of keys you want to use to lookup customer data.
  • Prioritize the order in which keys are resolved.
  • Handle any blank keys that may be passed across from MXO.

Target URLs

When configuring the JSON Pull Web Service Data Adapter to work with your web service, you must provide a Target URL. This is the URL required by MXO to access the service. The target URL must contain a placeholder for the key or keys to your target service. These keys can be:

  • Any captured attribute, selected from the Select the attribute to use as a key drop-down, or
  • Any key attribute configured by you for your Space.

At runtime, all attributes are replaced by the values for an individual customer profile in MXO.

The JSON Pull Web Service Data Adapter supports the use of port numbers in Target URLs. We recommend, however, that port numbers are only used in URLs for testing purposes and not in a production environment.

Example Target URL (single key used for lookup)

https://mycompany.com/myRESTservice/customerdata/

Example Target URL (multiple keys used for lookup)

https://mycompany.com/myRESTservice/customerdata/customer?crmid=&loginid=

Points to Note

  • The attribute selected in the Select the attribute to use as a key drop-down is referenced using the format . Note that any captured attribute held in the Customer Metadata data adapter can be used.
  • Any key attributes in the target URL must be referenced using their API name in MXO.
  • Attributes can be located anywhere in the URL.
  • The data returned by your REST service, in response to a call from MXO, must be in JSON format.

Create and Validate your Schema

You must provide a valid JSON schema for the data that will be sent to MXO by your web service.
Note: The JSON Pull Web Service Data Adapter currently supports JSON Schema Draft 4.

Example schema

{
   "$schema":"http://json-schema.org/draft-04/schema#",
   "id":"http://jsonschema.net",
   "type":"object",
   "properties":{
      "address":{
         "id":"http://jsonschema.net/address",
         "type":"object",
         "properties":{
            "streetAddress":{
               "id":"http://jsonschema.net/address/streetAddress",
               "type":"string"
            },
            "city":{
               "id":"http://jsonschema.net/address/city",
               "type":"string"
            }
         }
      },
      "phoneNumber":{
         "id":"http://jsonschema.net/phoneNumber",
         "type":"array",
         "items":{
            "id":"http://jsonschema.net/phoneNumber/0",
            "type":"object",
            "properties":{
               "location":{
                  "id":"http://jsonschema.net/phoneNumber/0/location",
                  "type":"string"
               },
               "code":{
                  "id":"http://jsonschema.net/phoneNumber/0/code",
                  "type":"integer"
               }
            }
         }
      }
   }
}

To create and validate your schema, we suggest building your REST service and calling that service in a browser.

Generate your schema by copying the output returned in the browser into a draft 4 JSON schema generator tool. This creates a valid JSON schema for you.

When you create your JSON Pull Web Service Data Adapter, MXO carries out further checks on the validity of your schema, using a parser, as part of the test process.

If MXO encounters any errors during this process, information about those errors is returned as a log. If you require any assistance with correcting these errors, contact MXO Technical Support.

Supported Authentication Methods

The JSON Pull Web Service data adapter supports the following authentication mechanisms for calling the web service:

  • HTTP Basic
  • OAuth 2.0

HTTP Basic

HTTP Basic provides the simplest method for authenticating communications between your web service and MXO.

HTTP Basic passes a username and password over the network as unencrypted base64 encoded text, and should only be used with HTTPS.

The HTTP Basic authentication method is based on the principle that each HTTP request from the data adapter includes an authorization header that specifies the credentials required to access the secure data.

A full description of the HTTP standard, is available at the following link: HTTP Authentication: Basic and Digest Access Authentication.

Example HTTP Authorization Header

An authorization header is constructed, as follows:

  1. Combine the username and password into a string username:password.
  2. Encode the resulting string using the RFC2045-MIME variant of Base64.You are not limited to 76 characters per line.
  3. Add the authorization method and a space, for example, Basic before the encoded string.

For example, if the username is 'Aladdin' and the password is 'open sesame' the header is formed, as follows:

Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==

OAuth 2.0

OAuth is an open standard for authorization, that provides a simple way to both publish and interact with secure, protected data. OAuth provides client applications with secure delegated access to server resources on behalf of the resource owner. It also specifies a process for resource owners to authorize third-party access to their server resources without sharing their credentials.

A full description of the OAuth2 standard is available at the following link: The OAuth 2.0 Authorization Framework.

OAuth 2.0 defines four main grant types:

  • Authorization code
  • Implicit
  • Resource owner password credentials
  • Client credentials

Both the JSON Pull and JSON Push Web Service Data Adapters use the client credentials grant type.

When configuring the Data Adapter, you must provide:

  • Username (Client_id)
  • Password (Client_secret)
  • URL used to obtain an OAuth 2 access token

Scope

The JSON Push Web Service adapter offers an optional Scope field you can use to specify the scope of the access request when using OAuth 2.0 authentication. Scope is an optional text field. If you do not define a scope, the adapter does not send this field.

For more information about setting up the client credentials grant type, see the Client Credentials Grant and Action Token Scope documentation.

Configure the Adapter

  1. In MXO, choose Collect.

  2. Select External Data, click Create.

  3. Choose JSON Pull Web Service as the type of Data Adapter you want to add.

  4. Enter a name for the Data Adapter you are adding. Using a descriptive name makes it easier for you and your team members to know when to use the Data Adapter.

  5. Enter a brief description of the Data Adapter you are adding and when it should be used.

  6. Enter the configuration details that define the adapter's behavior.

    FieldDescription
    Select the attribute to use as a keySelect the captured data attribute you want to use as the key for looking up data. Captured data attributes are stored in the Customer Metadata Data Adapter.
    Use as Action SourceSpecify whether or not to use this data adapter as an Action Source for a Dynamic Actions template. You can select: - Not set, which acts as if No is selected, - Yes, use this data adapter as an Action Source for a Dynamic Actions template, or - No, do not use this data adapter as an Action Source for a Dynamic Actions template.
    SchemaJSON schema that defines the service. This should be copied and pasted based on the structure of the data returned by your organization's web service.
    Target URLCustomer data URL. Example (using single key for lookup): https://mycompany.com/myRESTservice/customerdata/4756097 Example (using multiple keys for lookup): https://mycompany.com/myRESTservice/customerdata/customer?crmid=4756097&loginid=30dd879c-ee2f-11db-8217-0800200c8a67
    KeyThe key (to your service) to use when testing the Data Adapter. Note that you can only use the attribute selected from the Select the attribute to use as a key drop-down to test the Data Adapter. During testing, the attribute is replaced by the value entered in this field.
    Authentication methodChoose either HTTP Basic or OAUTH2. HTTP Basic requires a username and password. OAUTH2 requires a username, password, and security token.
    Security URLURL used to obtain an access token for OAUTH2.
    UsernameValid API username.
    PasswordValid API password.
    ScopeOptional. Text value representing the scope of the access request when using OAuth 2.0 authentication.
  7. Click Test configuration. This tests and verifies that the new configuration details are valid.

  8. Click Apply.

  9. Click Save.

Data Adapter Timeout Period

Requests from the JSON Pull Web Service Data Adapter are configured to timeout after one second. If MXO does not receive a response to a request within one second, the request will fail.

Common Error Messages

The following table outlines the most common errors reported in MXO.

ErrorReason
Could not connect to Target URL.Target URL is invalid. For example, it may contain a typo or incorrect hostname.
Customer response does not match defined schema.The schema is valid, but the data returned using the sample key does not match the schema. Any errors found by the parser are listed.
Customer response is empty.No data was available in the external system when MXO sent a request.
Schema is incorrectly formatted.Schema syntax is incorrect.
Response is incorrectly formatted.The schema is valid, but the data returned using the sample key does not match the schema. No information is available from the parser.
Security information is incorrect.Username and/or password is incorrect or there may have been an issue retrieving the OAuth 2 token.
Security Token URL is required for OAuth2.OAuth 2 was selected as the authentication type, but no security token URL was provided.
Invalid user credentials.Username or password is incorrect.
Invalid configuration.JSON Pull Web Service Data Adapter has not been configured correctly.
Only HTTPS connections are supported.Target URL and Security Token URL must be HTTPS URLs.

Points to Consider

This section is intended to provide you with a set of points to consider when establishing the Web Service and harnessing back-end data as part of your omni-channel, personalized customer experience. Not all topics will be applicable to all clients.

What steps must I take to open an external firewall port?

Remember that MXO is a SaaS solution and that connecting your back-end data source with MXO will require communicating outside of your firewall.

What volume of requests driving what volume of calls to the back-end data system will be applicable?

Consider the volume of customer interactions your organization supports today across all channels. It is these interactions that will drive calls to the back-end data source. It is important to think about any performance implications related to the volume of those interactions.

Should I use my Production schema?

Consider, from an architectural point of view, whether it is better to support the data exchange through your production schema or through a separate schema created for this purpose. For example, if your production schema is updated in real-time would a separate schema also be updated in real-time?

What is an appropriate data caching approach?

It is unlikely that you will want MXO to call the back-end data system for every individual customer request within an interaction, such as every time a customer visits a new page within the same session. Think about, for example, how long you are comfortable with MXO holding data in memory without refreshing. 5 minutes? 10 minutes? 20 minutes? Note that the key dependency will be the refresh frequency of the back-end data source itself.

What data attributes should I make available to MXO?

The data will be used by your business users to drive appropriate, personalized conversations for customers across all channels. You do not need to provide the business with every available piece of data you hold for a customer. Think about the key attributes you want to provide, based on agreed use cases. This data is likely to include both segmentation and some demographic data but what else would be of value to the business?

Do I need to create a mapping service?

Aligned with the content on linking customers to their customer data is the question about the composition and implementation of an abstract customer key service. MXO does not store Personally Identifiable Information (PII) and you will need to create a mapping service that translates a customer name into an abstract customer key.

The key is stored by MXO and used to communicate to the back-end data source. For example, Mary Stevens = ABC12345.

How quickly should my web service respond to MXO?

The web service should respond to MXO in as short a time as possible to ensure the data is available for use in decisioning.

What do I need to consider when scaling?

MXO will only make a request to the web service to retrieve customer data if that data is not already cached. You configure the caching period when setting up the data adapter.

If the web service is unavailable, MXO continues to work, but does not have access to the relevant customer data required for decisioning.