Receiving emotional intelligence data

The methods described in Requesting emotional intelligence data will include emotional intelligence data in the JSON output file that is returned from the transcription server. Each transcribed utterance will have its own emotion value. Emotion values are one of the following:

  • Positive: Only positive emotions or sentiments were detected.

  • Mostly Positive: Most emotion and sentiment values were positive but some negative values were also present.

  • Neutral: No emotions or sentiments were detected, or the numbers of positive and negative sentiment-bearing phrases were equal.

  • Mostly Negative: Most emotion and sentiment values were negative but some positive values were also present.

  • Negative: Only negative emotions or sentiments were detected.

Consider an example consisting of the following statements, “The matter was resolved in a very professional manner. Your employees are very good.” The JSON data returned by the transcription server begins like this:

{
    "emotion": "Positive",
    "source": "sample1.wav",
    "confidence": 0.8, 5 "utterances": [
        {
        "emotion": "Positive",
        "confidence": 0.8,
        "end": 6.48,
        "sentiment": "Positive",
        "sentimentex": [ 
            [ 3, 0 ], 
            [ [ "+", 1, [ 1, 4 ] ],
            [ "+", 1, [ 6, 9 ] ], 
            [ "+", 1, [ 10, 14 ] ] ]
        ],
        "start": 0,
        "recvdate": "2015-05-20 11:35:46.907486",
        "events": [
            {
            "confidence": 0.63,
            "end": 0.66,
            "start": 0.55,
            "word": "The"
            },
            {
            "confidence": 0.61,
            "end": 1.02,
            "start": 0.66,
            "word": "matter"
            },
  1. The first occurrence of emotion is the overall emotion for the file, "Positive" in this case. This overall rating will be included only if all utterances in the file have the same value. For this simple example, there was only one utterance.

  2. The sentimentex (sentiment extension) entry indicates the sentiment value of phrases in the transcribed text. The section is a series of lists.

  3. Note the first in a series of lists in the sentimentex entry; this first list contains the number of positive phrases (3) followed by the number of negative phrases (0)

  4. The other lists in the remainder of the sentimentex entry indicate the location of sentiment-bearing phrases in the text. For example, this line ( [ "+", 1, [ 6, 9 ] ], )indicates the following:

    • The “+” indicates that this is a positive phrase. If it had been negative, “-” would appear instead.

    • The 1 indicates the weight of the phrase. This value is not currently used and will always be 1.

    • The 6 and 9 indicate the index of the first word in the phrase and the index of the first word after the phrase. (Note that the first word is counted as index 0.) This means that the phrase will consist of 3 = 9 – 6 words.

Emotional intelligence data in V‑Spark

V‑Spark performs additional processing of the JSON data computed by the transcription servers. When JSON data is downloaded from V‑Spark, selected values are collected into a single section entitled “app_data” as shown below:

"app_data": {
    "agent_clarity": "0.708",
    "agent_emotion": "Positive",
    "client_emotion": "Improving",
    "overall_emotion": "Positive",
    "client_gender": "male",
    "client_clarity": "0.689",
    "duration": "0:29:49",
    "diarization": 2,
    "agent_channel": 0,
    "url":"http://server:3000/fileDetails/org/folder/date/123456.wav",
    "overtalk": "0.359",
    "agent_gender": "female", 
    "silence": "0.831"
},

One of the significant computations that V‑Spark performs is determining which channel of the audio contains the agent and which contains the client. Once this information is available, the emotion for individual speakers can be indicated.

In addition to providing utterance-level Emotional Intelligence values like V‑Blaze, V‑Spark also provides overall Emotional Intelligence values for the whole call. The Emotional Intelligence values computed by V‑Spark are shown in the agent_emotion , client_emotion , and overall_emotion fields in the previous JSON example. They are indicative of the change in emotional states of the speakers as the conversation progresses. Possible values are:

  • Positive The conversation began with positive or neutral emotions and remained positive or neutral.

  • Improving The conversation began with negative emotions and changed to positive or neutral emotions.

  • Negative The conversation began with negative emotions and remained negative.

  • Worsening The conversation began with positive or neutral emotions and changed to negative emotions.