V‑Spark Version 3.5.0 provides improvements in a number of areas.

1. V‑Spark now supports distributed processing across multiple networked "nodes". This increases the availability, performance, and maintainability of your V‑Spark installation while providing a single integrated interface.

   • Running multiple components on multiple hosts increases availability by duplicating vital functions. Components can be started and stopped individually, enabling you to update single components without disabling the entire system.

   • Nodes share a modular storage architecture and distributed filesystem. This enables the system to handle uploads and downloads from more clients without storage bottlenecks.

   • V‑Spark now uses an improved queue-based processing flow. This reduces resource contention, increases the system's ability to utilize multiple resources, and improves your ability to monitor the progress of your data.

   If your V‑Spark processing needs exceed the computing capacity of one host, contact support@vocitec.com for more information about multi-node processing.

2. There have been multiple changes and enhancements made to the V‑Spark REST API. If your workflow depends on the API, you will want to review this section carefully.

   1. Log API no longer supported - The functionality of the /log API has been incorporated into the /request API, and the /log API is now deprecated and its jobmanager "processed.log" files are no longer supported. Any external tools that depend on this API call will need to be re-written.

   2. Status API modified - The JSON schema of the data returned by the /status API has been changed. The organization and content of the output has been significantly modified, and the value of the queued count now reports the number of queued requests and not the number of queued files. Any external tools that depend on this API call may need to be modified.

   3. Partial /config calls may now be posted for companies - When using the V‑Spark API to create a new company, it is no longer necessary to POST a complete JSON configuration. You may now POST a partial config that only includes mandatory values for the new company. When using the V‑Spark API to modify a company's configuration, it is no longer necessary to POST a complete config that includes even unchanged values. You may now POST a partial config that only includes any changes to existing configuration values.

      This change only applies to company configurations. Posting of partial configuration updates for organizations, folders, applications, or users using the REST API are not supported.

   4. New /config/perms API - The REST API now has a /config/perms call that returns information about which users have which permissions to which parts of the system.

   5. Error message for invalid JSON - any V‑Spark API that accepts JSON content does a check to determine the validity of any JSON content that is POSTed, and returns an Invalid JSON error if the JSON is not valid.

   6. Requests to the /transcribe API that include the s3key option to submit a file stored in AWS S3 must specify the Amazon S3 region of the S3 bucket using the region option or the request fails silently.

   7. Failed File Details in /request - The V‑Spark /request endpoint displays files that were unable to be transcribed. Details explaining why the files were not transcribed are included; for example, the file type was not supported or the file contained bad metadata.

3. Data loading directly from file system no longer supported - V‑Spark no longer supports direct ingestion of data from the local filesystem. Always upload audio via the V‑Spark UI or API so that all inputs are properly queued and logged. If your workflow requires file system audio upload, contact support@vocitec.com to discuss solutions.

4. Improvements to Folder processing status dialog - The ASR processing status section of this dialog has been improved to better show the progress of requests and files through V‑Spark's processing queues. The Queued column now shows the number of processing requests waiting to be processed, and not the number of files in the queue.

5. The following improvements have been made to V‑Spark Application building and processing.

   1. Reprocess Applications by date range - When reprocessing Applications, you may now choose a date range instead of simply a start date for reprocessing. Only files within the selected date range will be reprocessed. You may still choose to have all files reprocessed, regardless of date.

   2. Category names can now include Unicode characters - The names of Application Categories may now include any character in the UTF-8 Unicode Standard. This includes accented characters and non-English characters of many types. The only restricted character is "." (Unicode 002E) which is used in API searches. New categories can be created with these characters in their names, and existing categories can be modified using the Application Editor so that their names better match the style of their source language.

   3. Metadata filter import and export - Exporting a single category now includes metadata filters. This makes it easier to build applications and categories, as you can now export and import metadata filters, including both built-in and custom metadata filters.

6. V‑Cloud tokens are now verified when entered - When you create a new Company that has a V‑Cloud authorization token, add a V‑Cloud authorization token to an existing Company, or update a Company's V‑Cloud authorization token, the system now verifies connectivity to V‑Cloud and the validity of the token before enabling V‑Cloud for the Company. This check occurs whether you are working through the V‑Spark user interface or the API.

7. Callback Improvements - Callback status messages were improved to include more specific information about status codes and whether the callback failed or succeeded. The improved callback messages will also make it easier to identify company, organization, and folder names in log entries.

8. Delete Files from V‑Spark - Transcript files can be deleted using the V‑Spark API or user interface. Deleted files will no longer be available in the file list /transcript view. Note that the summary data, stats, and dashboard views will not be updated to reflect deletion.

9. For greater system security, systems administrators can now configure a set of password policies.

   1. Minimum password length - Administrators can now configure a minimum password length. Account passwords cannot be set or changed to passwords that do not have at least the required number of characters. The minimum password length is configurable, and defaults to 7 characters.

   2. Password aging - Passwords can now expire after a configurable number of days. The default is for passwords to expire after 90 days. When an account password expires, the next time the user attempts to log in they will be redirected to a page where they will update their account password. This policy can also be turned off so that passwords never expire.

This section identifies specific issues that have been resolved in the 3.5.0 release of V‑Spark:

1. Companies and Orgs being deleted are restricted - Both the V‑Spark UI and API prevent you from creating new organizations, folders, or users if the company or organization that would own those items is in the process of being deleted.

2. No usernames that are reserved words - Accounts cannot be created with usernames that are on the list of words that are reserved for use by V‑Spark internals. As of this release, the list of reserved words is: 'company', 'companies', 'org', 'orgs', 'app', 'apps', 'folder', 'folders', 'jobmgr', 'jobmgrs', 'user', 'users', 'all', 'system', and 'readonly'.

3. Improved clean up of temporary files - Temporary files that are created during searches and downloads were not being deleted promptly and could result in filesystem capacity problems. Temporary files are now deleted properly.

This section identifies fixes that were introduced in patch releases and hot fixes since the initial 3.5.0 release, and are therefore now included in the latest version of the 3.5.0 release. Maintenance versions and associated fixes are listed most recent to earliest:

1. Improved response handling for systems using V‑Cloud. Eliminated extraneous 503 errors generated when communicating with V‑Cloud to minimize unnecessary noise in Job Manager log files. (3.5.1-5)

2. Backend V‑Spark services now properly stop when Redis is down - Prior to this change, stopping the backend service while Redis was down could lead to errors that required killing backend processes manually in order to restart them. (3.5.1-4)

3. Update to support the latest version of the Datatool - if your V‑Spark installation has a copy of the Datatool provided by the Voci support team, this release will add value by providing some core changes that enable the Datatool to exercise its sampling feature with V‑Spark 3.5.x releases. (3.5.1-4)

4. Improved handling of hardware boot and reboot events - The V‑Spark initialization process was updated to permit V‑Spark services to start automatically during hardware boot and reboot. (3.5.1-3)

5. Dashboard performance optimization - Loading optimization updates were made to allow all folder-level dashboard widgets to load faster. (3.5.1-3)

6. Google Chrome 83 support update - Chrome 83 now blocks all downloads initiated from or instantiated in a sandboxed iframe by default. This update enables V‑Spark to override that default behavior and allow file downloads. (3.5.1-2)

7. Failed file details in /request - The V‑Spark /request endpoint displays files that were unable to be transcribed. Details explaining why the files were not transcribed are included; for example, the file type was not supported or the file contained bad metadata. (3.5.0-17)

8. Callback improvements- Callback status messages were improved to include more specific information about status codes and whether the callback failed or succeeded. (3.5.0-11)

9. Delete files from V‑Spark- Transcript files can be deleted using the V‑Spark API or user interface. Deleted files will no longer be available in the file list /transcript view. However, summary data will not be updated to reflect deletion at this time. (3.5.0-10)

The V‑Spark 3.5.0 Application Development Guide has been significantly updated, reorganized, and enhanced for this release. Improvements include:

1. The "Application Scoring" chapter now explains the different types of scoring in more detail.

2. The "Viewing Application Scores" chapter has been reorganized and now contains significantly more detail about V‑Spark features that enable you to view and interpret the analysis results produced by your applications.

3. Information on reprocessing audio and searching transcripts has been expanded with more detail.

4. Most figures have been updated and their callouts clarified.

5. Application Leaf-level Scores - Leaf-level application scores have been renamed from "count scores" to "leaf scores" throughout documentation.

1. Partial configuration updates for a

   • organization
   • folder
   • application
   • or user

   using the REST API are not currently supported. Attempting to update a subset of one of these types using the REST API currently returns HTTP error code 400. You must supply all fields for the configuration to update any field(s) using the REST API.

2. Application re-processing of more than 100,000 files in a single operation may time out.

3. Issues with Dashboard Displays

   • Files uploaded to a folder that is linked to a disabled application or that have not been re-processed will still appear in the Files View on the application dashboard

4. Issues with Search

   • Time-based search

     • Searches are turn-based and may return results for phrases that are outside of the requested time frame if those phrases are within the same speaker turn

     • Ad-hoc Search snippets in the Files View may display results outside the specified time interval

   • Exceptions to standard File Details highlighting:

     • Searches using (~n) may highlight phrases with the specified number of words in between each of the search terms, rather than between all of the search terms. For example, "get help phone ~4" may highlight phrases with 4 words between "get" and "help" and another 4 words between "help" and "phone". Searches using (~t) and (~n) will only be highlighted if the words matched appear in the order specified in the search tag.

     • Highlighting may be incomplete for matches marked with emotion

     • Highlighting may be incomplete for matches with intervening silence tags greater than 2 seconds

     • Matches for "@*" (such as when searching for email addresses) will only be highlighted until the first "." in the email address. Regular expression searches may be used to circumvent this issue.

     • Excluded application search phrases may be highlighted on the File Details page. Note that these exceptions only affect highlighting, and that all search results using the above characters will still be accurate.

5. Other

   • Users playing longer audio files via Firefox may encounter playback/syncing issues when jumping to different sections of the transcript. This is an external issue caused by the Firefox browser. Users who encounter this issue can simply wait for the browser to finish buffering the file before resuming their search.

   • App Reprocessing will encounter an error if the reprocessing occurs while an App or Category is being deleted. This error is recorded in the server logs.

   • Searches that contain both wildcards and spaces do not return results.

   • Most punctuation characters are not supported in search strings.

   • The /search API allows searches for application categories that have hyphens and other restricted characters in their names, however these searches fail. Voci application templates that had hyphens and other restricted characters in their names have been modified to remove the restricted characters. All existing application category names should be modified to remove restricted characters.