How does this work exactly?

Language models version 7.5 and greater have a space in their internal data structures for a "keyhole model" that is generated when the ASR engine processes a transcription request with OOV enabled. This keyhole model is generated once for each unique OOV dictionary received; subsequent OOV requests that specify the same dictionary use the cached model to save on load time. The OOV transcription request is processed using the language model augmented with the OOV keyhole model in place of the base language model during the core decoding process.

In alpha OOV development, it was determined that collateral damage to the transcript on words immediately surrounding OOV terms was significant enough to correct by running the same transcription request twice and splicing the results together to produce a more accurate result. This splicing is controlled with the effort option.

JSON output fields

The field oovkey in JSON transcript output indicates that OOV was used. The value of oovkey indicates the dictionary used. There is no count or flag in JSON transcript output reflecting whether OOV terms were found during decoding. OOV uses substitution when preserving case, so nsubs could go up, but isn't really measuring OOV.

What are OOV's limitations?

When a transcription request includes OOV, the ASR engine assumes that it is much more likely to detect OOV words in processed audio. In other words, when transcribing speech, words decoded with the OOV dictionary are strongly weighted over non-OOV matches in the base model. As a result, poorly matched OOV sets may produce more false positives.

Generally, OOV performs better when looking for small phrases rather than single words. So context (that is, using a phrase instead of a single word) becomes more important as OOV words sound more like words in the model's standard vocabulary. But ultimately, the most critical element is how the word itself sounds. The more unique the word sounds, the less context OOV needs to transcribe the word accurately.

How does OOV impact throughput and latency?

Transcription requests with OOV enabled incur additional processing in two ways:

1. Each transcription request with OOV includes a dictionary of terms. The first request that includes a unique OOV dictionary incurs a one-time, multi-second delay as a new model is generated for those terms.

   The new models generated are cached for future use; if the same dictionary is submitted again in a separate request, the ASR engine recognizes the dictionary and reuses the previously generated OOV model.

2. Cached OOV models are used as needed to decode utterances, resulting in high parallelization and enabling the use of a wide variety of OOV configurations. This process has a negligible impact on utterance processing latency, typically limited to milliseconds of additional decode time.

For self-hosted V-Blaze, cached OOV models are stored in /opt/voci/ramfs/oov. Caches are retained across service restarts, but not across system reboots.

[EXTRA INTERNAL ONLY] All the decoders run as they always did, except if they get an utterance that specifies OOV, they insert the cached keyhole model. This process is quick, and once it's done, it pulls the keyhole model out and keeps going. So a decoder can handle utterances from extra streams with no added penalty, because it's exactly the same no matter what your OOV configuration is.

How do I measure OOV's effectiveness?

By measuring accuracy or WER across a test transcript set. There is no count of OOV matches in transcription output JSON like there is with substitutions because OOV is included in the core decoding process.

[INTERNAL ONLY] Support and usage notes

[Special usage note] Ran some quick experiments with giving digits to OOV and turns out eng-us LTS will spell out a digit string of up to 4 digits. Over that it will throw an error. 

[Special usage note] If you mix letters and digits then it will not spell out digits (they will be silently dropped). 

[Special usage note] Just tried digits with spa and por OOV and they both fail straight away so seems just an eng or eng-us special on the 4 digits or less strings.


[Limitation] However, I also found I could not get OOV to sub a number sequence at all (soundouts or not). Maybe it's just the examples I tried but seems the LM locks in to number decoding and will not divert into OOV. 

* OOV files with unicode characters are ignored.

* The scoring (acoustic) model has nothing to do with OOV, it's purely a searching (little language) model extension

* IF there is capitalization in OOV, it will be preserved

* Can't force lowercase without substitutions

* Can force partial capitalization

Q: theoretical or practical limit to OOV rules

A: it's a lot. many thousands at least - originally 1000, then condensed it? can it do 5000 (customer ask)?
accuracy will get weirder as complexity increases, but from a latency perspective, it doesn't care once it's built the sub-version of the model by combining the model with OOV stuff

22500 is the upper-limit for one word per OOV line (phrase)

So 5000 should be fine