Highlights of the new engine

Here are the highlights of the features of the new LumenVox ASR engine. Subsequent sections drill down into these features, giving more details:

• End-to-end acoustical modeling with Convolutional Neural Networks (CNN). This extends the benefits of neural networks from the sound-recognition stage all the way to the production of text, removing the need for separate speech engines to accommodate dialects and accents.

• Transfer-learning techniques to improve learning efficiency.

• A streaming model that operates in an online matter, thereby boosting responsiveness and performances.

• Performance and accuracy aided by statistical n-gram models.

• Performance and accuracy aided by traditional SRGS grammars.

• Use of quantized sort-tree n-gram Statistical Language Models (SLM), efficiently compressed in storage, enabling higher performance in less memory.

End-to-End Acoustical Modeling with Convolutional Neural Networks (CNN)

Convolutional Neural Networks (CNN) are a more advanced version of the neural-net and DNN technologies. A CNN approach develops patterns in successive layers, each building upon the layer before it.

In the visual-pattern recognition world, CNN starts by recognizing dots and small areas on the screen. Then it builds on these items to “see” lines and curves. Finally, it builds on lines and curves to see shapes.

In the speech recognition world, CNN does something similar: it hears and recognizes small bits of sound; then it builds on sounds to recognize phonemes. Finally, it builds on phonemes to recognize words and phrases.

In contrast, the legacy model does the following:

1. The ASR produces a series of phonemes (specific pronunciations).

2. A language model is then used to make sense of these phonemes and build them into words.

The legacy approach may work until the engine encounters a new dialect or a new accent… or sometimes just idiosyncrasies in the way a person talks. At that point, it fails. Under the old model, every time the dialect or the accent changes, a new speech model is required. That can stop customer service in its tracks.

With the new speech engine, the neural-net technology is end-to-end because it uses the neural net to fold in the problem of dealing with different pronunciations. In other words, we let the most expert system—the neural network itself—handle the problem. In doing so, the engine takes advantage of machine-learning technology to solve the problem of variations in speech.

Transfer Learning Techniques

Transfer learning is a machine learning technique where once a model has been trained for a specific task, it is then reused to initialize a model for a different task.  This is a very common approach for deep learning where pre-trained models are being used as the starting point on many fields.

In our case this means that we can reuse models trained on thousands of hours of data, an amount only available for a small fraction of languages (e.g., English), to initialize the training of models for languages where only a reduced amount of data is available (e.g., Italian).

This results in much better accuracy and performance than when training such a model from scratch using only target language data. Furthermore, transfer learning speeds up the training process considerably (days versus months). This lowers the cost of training models for new languages.

Statistical n-gram Language Models

An “n-gram” is a sequence of n words that is statistically likely. For example, “to beak” is an unlikely bi-gram (group of 2) but “to be” is likely a bi-gram; we should expect the words “to be” to appear in that order but not “to beak.”

The advantage of this technology is that it aids in determining what words were probably said, thereby making the engine less likely to make a mistake. For example, let’s say the speaker said something like this:

I’d like to book a fly please.

Using n-gram analysis, the software can determine that it’s much more likely that the speaker meant the following sentence, but the words were not heard perfectly. “Book a flight” is a common tri-gram.

I’d like to book a flight please.

By understanding what combinations of words are more likely to appear together, the speech engine can more accurately determine what the speaker meant to say.

Quantization of Sorted-Tree n-gram Speech Language Models (SLM)

With the technology used by the new engine, language models n-grams (common sequences of words) are stored using sorted trees—a data structure that can be searched with great efficiency. The new engine also uses a compression scheme that allows larger and deeper language models more of these n-grams to be stored in a smaller space. More detailed language models are therefore represented in a smaller space and are more efficient, doing more with less memory.

Use of Traditional SRGS Grammars

SRGS grammars use BNF or XML style to model patterns of words that are expected to appear together[SH3]  and in what sequence. The new engine uses these grammars as another way to predict what patterns of words are more likely. This enables more accurate processing of sounds into text.

Streaming Models

The new engine makes direct use of speech input in real-time[SH4] . Incoming sounds are processed as received, and knowledge picked up during one pass is utilized right away. As a result, there are no multiple passes or extra-cycles of processing. Consequently, there is no latency or perceptible delay between the caller speaking and the system responding. The result is a system that is faster and more responsive and makes for happier callers.