In recent years, there has been a lot of research in the area of sequence to sequence learning with neural network models. These models are widely used for applications such as language modeling, translation, part of speech tagging, and automatic speech recognition. In this talk, we will give an overview of sequence to sequence learning, starting with a description of recurrent neural networks (RNNs) for language modeling. We will then explain some of the drawbacks of RNNs, such as their inability to handle input and output sequences of different lengths, and describe how encoder-decoder networks, and attention mechanisms solve these problems. We will close with some real-world examples, including how encoder-decoder networks are used at LinkedIn.

In the last decade, deep neural networks have created a major paradigm shift in speech recognition. This has resulted in dramatic and previously unseen reductions in word error rate across a range of tasks. These improvements have fueled products such as voice search and voice assistants like Amazon Alexa and Google Home.

The main components of a speech recognition system are the acoustic model, lexicon, and language model. In recent years, the acoustic model has evolved from using Gaussian mixture models to deep neural networks, resulting in significant reductions in word error rate. Recurrent neural network language models have also given improvements over the traditional statistical n-gram language models. More recently sequence to sequence recurrent neural network models have subsumed the acoustic model, lexicon, and language model into one system, resulting in a far simpler model that gives comparable accuracy to the traditional systems. This talk will outline this evolution of speech recognition technology, and close with some key challenges and interesting new areas to apply this technology.