Personalizing Natural Language Understanding using Multi-armed Bandits and Implicit Feedback
Fabian Moerchen, Patrick J. Ernst, Giovanni Zappella · Proceedings of the ... ACM International Conference on Information & Knowledge Management · 2020
Natural Language Understanding (NLU) models on voice-controlled speakers face several challenges. In particular, music streaming services have large catalogs, often containing millions of songs, artists, and albums and several thousands of custom playlists and stations. In many cases there is ambiguity and little structural difference between carrier phrases and entity names. In this work, we describe how we leveraged multi-armed bandits in combination with implicit customer feedback to improve accuracy and personalization of responses to voice request in the music domain. Our models are tested in a large-scale industrial system containing several other components. In particular, we focused on using this technology to correct errors made by upstream NLU models and personalize responses based on customer preferences and music provider functionality. The models resulted in significant improvement of playback rate for Amazon Music and are deployed in systems serving several countries and languages. We further used the implicit feedback of the customers to generate weakly labeled training data for the NLU models. This improved the experience for customers using other music providers on all Alexa devices.
Cited by 9 works. doi:10.1145/3340531.3412736 Open-access PDF
Authors
References
- Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
- Neural Architectures for Named Entity Recognition
- Learning to rank using gradient descent
- Advances in Neural Information Processing Systems 27 (NIPS 2014)
- Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
- Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
- Analysis of Thompson Sampling for the multi-armed bandit problem
- Thompson Sampling for Contextual Bandits with Linear Payoffs
- A Survey on Recent Advances in Named Entity Recognition from Deep Learning models
- Learning joint query interpretation and response ranking