Sample Efficient Adaptive Text-to-Speech
Yutian Chen, Yannis Assael, Brendan Shillingford et al. · arXiv (Cornell University) · 2018
We present a meta-learning approach for adaptive text-to-speech (TTS) with few data. During training, we learn a multi-speaker model using a shared conditional WaveNet core and independent learned embeddings for each speaker. The aim of training is not to produce a neural network with fixed weights, which is then deployed as a TTS system. Instead, the aim is to produce a network that requires few data at deployment time to rapidly adapt to new speakers. We introduce and benchmark three strategies: (i) learning the speaker embedding while keeping the WaveNet core fixed, (ii) fine-tuning the entire architecture with stochastic gradient descent, and (iii) predicting the speaker embedding with a trained neural network encoder. The experiments show that these approaches are successful at adapting the multi-speaker neural network to new speakers, obtaining state-of-the-art results in both sample naturalness and voice similarity with merely a few minutes of audio data from new speakers.
Cited by 94 works. doi:10.48550/arxiv.1809.10460 Open-access PDF
Authors
- Yutian Chen
- Yannis Assael
- Brendan Shillingford
- David Budden
- Scott Reed
- Heiga Zen
- Quan Wang
- Luis C. Cobo
- Andrew Trask
- Ben Laurie
- Çağlar Gülçehre
- Aäron van den Oord
- Oriol Vinyals
- Nando de Freitas
References
- ImageNet classification with deep convolutional neural networks
- Going deeper with convolutions
- Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
- The formation of learning sets.
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- WaveNet: A Generative Model for Raw Audio
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Librispeech: An ASR corpus based on public domain audio books
- Pixel Recurrent Neural Networks
- Learning to Learn