Audio samples from "Towards Natural Cross-Lingual Voice Conversion Based on Neural TTS Models and Phonetic Posteriorgrams"

Authors: Shengkui Zhao, Hao Wang, Trung Hieu Nguyen, Bin Ma

Alibaba Group, MIT Singapore Lab



Note: In our VC experiments, we have used the open TTS corpus JSUT from R. Sonobe, S. Takamichi, and H. Saruwatari, “JSUT corpus: free large-scale japanese speech corpus for end-to-end speech synthesis,” arXiv:1711.00354 [cs.CL], 2017.



The English target speaker's voices from professional records:

Text: I think it could be something to do with the soil and the climate.

Text: We are very concerned that it will not happen and we will be engaged.

Text: They arrive at the door of the food bank from all over the city.


The Mandarin target speaker’s voices from professional records:

Text: 五千年传统文化信手拈来,被涂脂抹粉,戏谑调侃。

Text: 今天,和平广场里盛开着这样的黄玫瑰。

Text: 他说,他会将车收藏一辈子,以后再传给儿女们。


Cross-lingual voice conversion:

Source speaker: Mandarin female; Target speaker: English female


Text: 五千年传统文化信手拈来,被涂脂抹粉,戏谑调侃。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: 他们家靠这个药,世代漂洗为生,日子过得艰难。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: 众所周知,潍坊因风筝而在全国闻名遐迩。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC



Source speaker: Mandarin male; Target speaker: English female


Text: 一场关于食品安全的激烈讨论就此展开。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: 一年年的春节,回家的故事一串串。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: 一打听才知道,她今天是来领神秘大奖的。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC



Source speaker: Japanese female; Target speaker: English female


Text: ギレスピーは、マッギーを通じて、イネスと知り合った。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: 森永のおいしい牛乳は、濃い青色に、牛乳瓶をあしらったデザインの、パック牛乳である。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: 同日朝に、大阪難波で、出発セレモニーが開催され、営業運転に、充当された。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC



Source speaker: English male; Target speaker: English female


Text: I think it could be something to do with the soil and the climate.
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: But the jury failed to reach a verdict on the other two traders.
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: I was rushed to hospital a couple of weeks ago with my angina.
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC



Cross-lingual voice conversion:

Source speaker: English female; Target speaker: Mandarin female


Text: Hazel would like to sell the business.
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: Many people have lost their jobs altogether.
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: This is a serious accident, and we will do our utmost to identify the cause.
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC



Source speaker: English male; Target speaker: Mandarin female


Text: I think it could be something to do with the soil and the climate.
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: But the jury failed to reach a verdict on the other two traders.
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: I was rushed to hospital a couple of weeks ago with my angina.
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC



Source speaker: Japanese female; Target speaker: Mandarin female


Text: ギレスピーは、マッギーを通じて、イネスと知り合った。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: 森永のおいしい牛乳は、濃い青色に、牛乳瓶をあしらったデザインの、パック牛乳である。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: 同日朝に、大阪難波で、出発セレモニーが開催され、営業運転に、充当された。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC



Source speaker: Mandarin male; Target speaker: Mandarin female


Text: 一场关于食品安全的激烈讨论就此展开。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: 一年年的春节,回家的故事一串串。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC


Text: 一打听才知道,她今天是来领神秘大奖的。
Source Speaker

Target Speaker

FastSpeach-VC

Tacotron2-VC

Transformer-VC



Controllable voice speed:

Source speaker: English female; Target speaker: Mandarin female


Text: Hazel would like to sell the business.
Source Speaker

Target Speaker

FastSpeach-VC (normal)

FastSpeach-VC (slow)

FastSpeach-VC (fast)


Text: Many people have lost their jobs altogether.
Source Speaker

Target Speaker

FastSpeach-VC (normal)

FastSpeach-VC (slow)

FastSpeach-VC (fast)


Text: This is a serious accident, and we will do our utmost to identify the cause.
Source Speaker

Target Speaker

FastSpeach-VC (normal)

FastSpeach-VC (slow)

FastSpeach-VC (fast)



Adaptation to limited data:

Target speaker: Japanese female (500 utterances)


Text: 伽马射线暴的一种致命来源是超超新星。
Source Speaker

Target Speaker

FastSpeach-VC


Text: 一场关于食品安全的激烈讨论就此展开。
Source Speaker

Target Speaker

FastSpeach-VC


Text: At last you will be able to sit back, relax and enjoy the fruits of your labour.
Source Speaker

Target Speaker

FastSpeach-VC


Text: Life is hard enough for them and they need a bit of support.
Source Speaker

Target Speaker

FastSpeach-VC