Emotion-controllable Speech Synthesis using Emotion Soft Label, Utterance-level Prosodic Factors, and Word-level Prominence: sample audio

Mar 14, 2023

First-stage controlling: conditioning on emotion soft labels

Angry = 1.0
Neutral = 1.0
Sad = 1.0

Second-stage controlling: Fine-conditioning on prosodic factors (Angry = 1.0)

Energy mean(+0.3)
Energy mean(-0.3)
Energy SD(+0.3)
Energy SD(-0.3)
Energy range(+0.3)
Energy range(-0.3)
Pitch mean(+0.3)
Pitch mean(-0.3)
Pitch SD(+0.3)
Pitch SD(-0.3)
Pitch range(+0.3)
Pitch range(-0.3)

Second-stage controlling: Fine-conditioning on prosodic factors (Neutral = 1.0)

Energy mean(+0.3)
Energy mean(-0.3)
Energy SD(+0.3)
Energy SD(-0.3)
Energy range(+0.3)
Energy range(-0.3)
Pitch mean(+0.3)
Pitch mean(-0.3)
Pitch SD(+0.3)
Pitch SD(-0.3)
Pitch range(+0.3)
Pitch range(-0.3)

Second-stage controlling: Fine-conditioning on prosodic factors (Sad = 1.0)

Energy mean(+0.3)
Energy mean(-0.3)
Energy range(+0.3)
Energy range(-0.3)
Pitch mean(+0.3)
Pitch mean(-0.3)
Pitch SD(+0.3)
Pitch SD(-0.3)
Pitch range(+0.3)
Pitch range(-0.3)

Second-stage controlling: Fine-condtioning on prominence (Angry)

NOUN (+0.3)
They forcefully keep them at a black hotel.
NOUN (-0.3)
They forcefully keep them at a black hotel.
VERB (+0.3)
They forcefully keep them at a black hotel.
VERB (-0.3)
They forcefully keep them at a black hotel.
ADJ (+0.3)
They forcefully keep them at a black hotel.
ADJ (-0.3)
They forcefully keep them at a black hotel.
ADV (+0.3)
They forcefully keep them at a black hotel.
ADV (-0.3)
They forcefully keep them at a black hotel.

Second-stage controlling: Fine-conditioning on prominence (Neutral)

NOUN (+0.3)
They forcefully keep them at a black hotel.
NOUN (-0.3)
They forcefully keep them at a black hotel.
VERB (+0.3)
They forcefully keep them at a black hotel.
VERB (-0.3)
They forcefully keep them at a black hotel.
ADJ (+0.3)
They forcefully keep them at a black hotel.
ADJ (-0.3)
They forcefully keep them at a black hotel.
ADV (+0.3)
They forcefully keep them at a black hotel.
ADV (-0.3)
They forcefully keep them at a black hotel.

Second-stage controlling: Fine-conditioning on prominence (Sad)

NOUN (+0.3)
They forcefully keep them at a black hotel.
NOUN (-0.3)
They forcefully keep them at a black hotel.
VERB (+0.3)
They forcefully keep them at a black hotel.
VERB (-0.3)
They forcefully keep them at a black hotel.
ADJ (+0.3)
They forcefully keep them at a black hotel.
ADJ (-0.3)
They forcefully keep them at a black hotel.
ADV (+0.3)
They forcefully keep them at a black hotel.
ADV (-0.3)
They forcefully keep them at a black hotel.