🤠 Add Hifigan vocoder

[dathudeptrai]

This PR is an implementation of the HiFi-GAN vocoder (https://arxiv.org/abs/2010.05646). The training process follows melgan_stft. The model logic follows by original HiFi-GAN PyTorch code (https://github.com/jik876/hifi-gan).

[machineko]

LGFM

[ZDisket]

looks good

[dathudeptrai]

@machineko @ZDisket can you guys try to training around 2k steps to verify if it works :))) I do not have GPU right now to test :))). There are so many differences between a private library and this opensource:v.

[machineko]

@machineko @ZDisket can you guys try to training around 2k steps to verify if it works :))) I do not have GPU right now to test :))). There are so many differences between a private library and this opensource:v.

Same not available GPU at the moment but will test in 2/3 days 📦

[machineko]

@dathudeptrai Did u wanna to use config v2 or v1 for it?

[ZDisket]

@dathudeptrai

can you guys try to training around 2k steps to verify if it works :))) I do not have GPU right now to test :))). There are so many differences between a private library and this opensource:v.

I could train 4k steps and counting with v1 config and mixed precision without problems. I even got eval samples at 5k.

[dathudeptrai]

@dathudeptrai Did u wanna to use config v2 or v1 for it?

v2 for faster :D

[dathudeptrai]

@dathudeptrai

can you guys try to training around 2k steps to verify if it works :))) I do not have GPU right now to test :))). There are so many differences between a private library and this opensource:v.

I could train 4k steps and counting with v1 config and mixed precision without problems. I even got eval samples at 5k.

is the loss ok ? , can you try to continue training both G and D around 1k steps :D

[ZDisket]

@dathudeptrai For some reason the loss exploded after 10k and the eval samples are either noise or silence, although I think it's just because of the small dataset. Going to restart training and train discriminator from 0 steps

[ZDisket]

can you try to continue training both G and D around 1k steps :D

Completed 2k steps of G+D starting from 0 steps. No problems.

[lesswrongzh]

@ZDisket could you share your hifigan tensorboard like this?

[EmreOzkose]

I also have this problem. My tensorboard:

and predictions are all same noisy sound. For example:

What could be the problem? I first trained generator and after resume.

[ZDisket]

@EmreOzkose The TensorflowTTS implementation is not faithful to the original when it comes to the optimizer. The official implementation uses AdamW optimizer with ExponentialLR, while the one in this repo uses Adam with PiecewiseConstantDecay. Plus there is no generator pretraining in the original.
https://github.com/jik876/hifi-gan/blob/4769534d45265d52a904b850da5a622601885777/train.py#L63-L72
A while back I implemented some changes to make it the same and it didn't die during training, but I didn't train it to completion so I haven't evaluated it enough to warrant a PR. Still, the .zip I'm attaching has the training script and a v2-based 44.1KHz config with all the changes. See if it helps.
hf.zip

[EmreOzkose]

I am checking out, thank you @ZDisket.

[EmreOzkose]

@EmreOzkose The TensorflowTTS implementation is not faithful to the original when it comes to the optimizer. The official implementation uses AdamW optimizer with ExponentialLR, while the one in this repo uses Adam with PiecewiseConstantDecay. Plus there is no generator pretraining in the original.
https://github.com/jik876/hifi-gan/blob/4769534d45265d52a904b850da5a622601885777/train.py#L63-L72
A while back I implemented some changes to make it the same and it didn't die during training, but I didn't train it to completion so I haven't evaluated it enough to warrant a PR. Still, the .zip I'm attaching has the training script and a v2-based 44.1KHz config with all the changes. See if it helps.
hf.zip

I tried a training with the same setup and got different signals from a noise. Thank you 😃

[ZDisket]

@EmreOzkose Any updates? Did it do well?