Few tips to train your vocal model using so-vits-svc

I’ve been playing sovits recently and tried to train my own timbre model (or vocal model). I have found out a little bit of experience. I think it’s nice to write it down here to share with you guys.

Let’s show the results first. Check out the vedio below:

https://www.bilibili.com/video/BV1Hs4y1M7na/

Tip1: When recording, the maximum vocal level should not exceed -6db.

On the right side of the Audition audio graph, you can see the level range of the current recording.

During recording and playback, the level at that moment will also be displayed below.

When I first prepared the data set, I recorded it with the upper limit of 0db. I wanted to learn from the right exposure in the field of photography. I thought the quality of this kind of recording should be excellent. As a result, I get two models with 10,000 steps and 20,000 steps, and they all sounds like a robot. Later, by controlling the sound below -6db, I get a far more better result.

Tip2: Record directly using 44100 sampling rate to save resample time

Audition Create a new audio file and select the sampling rate 44100

Tip3: Use Amplitude and Limit -> Dynamics to quickly remove noise between pieces of utterances

After selecting all audio, go to Effects -> Amplitude and Compression -> Dynamics.

You can refer to this setting to enable automatic doors.

You need to pay attention to the threshold. According to your own audio, look at the level on the right to find the smallest voice you speak. For example, mine is about -30db. The threshold can be set slightly smaller than this.

Tip4: If you are on a decent machine, be sure to turn on all_in_mem

Change all_in_mem to true in configs/config.json.

This parameter is turned off by default. Official document says only this should only be used on machines with poor IO performance or when the memory is much larger than the data set. My hard drive is a Ti7100 2TB PCIe Gen4 drive, so the IO is not too bad. But when this option is turned off, 4090 actually takes about 4 minutes foreach epoch (that is, every 200 steps) processing 600 pieces of data. And I can see the power consumption of my GPU is only 50W. It is obviously an IO bottleneck. Think about that, repeatedly performing 4K reading and writing of more than 600 small files from the hard disk will really make a definitely difference from reading them directly from the memory. After setting this to true, the training speed is increased by about ten times. In addition, the size of these 600 pieces of wav files is about 2GB. So if your free memory is greater than 5 times the wav training data, you can enable this setting.

Tip5: How many steps are suitable for training? My experience is that a model with around 9,000 to 11,000 steps is already nice to use. If it’s too low, it won’t sound like the training data; and if it’s too high, it’ll sound robotic. It is worth noting that it only takes about 18 minutes to train on 4090 with these steps. Many other bloggers have shared that it takes several hours, or even a whole day for them to try. Don’t be misled by these data. The training of sovits is relatively fast if you get a powerful GPU.

Tip6: Only high-quality training data can generate good-sounding songs. Because when I collected training data, I was in a non-private environment, so I was embarrassed to sing loudly. Therefore, the generated singing voice is only suitable for low-pitched singing. When it comes to high notes and chorus parts, the generated voice sounds bad. Therefore, if you want to train a model that sings well, you must be a good singer yourself.

The vocal of the original song

Singing vocals trained based on my model

You can see the vocal spectrum diagram of the original song, and the energy is evenly distributed in the high, middle and bass. The generated singing voice of mine only has energy in the bass part. Because the number of overtones in my training data itself is not enough. This will lead to the fact that when the two tracks are stacked together, even if my singing is set to a stronger level, the voice from the original singer sounds still louder. This problem is reflected in the result I showed above. You can hear it in the later half.