What is "demucs" exactly?

[quopquai]

Can someone please explain what "demucs" is, and how it might improve transcriptions with stable-ts? Thanks!

[jianfch]

Demucs is a music source separation model. When demucs=True, stable-ts runs the audio through Demucs to isolate the vocals. So we end up with audio that is just vocals/speech (or most of the other sounds removed/reduced). This can also work with audio that isn't music. Essentially, using it as a noise remover. When there is only speech and the rest is silent, Whisper hallucinates less and timestamp correction becomes more effective because it is easier to tell there is no speech in a section when it is silent instead of being filled of noise.

[orionflame]

But this demucs still keeps non speech parts like throat clearing right? In my tests it kept them in vocals so that was surprising. I am looking for a way to silence non speech parts.