Basic Real Time Streaming Capability

[kanjieater]

I'm wondering if there's any functionality that stable-ts could support in terms of streaming audio and showing it in real time. There's a demo here of something to this effect:

https://github.com/davabase/whisper_real_time/blob/master/transcribe_demo.py

[jianfch]

Stable-ts does not have any "real time" functionality but can be used with methods similar to the one you linked.

The method transcribes the audio in few second chunks and skips the chunks below a threshold. An obvious limitation is the high latency. The transcribe function treats the audio as complete, so there is preprocessing/postprocessing overhead that can be avoided if the function was written to treat audio as a continuous stream.
Another limitation is that chunks can split in the middle of a word which can cause the model to hallucinate or skip words. Determining the boundary of a word is less straightforward when the audio has other sounds that are not speech.
Since one of the focuses this repo is make Whisper reliable, it will have to use another method that does not suffer from the aforementioned limitations when implementing a real time function.