Whisper very slow

Hello,

this might be a stupid question. I need to use very long transcripts as my input (15-30 minutes), I tried using Whisper locally for this as my input in OpenWeb UI, it takes roughly half the time of the conversation to transcribe once I hit send. If I use the same Whisper model or even a bigger one on Whisper-WebUI for example it is much, much faster. Is this because I don't have to have a LLM loaded as well or is there another reason? I use a 3070 for now, if that makes any difference.

Posted Nov 12, 2024, 8:08 PM UTC

23
Loading comments…