Need to build a Fully Open Source Voice Calling Agent

I built one using 11 Labs. Now To switch to Open Source.

Orchestration: LiveKit
Telephony: twilio
VAD: Silero

Now Could you guys suggest me mainly on my TTS/STT LLM models.

STT mostly will go with Whisper only

but for TTS- I have Chatterbox, Kokaro in my mind.
But please note i need multi lingual support for it: atleast 7 Languages (Hindi, English, german , French, Spanish, Korean Chinese , Arabic should do ig)

and for LLM QWEN 3, 3.5, 4

Could you suggest me something better or anything that i could be choosing wrong??

My requirements:

  1. I would be calling people who could be UK not talking in english so i would like mid turn in language
  2. Would definately be extracting certain pieces of info from their answers

Posted Jul 29, 2026, 12:56 PM UTC

01
Loading comments…