Resources
If you are building it yourself.
Open-source projects, documentation and standards for each layer of the stack. Grouped by where they sit, so you can start at the layer you are stuck on.
Layer 01
Telephony
Gets the call in and out
- Asterisk ↗Open source
The long-standing open telephony engine. Still the default answer for bridging SIP into something you control.
- FreeSWITCH ↗Open source
Telephony platform used under the hood by a surprising share of the vendors in this directory.
- Twilio documentation ↗Vendor documentation
Media Streams is the usual reference implementation for getting live call audio into your own runtime.
- SIP (RFC 3261) ↗Standard or protocol
The session protocol underneath all of this. Worth an hour when call setup starts failing in ways no dashboard explains.
- WebRTC ↗Standard or protocol
How real-time audio reaches browsers and apps, and the transport most agent frameworks build on.
Layer 02
Speech to text (STT/ASR)
Turns caller audio into words
- Whisper ↗Open source
The reference open speech recognition model. Not streaming out of the box, which is the first thing you will discover when you try to use it on a live call.
- Silero VAD ↗Open source
Small, fast voice activity detection. Most self-built stacks end up using this or something very like it to decide when the caller has stopped speaking.
- NVIDIA NeMo ↗Open source
Toolkit for training and fine-tuning speech models, including multilingual and Indic work. The route if you need a model tuned to your own audio.
- Vosk ↗Open source
Offline recognition with small models across many languages. Useful when data cannot leave the building.
- Open ASR Leaderboard ↗Reference
Comparative word error rates across open models. Read it knowing the evaluation audio is nothing like a phone call.
- Deepgram documentation ↗Vendor documentation
Streaming recognition API with telephony-oriented models. Their docs on endpointing and interim results are worth reading whichever provider you pick.
- AssemblyAI documentation ↗Vendor documentation
Recognition API with real-time streaming and speaker labelling.
Layer 03
Language model
Decides what to say next
- Anthropic documentation ↗Vendor documentation
Streaming, tool use and prompt caching — the three things that decide whether a model is usable inside a voice latency budget.
- OpenAI platform documentation ↗Vendor documentation
Includes realtime and speech-to-speech APIs, which collapse several stack layers into one call and change the cost model entirely.
- vLLM ↗Open source
High-throughput serving for open models. The practical answer if you are self-hosting and care about time-to-first-token.
Layer 04
Text to speech (TTS)
Speaks the reply
- Piper ↗Open source
Fast local neural speech synthesis, including many languages. Runs on modest hardware, which matters if you are synthesising at volume.
- Coqui TTS ↗Open source
Training and inference toolkit for speech synthesis. Archived upstream but still the most complete open reference for how these systems fit together.
- ElevenLabs documentation ↗Vendor documentation
Streaming synthesis and voice cloning. Test any voice you like over an actual phone line before committing to it.
- Cartesia ↗Vendor documentation
Low-latency synthesis, explicitly optimised for the real-time case rather than for offline narration.
Layer 05
Orchestration and turn-taking
Runs the conversation loop
- LiveKit Agents ↗Open source
Framework for real-time voice agents — transport, turn detection and the conversation loop. The clearest open reference for what orchestration actually involves.
- Pipecat ↗Open source
Pipeline framework for voice and multimodal agents. Good for understanding how the pieces are wired before you commit to a vendor.
- Jambonz ↗Open source
Open-source CPaaS you can self-host. Relevant when data residency or carrier control rules out a hosted platform.
Cross-cutting
Compliance and regulation
- TRAI ↗Regulator
India's telecom regulator. The source for commercial communication rules, DLT registration and do-not-disturb obligations on outbound calling.
- Meity — DPDP Act ↗Regulator
India's data protection framework, which governs call recordings, transcripts, retention and deletion. Read it before designing your storage, not after.
How this list is kept
24 entries, all project or documentation homepages rather than deep links into individual articles — deep links rot fastest, and a broken link on a page about evaluating vendors would be self-refuting.
Inclusion is not endorsement and nothing here is ranked. Know something that belongs here? Tell us.