The Hinglish problem: why your voice bot will fail on real Indian callers

Almost every voice AI vendor selling into India will tell you they support Hindi and English. Most of them are telling the truth. And a large share of their bots will still fall apart on a real caller, because the thing Indian callers actually do is not “speak Hindi” or “speak English”. It is this:

“Haan bhai, main interested hoon but abhi budget thoda tight hai — can you call me next month?”

One sentence, two languages, three switches. Nobody chose a language at the start of that call. The caller moved between them the way anyone does when both are theirs.

Why it breaks

The failure is architectural rather than a matter of model quality, which is why throwing a better model at it does not help.

The recogniser is usually locked to one language

Most streaming speech recognition takes a language as a parameter. You open the session as hi-IN or en-IN and it decodes against that language’s acoustic and lexical model for the whole call. Set it to Hindi and the English words arrive as approximate Hindi phonemes — nonsense that is often confidently transcribed, because the decoder’s job is to produce the most likely Hindi string, and it will.

So “budget tight hai” comes through as something plausibly Hindi and completely wrong. Set the session to English instead and the same thing happens in reverse.

Nothing downstream can recover it

The language model reasons over text. It never hears the audio. If the transcript says the wrong thing, the model answers the wrong thing — fluently, in a well-constructed sentence, with no signal that anything went wrong. The transcript is the ceiling on everything above it.

This is why the demo looked fine. In a demo, somebody speaks one language carefully into a good microphone.

Then the same problem in reverse, on the way out

Speech synthesis has the mirror version. A Hindi voice reading an English word often applies Hindi phonology to it, and an English voice reading Devanagari either mangles it or refuses. A reply that mixes both — which it must, if the caller did — needs a voice that can carry both without switching speaker mid-sentence, or the bot appears to hand the call to a different person halfway through.

Why the blended accuracy number hides it

Ask a vendor for intent accuracy and you will get a single figure across all traffic. If code-switched utterances are, say, a fifth of the calls, a catastrophic score on that fifth moves the blended number by only a few points — well inside the range where it still reads as a good result.

87% sounds like a working system. It is a working system for four callers in five and a broken one for the fifth — and the fifth is not randomly distributed. Code-switching correlates with comfort, speed and emotion, which means it clusters exactly where the conversation matters: the annoyed caller, the hurried one, the one being sold to.

Those figures are illustrative arithmetic, not measurements from any product. The point is the shape: a blended average cannot tell you whether the hard subset works.

Then it gets harder: this is not one problem

“Hinglish” is a convenient label for something that varies by region, register and speaker:

• Insertion. An English noun in a Hindi frame — the most common, and the most tractable. Domain vocabulary and hints help here.
• Alternation. Whole clauses switching, often at a comma. Harder: both halves need full decoding.
• Anglicised morphology. English stems with Hindi grammar — “update kar dijiye”, “cancel ho gaya”. Neither language’s lexicon contains the resulting form.
• Numbers and identifiers. Amounts, dates and account numbers get spoken in whichever language is faster, sometimes changing mid-string. This is where errors are most expensive, because they are silent — a wrong digit is still a valid digit.

And none of it is Hindi-specific. Marathi-English, Tamil-English, Telugu-English and Bengali-English behave the same way, with different phonetics and often much thinner training data.

What actually works

This is solvable. Several approaches do work, and the reason to know them is so you can tell whether a vendor has done anything at all:

• Genuinely multilingual recognition, decoding without a fixed language assumption, rather than a per-session language switch.
• Training or fine-tuning on code-switched speech from the target market — which is scarce and expensive, and therefore a real differentiator when a vendor has it.
• Romanised-script handling, since Hinglish is very often written in Latin script, and any retrieval or matching downstream has to cope with that.
• A single voice that carries both languages, so the reply does not change speaker mid-sentence.

How to test it in twenty minutes

You do not need a procurement process to find out. You need ten recordings and a phone.

1. Pull ten real calls from your own contact centre — the messy ones, not the clean ones. Redact them.
2. Ask every candidate to run those exact files and return the raw transcripts. Not a summary, not a score. The transcripts.
3. Read them against the audio yourself. You will not need a metric. The failures are obvious to anyone who speaks both languages.
4. Then call the demo number and interrupt it in Hinglish. Switch languages mid-sentence, then interrupt the reply. Interruption and code-switching together is the real test, and it is where assembled-last-month pipelines come apart.

A vendor who cannot run ten of your files within a week is telling you something about their implementation timeline as well as their recognition.

Where this sits in the stack

Code-switching lands mostly on speech recognition, with a second failure at synthesis. It is not a language-model problem, which is why it survives a model upgrade untouched.

In this directory, code-switching is recorded as a field separate from language proficiency, and you can filter the directory to providers who claim it — with the proof level attached, so you can see whether anyone has checked.