Voice AI Interviews in Indian Languages: Accuracy and Pilots
Voice AI interviews in Indian languages: how accurate, and how to test
If you plan to run research interviews by phone in Indian languages, the AI’s questions are rarely the weak point. The transcript is. Everything the researcher later reads, codes and reports depends on how well the system turned a phone call in Tamil or Odia into text. This post sets out what the best public benchmark found, why real fieldwork is harder still, and how to pilot each language before you scale. It is a deep dive from our guide to AI interviews for research.
What the benchmark found
The Voice of India benchmark (April 2026, built by Josh Talks with AI4Bharat and IIT Madras) tested speech recognition on unscripted telephone speech across 15 languages. For the best system in its table:
| Language | Word error rate (best system) |
|---|---|
| Hindi | about 5% |
| Tamil | 14.2% |
| Telugu | 18.2% |
| Malayalam | 18.9% |
District-level error ranged from about 4% to 44%. Some widely used global models did far worse on Indian phone audio than their public benchmark scores suggested. In plain terms: a model that looks excellent on a leaderboard can still mishear one word in five on a call from a village in your sample.
Treat the figures as a guide, not a guarantee. The benchmark comes from commercial parties, and we use only its per-language table figures.
Why your fieldwork will be harder than the benchmark
Phone research in India adds conditions no benchmark fully captures: shared handsets, background noise, people switching between languages mid-sentence, regional dialects, and patchy network. J-PAL’s phone-survey guidance adds the human side: keep calls under about 30 minutes and plan for around 50% response on new samples. An AI interviewer inherits all of it.
Two design choices help. First, let participants pick their language and switch mid-call; Zolabs supports this across 11 interview languages (English plus Hindi, Tamil, Telugu, Kannada, Malayalam, Bengali, Punjabi, Gujarati, Odia and Marathi). Second, always keep the original audio and the original-language transcript next to the English translation, so a reviewer can go back to source.
Meaning matters more than word error rate
Word error rate counts wrong words. Research cares about wrong meanings. A 15% error rate can still carry the right meaning, or it can flip a “no” into a “yes”. Code meaning, not words, and never analyse only the English translation. Only listening tells you which kind of error you have.
How to pilot each language: step by step
- Pick 20-30 pilot calls per language, spread across the districts and dialects in your real sample, not just the easiest to reach.
- Assign a fluent reviewer per language who did not write the guide.
- Have the reviewer listen to every pilot call in full and compare it with the original-language transcript and the English translation.
- Log errors that change meaning, especially reversed answers, wrong numbers and wrong names of places or services.
- Check accuracy by district and dialect, not just by language. A language that passes overall can fail in one district.
- Check probing, not just transcription: did the AI follow up where it should, in that language?
- Decide per language: go, fix and re-pilot, or use human interviewers for that language.
- Keep listening at scale. After launch, have a fluent reviewer listen to a share of calls in full. J-PAL suggests back-checking 20-25% of phone surveys; the field data quality logic is the same as in our guide to back-checks and audio audits.
| Column | What to record |
|---|---|
| Call ID | From the platform export |
| Language / dialect | As spoken, not as assigned |
| District | From the sample frame |
| Reviewer | Fluent reviewer's initials |
| Meaning-changing errors | Count, with timestamps |
| Reversed answers (yes/no flips) | Count, with timestamps |
| Probing quality | 1-5, with a one-line reason |
| Drop-off point | Question number, if the call ended early |
| Language switches | Yes / No; handled well? |
| Reviewer verdict | Go / Fix and re-pilot / Human-led |
| Summary tab: totals per language and per district; any reversed answer flagged in red. Suggested starting rule (set your own with your research lead): any district with repeated reversed answers stays human-led until a re-pilot passes.] | |
Who gets left out
Voice-first calls reach people text-based tools exclude, but not everyone. People without their own phone, people who are away during call hours, and people who share a handset with a family member who answers first can drop out silently. Track response by gender, location and language from the first wave, and add in-person interviews for any group the calls don’t reach. If your interviews involve health or other sensitive information, check your consent flow against our plain-English guide to the DPDP Act for NGOs.
Where this leaves you
Voice AI in Indian languages is good enough to pilot and not yet good enough to trust without listening. Hindi is ahead; Tamil, Telugu and Malayalam need closer review; district matters as much as language. For what the research says about AI interview quality more broadly, read AI interviews vs human interviewers.
Want help designing a multilingual pilot? Book a free 30-minute AI Clinic →