VaaniMesh is a live voice-translation mesh for multilingual rooms. A Telugu speaker talks; the Bangla listener hears Bangla, the Hindi listener hears Hindi — each with a spoken voice and a written transcript, a few seconds behind the speaker.
Try the live demo Read the docsIndia runs its meetings, market negotiations, panchayat sessions, and family calls across language lines every day. The usual fixes all tax the conversation: someone volunteers as interpreter and stops participating, everyone falls back to a second language nobody loves, or the group splits into side conversations. Text chat translators exist — but the moment the medium is voice, in real time, with more than two languages in the room, the tooling disappears.
VaaniMesh treats the many-language room as the normal case, not the edge case: up to five participants, five languages, one conversation.
Each participant opens a browser page — a speaker console — picks their language, and talks. Utterances travel to an independent relay that transcribes them, identifies the language actually spoken, translates the meaning (never a phonetic transliteration) into each listener's language, synthesizes a voice, and routes every listener exactly their own version. An operator dashboard watches the whole mesh live.
New rooms start in manual mode: speak, review the captured clip, press Send. When the relay has proven the chain — five clean deliveries, latency in budget — each speaker may flip to automatic.
The relay stamps a mesh header on everything it forwards: source speaker, languages, per-stage latency, sequence number, audio format. What you hear is auditable.
The dashboard shows each connected speaker's mic status, loudness, settings, round-trip time, and the pipeline's rolling latencies — the same numbers we use to debug it.
Turn-based (default), streaming-simultaneous (translation begins mid-sentence), and shared-mic (one phone on the table, speakers identified by the language they speak).
Translation runs on a frontier language model prompted as a live interpreter — colloquial register, native script, fragments handled gracefully.
Recognition, translation, and speech synthesis are independent, swappable stages with a benchmark harness that scores any provider on real reference recordings.
Three wired nodes. Consoles capture and play audio; the relay — a separate server process — owns the entire AI pipeline; the dashboard observes. Nothing about a conversation lives in the browser.
A one-day hardening pass (2026-09-16) closed 17 tracked gaps — each verified by a test that actually ran, on an honest buildplan.
Both servers gate joins behind an access PIN — and wrong attempts are rate-limited (five per minute per client), so the PIN can't be brute-forced through the public tunnel.
A daily budget caps paid AI calls at the adapter itself; when it trips, the room shows a clear operator message instead of quietly billing on. Key loss degrades loudly, never silently.
Transcript content never reaches server logs by default — metadata only, logs rotate and age out, no audio is stored. The retention policy is written down, not implied.
A watcher polls both health endpoints and emails the operator the moment a service goes down or pipeline latency degrades — proven by killing a service and watching the alert arrive.
102 automated tests (server, relay, and the browser audio/caption modules) run in CI on every push to the off-machine repository.
The public entry lives on our domain and heals itself when the underlying tunnel rotates — the link you bookmark keeps working.
Every requirement, with its real status. A matrix that shows everything green is a matrix nobody believes.
| Requirement | What exists | Status |
|---|---|---|
| Five-language voice translation (te, bn, mr, hi, en) | Full chain live-verified per language, including the Hindi/Marathi shared-script disambiguation | Built |
| Three-node mesh (consoles · relay · dashboard) | Independent relay server, per-speaker settings, live telemetry | Built |
| Mesh header on every forwarded message | Source, mode, per-stage latencies, sequence, audio format | Built |
| Manual send with earned automatic mode | Pending-tray review; auto unlocks on relay-verified chain health | Built |
| Operator dashboard | Speaker cards, loudness, audio rate, RTT, rolling latencies, message log | Built |
| PIN access gate for public demos | Every lane gated (both servers' joins + dashboard); wrong attempts rate-limited with backoff | Built |
| Abuse & cost guards | Brute-force limiter on all PIN surfaces; daily budget on paid AI calls, enforced before any network request (2026-09-16) | Built |
| Privacy: transcripts out of logs | Content logging off by default (metadata only), rotating logs, written retention policy (2026-09-16) | Built |
| Continuity: backup · CI · alerting | Off-machine repository, CI running 94 server + 8 browser tests on every push, email alerting on service degradation (2026-09-16) | Built |
| Shared-mic room (one phone on the table) | Language-identification routing verified with two real recorded speakers through one mic; voice-fingerprint tie-break implemented but not yet validated with live humans | Partial |
| Streaming-simultaneous mode (translation mid-sentence) | Full machinery implemented and test-verified against protocol mocks; live operation is descoped for now — the platform runs a single provider (OpenRouter), which has no realtime speech-recognition socket | Partial |
| Provider benchmark harness | Scores any configured provider on real reference clips; live rows recorded for the OpenRouter stack; alternative-provider rows were descoped with the single-provider decision (2026-09-16) | Partial |
| Cloud-twin deployment (no-install, 24×7) | Deployment scaffold drafted; demo currently runs from a secure tunnel to our hardware | Roadmap |
| Narrated demo video | Not yet recorded | Roadmap |
| Two speakers sharing a language in shared-mic rooms | Requires acoustic diarization (design selected: streaming Sortformer) | Roadmap |
VaaniMesh is a small Python service (FastAPI + a Pipecat pipeline) plus
static browser consoles — it runs on a single modest machine under a process
manager. The demo you can try below runs on our hardware, exposed through a
secure tunnel; AI stages call cloud models through one metered API key held
server-side. An on-premises install needs only Python 3.12, one API key, and
a browser; a no-install cloud twin is on the roadmap (a deployment scaffold
ships in the repository under deploy/twin/).
Live demo, gated by PIN. Request access below — the PIN arrives by email within a minute. Then open the app on two devices (earphones recommended), enter the PIN, pick different languages, and talk. Keep the dashboard open on a third screen if you want to watch the mesh work.
Open the live app Request access by email
Demo sessions run on our hardware; the link redirects to the current session address. If it doesn't respond, the session is offline — ask us to bring it up.