Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

voice-ai-landscape

A practitioner's map of the 2026 voice-AI / TTS landscape — with ElevenLabs in focus.

License: MIT Prose: CC BY 4.0 Part of: AI/ML Lab Website Snapshot

Why this repo

I've spent thirty years building and selling technology, and I've always believed you don't really understand a market until you've built in it. This is a builder's map of voice AI as it stands in 2026 — who leads on what, how to read the benchmarks without fooling yourself, and what it actually takes to put a voice agent into production — written by someone who both analyzes the field and calls the APIs. ElevenLabs is kept in focus throughout because it sets the reference point much of the market measures itself against.

Snapshot discipline. Every quantitative claim below is dated and linked in §9. The voice-AI leaderboards move weekly — treat the numbers as a June 2026 snapshot, not gospel, and re-check the sources.

Contents

  1. The voice-AI stack
  2. The landscape, at a glance
  3. How to read the benchmarks (without fooling yourself)
  4. ElevenLabs in focus
  5. The enterprise & agentic angle
  6. The European lens
  7. Curated repos & resources
  8. Hands-on artifacts in this repo
  9. Method & sources

1. The voice-AI stack

"Voice AI" is several markets wearing one coat:

  • Text-to-speech (TTS) — text in, speech out. The layer this repo focuses on.
  • Speech-to-text (STT/ASR) — the inverse; the other half of any voice agent.
  • Real-time conversational agents — STT → LLM → TTS in a sub-second loop, with turn-taking, interruption (barge-in), and tool use.
  • Voice cloning — building a custom voice from a short reference sample.
  • Dubbing & localization — translating and re-voicing existing audio/video.

The dimensions that actually decide a deal:

Dimension Why it matters
Expressiveness Naturalness, emotion, emphasis — the thing demos sell on.
Latency (TTFA) Time-to-first-audio governs whether a conversation feels live.
WER / intelligibility Word-error-rate; can the listener actually parse it?
Multilingual coverage Languages and accents, not just one flagship locale.
Voice library / cloning Breadth of ready voices + custom-voice quality.
Price at scale $/1M characters once you're past the free tier.
Licensing Proprietary API vs. open weights vs. permissive (MIT).
Deployment Cloud-only vs. self-host / on-prem (matters for compliance).

No single model wins every column — which is exactly why a map is useful.

2. The landscape, at a glance

Snapshot: June 2026, basis = vendor docs + the Artificial Analysis Speech Arena + third-party 2026 round-ups. The table is generated from data/landscape.csv (the single source of truth) by scripts/render_table.py; every numeric cell is sourced in §9.

System Vendor Category AA Arena Elo Latency Price / 1M chars Languages Licensing Notable positioning (see source)
Eleven v3 / Flash v2.5 / Turbo v2.5 ElevenLabs TTS + Realtime 1197 Flash v2.5 targets sub-300ms streaming Usage-based; premium tier v3 70+; Flash/Turbo 32 Proprietary API Quality/expressiveness benchmark; 10k+ voice library; 70+ languages on v3; conversational agents + dubbing; consistently #2-3 on the AA TTS Arena with the largest sample size
Realtime TTS 1.5 Max / Realtime TTS-2 Inworld AI Realtime TTS 1238 Realtime-optimized pipeline $35/M list; $5-10/M at scale/enterprise Multilingual Proprietary API Top of the AA TTS Arena in early-mid 2026 (held 3 of top 5 in some snapshots); full speech pipeline; aggressive price-at-scale
Gemini 3.1 Flash TTS Google TTS 1211 Low latency Low Multilingual Proprietary API Near the top of the AA Arena at low price (ranked #2 globally in some late-May 2026 snapshots within ~4 Elo of the leader)
MiniMax Speech 2.8 HD / 2.8 Turbo MiniMax (Hailuo) TTS - Turbo tier for lower latency ~$40-50/M Multilingual Proprietary API Premium-tier quality near the Arena leaders; HD and Turbo variants released Feb 2026
Sonic-3 / Sonic 3.5 Cartesia Realtime TTS - ~90ms TTFA (optimized); P50 ~188ms; SSM (non-transformer) architecture Usage-based Multilingual Proprietary API Built for streaming and low tail-latency; Sonic 3.5 reached the AA Arena top 5; consistent P99
Aura-2 Deepgram Realtime TTS - Sub-300ms target (third-party P50 ~313ms) Usage-based English-focused + multilingual Proprietary API Low-latency voice-agent focus; pairs with Deepgram Nova STT for full pipelines
gpt-4o-mini-tts / tts-1 OpenAI TTS - Streaming available Low Multilingual Proprietary API Simple cheap multilingual synthesis; fewer creative/expressive controls than the leaders
SIMBA 3.0 Speechify TTS - Not reported Low (vendor claim) Multilingual Proprietary API Vendor-reported break into the AA Arena global top 10 in 2026 outranking some incumbents (Speechify press release - treat as vendor claim)
Chatterbox Multilingual v3 / Chatterbox Turbo Resemble AI TTS (open) - Chatterbox Turbo ~75ms class Free (self-host) / usage-based hosted 25 (incl. 4 dialects) on v3 MIT (open source) SoTA open-source TTS on a 0.5B Llama backbone; PerTh watermark embedded by default with an open detection library - provenance-ready for EU compliance
S2 / S2 Pro Fish Audio TTS (open) - Streaming available Low; free self-host Multilingual Open weights Competitive quality at low price with an open-weights option

A blank Elo cell means I don't have a verified Arena score for that system at this snapshot — not that it's unranked. Only scores I could source are charted:

Artificial Analysis TTS Arena — Elo, verified subset (late May 2026)

3. How to read the benchmarks (without fooling yourself)

The single most common mistake in this market is reading one leaderboard as if it settled the question. It doesn't:

  • Elo ≠ WER ≠ streaming latency. The Artificial Analysis Speech Arena Elo measures single-prompt naturalness as judged in blind A/B preference votes. It says nothing about production latency, word-error-rate, or how a model behaves under streaming load. A model can top the Arena and still be the wrong call for a sub-200ms voice agent.
  • Arenas move weekly. Across sources gathered for this snapshot, the same week had Inworld's Realtime TTS 1.5 Max at #1, Google's Gemini 3.1 Flash TTS within ~4 Elo at #2, and ElevenLabs v3 at #2–3 with by far the largest vote sample — the ranks reshuffle as checkpoints ship. Any "#1 TTS" claim is a timestamp, not a fact.
  • Vendor benchmarks are marketing until proven otherwise. Press-release claims ("we beat X on the leaderboard") are flagged as vendor claims in the table. Trust the neutral arena and your own measurements over a vendor's blog.
  • Your numbers are the only ones that count. Latency depends on your region, network, audio format, and concurrency. That's why this repo ships a tiny latency micro-benchmark you run yourself rather than a borrowed latency table.

4. ElevenLabs in focus

ElevenLabs is the reference point much of the market measures itself against, and its lineup is deliberately tiered:

  • Eleven v3 — the expressiveness flagship: emotion, emphasis, audio tags, and the broadest language coverage (70+ languages). The quality benchmark, and the model carrying the largest vote sample in the Arena top tier.
  • Flash v2.5 / Turbo v2.5 — the latency tiers (32 languages), built for real-time agents; Flash targets sub-300ms streaming and is recommended over Turbo for essentially all use cases.
  • Breadth — a 10,000+ voice library, plus voice cloning, Conversational AI agents, and dubbing in one platform.

Where it leads: creative expressiveness, voice-library breadth, and a mature product surface (agents + dubbing) that few challengers match end-to-end.

Where challengers push: price-at-scale (Inworld lists $5–10/M at volume vs. ElevenLabs' premium tier), raw streaming latency (Cartesia's SSM-based Sonic, ~90ms optimized TTFA; Deepgram Aura-2 for agents), and open weights (Resemble's MIT-licensed Chatterbox, Fish Audio). The story of 2026 is less "is ElevenLabs good" — it plainly is — and more "for this workload, is the quality premium worth the price and latency, versus a cheaper or self-hostable alternative?"

5. The enterprise & agentic angle

The interesting frontier isn't a better demo voice; it's deploying voice agents into real enterprises — telecom, financial services, healthcare, retail. That's where a vendor's product depth meets an operator's reality, and it's the part of this market I care about most. What "deploying voice AI" actually involves:

  • Latency budgets. The agent loop is STT → LLM → TTS. If you want sub-second turn-taking, every hop has a budget; TTS TTFA is one line in a P99 spreadsheet, which is why streaming-first engines matter more than Arena Elo here.
  • Compliance & data residency. Regulated industries ask where audio is processed, whether voices are watermarked, and whether the stack can run in-region or on-prem — pushing open-weights / self-host options up the shortlist.
  • Observability. Production voice needs logging, eval harnesses, barge-in handling, and fallback voices — the unglamorous plumbing that decides whether a pilot survives contact with real call volume.
  • Total cost at scale. Per-character pricing that looks trivial in a demo is a real line item at millions of minutes; this is where price-at-scale challengers win deals the Arena would never predict.

This is the lens a general manager brings: not "which voice sounds best in a quiet room," but "which stack clears the latency, compliance, and cost bars for this deployment."

6. The European lens

For a Europe-based deployment there's a hard date on the calendar. Under the EU AI Act, Article 50 (transparency obligations), providers of generative AI must mark synthetic audio in a machine-readable format identifying it as AI-generated. The obligations become enforceable on 2 August 2026, with a grandfathering grace period to 2 December 2026 for systems already on the market before that date; penalties reach €15M or 3% of global turnover. (Legal status evolving — verify against the official text before relying on it.)

The practical consequence: provenance becomes a feature, not a nicety. Embedded watermarking like Resemble's PerTh (shipped by default on open-source Chatterbox, with an open detection library) and broader C2PA / SynthID-style content credentials move from "good citizenship" to a procurement checkbox for any EU deployment. A voice stack that can prove what it generated has a real edge with European buyers — a differentiator worth weighing alongside quality and latency.

7. Curated repos & resources

Annotated, and verified to resolve as of June 2026:

  • resemble-ai/chatterbox — SoTA open-source TTS (MIT), 0.5B Llama backbone, PerTh watermark by default. The open-weights model to beat; the free baseline used in this repo's benchmark.
  • fishaudio/fish-speech — open-weights multilingual TTS; competitive quality at low/zero marginal cost if you self-host.
  • coqui-ai/TTS — foundational open-source TTS toolkit; widely forked but note its maintenance status (the company wound down; the community carries it). Historically important rather than a 2026 frontier pick.
  • Artificial Analysis Speech Arena — the neutral blind-preference leaderboard this repo cites. Read it as naturalness, not latency.

8. Hands-on artifacts in this repo

This isn't only an essay — the claims are backed by code you can run:

  • scripts/render_table.py — regenerates the comparison table and the Elo chart from data/landscape.csv. Deterministic, no keys: uv run --with matplotlib python scripts/render_table.py.

  • bench/ — a tiny latency micro-benchmark that synthesizes one fixed multilingual sentence and records time-to-first-audio across engines. It runs a free local neural baseline (Piper, no key) out of the box and adds ElevenLabs when you drop a key in .env. It is plumbing, explicitly not a quality benchmark — see bench/README.md and bench/samples/RESULTS.md.

    A nice illustration from my own run (2026-06-19, 3 runs each): the local Piper baseline had the lower time-to-first-audio (~174 ms vs ElevenLabs Flash v2.5's ~295 ms, which pays a network round-trip), yet ElevenLabs returned the full clip roughly 2× faster (~482 ms vs ~1017 ms) thanks to cloud GPU throughput. TTFA and total latency are different questions — exactly why a borrowed latency table can mislead.

9. Method & sources

The table is built from data/landscape.csv; each row carries its own source and verified date. Numbers were re-verified against live sources on 2026-06-19; the voice-AI leaderboards change weekly, so re-check before reuse.

Primary sources

Prose and figures in this repo are © 2026 Antonio Rodriguez-Moral, licensed CC BY 4.0; code is MIT.


🌐 arodmor.me · 💻 github.com/arodmor · ✉️ antonio.rodriguez.moral@pm.me

Part of a series: AI/ML Lab · voice-ai-landscape · arc-agi · recursive-reasoning-models

About

A practitioner's map of the 2026 voice-AI / TTS landscape, with ElevenLabs in focus.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages