Beyond TTFT: Measuring the Latency Voice AI Callers Actually Hear

When a language-model provider says its time to first token is 200 milliseconds, it is tempting to conclude that a voice agent should begin answering in roughly the same time.

However callers don't hear tokens, they hear audio after the caller's turn has been detected, a response has been generated, enough text exists to synthesise natural speech, the speech provider has returned audio, the output buffer has released it, and the telecommunications network has carried it back to the phone.

That is why time to first token is a useful component measurement but a poor description of conversational latency.

This article proposes a more useful end-to-end metric: Time to First Playable Phrase, or TTFPP.

Quick definition

Time to First Playable Phrase is the elapsed time between the caller actually finishing a turn and the first coherent, correctly paced phrase becoming audible to that caller.

It is deliberately stricter than time to first token, time to first byte, or even time to first audio. A few PCM samples are not a conversational response. Nor is an isolated word that has been synthesised without enough context to sound natural.

The more relevant metric in a phone conversation is:

How long did the caller wait before they could hear that the agent had understood them and begun a usable response?

Why the usual latency metrics are incomplete

Voice AI is a pipeline of overlapping systems. Each supplier naturally measures the part it controls.

Groq, for example, distinguishes time to first token from total completion latency in its latency guidance. Realtime TTS systems also describe an explicit quality trade-off: ElevenLabs documents that smaller text buffers can reduce latency but give the model less context for natural synthesis.

Both measurements are valuable but they do not explain when a telephone caller hears a useful reply.

Defining the start of the measurement

The start timestamp is more difficult than it appears.

If it is taken from the STT provider's final packet, the measurement includes the provider's endpointing behaviour but may not represent the actual end of speech. If it is taken from a Voice Activity Detection event, a pause inside an unfinished thought can be mistaken for the end of the turn. If it is reconstructed from an application log, clock drift and asynchronous packet delivery can distort the result.

For a reproducible benchmark, we recommend keeping two timestamps:

1. Acoustic end of turn: the labelled point in the caller recording at which the final intentional speech sound ends.
2. System turn acceptance: the point at which the runtime commits to treating the caller's input as a complete turn.

The difference between them is the turn-detection cost. Keeping it separate prevents an aggressive endpointing configuration from appearing fast when it is actually cutting callers off.

Defining a playable phrase

The end timestamp should be taken at the first audio sample of a response that meets four conditions:

• It is part of the response the caller ultimately hears, not audio generated and discarded after an interruption.
• It contains enough language to communicate a stable conversational action.
• It is not a filler, cough, backchannel or holding sound unless the benchmark explicitly measures those behaviours.
• Its pacing and pronunciation are acceptable under the same quality rules used for the rest of the response.

This definition prevents a system from winning a benchmark by emitting "I..." quickly and then pausing while the useful answer is generated.

For most cascaded systems, the first playable phrase will be a clause or short sentence fragment. The exact boundary should be recorded alongside the measurement so results remain auditable.

The latency event model

A complete trace should record at least these events under one response identifier:

1. Caller acoustic speech ends.
2. VAD changes from speech to silence.
3. The accepted transcript becomes available.
4. The runtime accepts the caller's turn.
5. The LLM request begins.
6. The first content token arrives.
7. The first speakable phrase is assembled.
8. The TTS request begins.
9. The first audio byte arrives.
10. The first decoded audio frame is ready.
11. The output gate releases the response.
12. The first phrase reaches the telecommunications recording point.

Without the response identifier, late packets from an interrupted response can be incorrectly attributed to the response that replaced it.

Evidence from our training analysis

Through Voxworks development and analysis, we measured a privacy-minimised staging extract containing 102,201 selected latency rows. Across the attributable instrumented responses in that evolving corpus:

• Median recorded total response latency was approximately 1.645 seconds.
• The 90th percentile was approximately 3.115 seconds.
• Median LLM time to first token was approximately 293 milliseconds.
• Median TTS time to first byte was approximately 237 milliseconds.

These figures should not be interpreted as one immutable product benchmark. The corpus spans changing models, architectures, test scripts and instrumentation definitions. It is useful because it illustrates the gap between fast component medians and the much larger delay experienced by the complete system.

The LLM and TTS medians account for only part of the end-to-end median. Turn detection, phrase assembly, state coordination, output gating, network delivery and tail events account for the rest.

The p90 is an important because a conversation containing five apparently normal turns and one three-second pause will be remembered for the pause.

Why faster text can produce worse speech

Streaming is necessary for low-latency voice AI, but indiscriminate streaming can move the bottleneck rather than remove it.

Consider an LLM that emits:

"Your appointment is available on Thursday at three thirty."

A TTS engine receiving the first word immediately has very little context for prosody. It does not yet know whether "Your" begins a statement, question, correction or list. Feeding isolated words can produce audible joins, unstable pace and misplaced emphasis.

Buffering the complete sentence produces better speech but throws away much of the latency benefit. The practical solution is usually to identify the earliest stable clause or phrase boundary, begin synthesis there, and continue streaming subsequent text with retained context.

This is why TTFPP should be measured together with a basic quality gate. Otherwise, a benchmark rewards audio that arrives quickly but sounds way worse.

Report a distribution, not one number

Every published result should include:

• p50, p75, p90, p95 and maximum TTFPP;
• the number of calls and measured turns;
• conversational turns separately from tool/API turns;
• interruption and recovery turns separately from clean turns;
• warm and cold connection results;
• the recording point used for the final timestamp;
• the telephone codec and network path;
• the model, region and configuration versions;
• the percentage of turns excluded and why.

A mean without a distribution conceals the pauses that damage real conversations. A provider benchmark without the recording point can also be structurally optimistic if it stops before audio reaches the telephone network.

A worked measurement example

Suppose a labelled call trace records:

The component metrics are:

• Turn-detection cost: 310 ms
• LLM TTFT from request: dependent on the request timestamp
• Phrase-assembly delay after first token: 115 ms
• TTS and delivery time after phrase assembly: 390 ms
• TTFPP: 1,080 ms

Optimising only the LLM TTFT would miss most of the available improvement.

What buyers should ask voice AI vendors

When a vendor advertises sub-second latency, ask:

1. Where does the timer start and stop?
2. Is the result measured at the application or on the telephone recording?
3. Does it include endpointing?
4. Is the first audio a filler or the actual response?
5. Are the results p50 or p95?
6. Are tool calls included?
7. Which telephone codec and geographic route were used?
8. How many turns and calls were measured?
9. How were interruptions and false cut-offs scored?

If those questions cannot be answered, the number is marketing, not a benchmark.

The proposed standard

TTFPP does not replace TTFT, TTFB or time to first audio. It connects them to the experience they are intended to improve.

A useful voice AI latency report should therefore show three layers:

• Component latency: STT, LLM and TTS measurements.
• Orchestration latency: endpointing, checks, phrase assembly, state coordination and output gating.
• Caller-experienced latency: Time to First Playable Phrase at the telecommunications boundary.

That structure makes optimisation honest. It also makes results comparable across architectures whose internal components may be completely different.

For a broader introduction to the pipeline, see Why Voice Is the Hardest Problem in AI. For the underlying concepts, see What Is Latency in AI Voice Calls?.

Limitations and next research step

No single latency metric captures conversational quality. A system can be fast and still interrupt callers, misunderstand speech or generate an inappropriate response. TTFPP should be reported beside false-cutoff rate, task correctness, barge-in latency and speech-quality measures.