Cue's dictation has a polish step: the raw transcript, with its filler words and missing punctuation, is turned into text that reads the way the speaker meant it. Where that step runs was decided by a measurement, and the measurement is public. This post follows Google DeepMind's case study, figures current as of May 2026, and adds nothing the page does not state.
The original design
The step was built to use a cloud model, with a local model as the fallback, on the assumption that a larger model would polish more accurately. The team's stated goal for the step was text that read like the user rather than like a model: punctuation restored, sentences segmented, fillers removed, without imposing a house style.
The measurement
The team benchmarked the step on 227 real voice samples covering English, other languages and mixed-language input. On that benchmark Gemma 4 E4B, running locally through Ollama on Apple Silicon, was chosen as the first-choice model. The median latency of the polish step fell from 876 ms to 488 ms, which the page describes as inside the budget the team had set for a step that has to feel faster than typing.
The reversal
After the benchmark, the architecture was inverted: the local model became the default and the cloud model the fallback. The desktop app checks at startup whether Ollama is running; if it is not, the polish step routes to a cloud model and dictation continues.
What changed for users, per the page
- Per-user dictation rose by about 30 percent across active beta users in the four weeks before and after the switch; users who had dictated short messages began dictating longer ones.
- The marginal inference cost of the step dropped to zero, and the page gives that as the reason dictation is unlimited on every tier, including the free one.
The boundary of the claim
Only the polish step moved. Transcription is done by a cloud speech-to-text service before the polish step, and Cue's agent mode relies on a cloud model. The page reports early evaluations of Gemma 4's function calling for self-contained agent tasks and calls them early. Our home page therefore says the polish runs on the user's machine and says nothing about the agent running locally; the platforms page records the same boundary.