specced · zero lines of the dual-plane build shipped — production voice agent today still runs the pipelined dispatcher/responder fallback
Every build-vs-buy argument about voice agents gets asked the same wrong way: build the whole stack, or buy a platform and skin it? That’s a single question about a single system, and it produces a single answer that’s wrong for half the system no matter which way you pick. The real system has two clocks running inside it, one with a sub-second budget and one with a budget measured in seconds, and each clock deserves its own answer. Rent what lives in the first one. Own what lives in the second. The line I wrote for myself when I first laid this out: rent the reflex arc; own the brain, memory, rubric, channels, and eval rig. The moat was never the audio model.
The reflex arc is a commodity, and that’s the point
The reflex arc is everything a live call needs inside about a second: who’s talking, whether the caller just finished or only paused, when to yield the floor the instant they start speaking over you. Turn-taking. Barge-in. Speech in, speech out, no text round-trip in between. None of it is where a product’s identity lives. All of it is where a bad millisecond is visible to a human on the other end of the line.
That layer is rentable today. OpenAI’s Realtime API does full-duplex speech-to-speech over WebRTC, WebSocket, or SIP. Gemini Live is a credible second source behind the same interface. The substrate underneath — the media transport, the room/session plumbing — has an open-source option (LiveKit) specifically so the fast plane isn’t hostage to one vendor’s uptime. None of that is a reason to build your own; it’s a reason to buy carefully, with a fallback path, and stop treating “we built our own real-time speech stack” as a credibility flex. It’s closer to building your own load balancer in 2026.
The tuning problem inside that layer is real, though — renting doesn’t mean it’s trivial, only that it’s not yours to solve from scratch. The central tension is semantic end-of-turn detection versus energy-only voice-activity detection. A caller who says “OK, go ahead” and then goes briefly quiet has finished their turn. A caller mid-sentence who pauses 800 milliseconds to think has not. Get that wrong in the direction of over-triggering and you talk over people. Get it wrong the other way and you get dead air — measured once, against a competitor’s own consumer voice-mode call, as roughly nine seconds of silence after a candidate finished speaking, before the candidate had to re-prompt the system that was supposed to be listening. Nine seconds is not a rounding error on a phone call. It’s the entire call, from the caller’s side.
The KPIs that came out of studying that failure are specific: time-to-first-audio p95 under 1.2 seconds, a backchannel watchdog that fires if nothing has gone out by 1.5 seconds, and disfluency tolerance around 800 milliseconds before treating a pause as a real end of turn. Those numbers exist to beat a documented failure, not to sound impressive. A separate, already-measured baseline for the pipelined predecessor — the older speech-to-text-then-LLM-then-text-to-speech architecture — puts median per-turn latency at 3,760 milliseconds across 173 real production turns. The dual-plane design’s stated bar is beating that by more than 3x on the median. Whether it actually does: I don’t have a post-ship measurement for the dual-plane system, because it hasn’t shipped. That’s the honest state of that number today — a target, not a result.
Everything with a real time budget is where the product lives
Sub-second is a reflex. Two to fifteen seconds is enough time to think, and thinking is what a rented API doesn’t do for you. That’s the deep plane: a set of workers that subscribe to the live transcript stream and inject guidance back into the conversation without ever blocking a turn. Named roster, five of them:
- Director runs the stage machine — a call moves through OPEN, CONSENT, RAPPORT, LOGISTICS, EXPERIENCE, DEPTH, VALUE_GIFT, RECAP, NEXT_STEPS, CLOSE — and computes the next-best-question at each stage instead of following a fixed script.
- Extractor pulls structured data out of unstructured talk: commute, availability, compensation expectations, stated preferences, experience claims, written into a schema the matching engine can actually read.
- Scorer keeps a running rubric score against the job description as the call unfolds, not a single number computed after the fact.
- Sentinel watches for compliance drift — more on this below, because it’s the piece that makes “own the brain” a liability claim, not just a product one.
- Scribe keeps a rolling summary that powers the recap stage live and becomes the post-call artifact.
None of these five sit in the reflex arc’s critical path. They read the transcript as it streams and write guidance back as context updates the fast plane can pick up on its next turn. If a worker is slow, the call doesn’t stall — it just doesn’t get that worker’s input on that particular turn. That asymmetry is the whole argument for why this plane gets owned: a vendor selling you sub-second speech has no reason to also sell you a stage-aware interview director tuned to one company’s rubric, and even if they offered it, “the brain” is the part of the product a competitor can’t just also buy from the same vendor.
A compliance watchdog that runs beside the call, not inside it
Sentinel is worth separating out because it’s a different shape of ownership decision than the other four. Director, Extractor, and Scorer exist to make the product better. Sentinel exists to keep the product from creating legal exposure while nobody’s watching the transcript in real time. Its job is protected-class drift detection — catching the conversation wandering toward age, family status, religion, health, national origin — and issuing a redirect back into the flow without ever blocking the live turn to do it. A compliance layer that has to interrupt the call to work is a compliance layer nobody will keep enabled. One that runs concurrently and quietly redirects is a compliance layer that survives contact with a real product deadline.
The receipts behind why this needs to be a standing worker and not a hope: the design mandates AI disclosure in the greeting sentence as non-negotiable, and a recording/transcription consent line in the opening — because Washington is an all-party-consent state, and the “yes” has to be logged, not assumed. Personality questions stay work-style-framed and explicitly not clinical or diagnostic. And before this scales past one jurisdiction, the spec names three specific regimes that need a counsel pass first: Illinois’s AI Video Interview Act, New York City’s Local Law 144, and Colorado’s AI act. None of those are hypothetical future problems. They’re named, current, and flagged as a blocker to multi-state scale, not a nice-to-have.
Here’s the honest complication, though, and I’m not going to bury it: as of the freeze note on the design page this doctrine came from, Sentinel was explicitly listed among the ideas mined from the original spec but not yet ported into either of its two successor documents. The pattern is right. The worker that embodies it, as far as the paper trail shows, had not yet made it into the build spec that actually ships. “Own the brain” is a clean sentence. Whether the compliance half of that brain is currently running code or currently a very well-argued paragraph is a distinction worth keeping, and right now it’s the second one.
The event is the product, not the audio
Ownership of the deep plane shows up concretely in what gets emitted when a call ends. This is the interface contract between the voice layer and everything downstream of it — the matching engine, a dashboard, the next channel in the sequence:
screen.completed {
candidate_id, tenant, source: applied|pool_offered,
transcript_uri, audio_uri,
extraction {commute, availability, comp, preferences, experience[]},
rubric_scores, flags[], recap, next_step,
consent {ai_disclosed: bool, recording_ok: bool, present_to_employer: bool|null}
}
Two fields in there are doing more work than they look like. consent is Sentinel’s compliance guarantees made queryable after the fact, not just enforced during the call. And source: applied|pool_offered is a provenance tag baked into the schema from day one specifically so a future “sponsored result” model — presenting a pool candidate alongside applicants — has clean billing data to work from, before that business model exists. That’s what owning the brain actually buys you that renting the reflex arc never could: the event schema is a product and monetization decision, not a latency decision, and no speech vendor is going to design your billing model for you.
There’s a smaller doctrine buried in how this whole design got specified in the first place, worth one paragraph: the stakeholder who scoped the product handed over a role-play recording of a hostile candidate call instead of a written requirements list, and the recording became the actual spec — timestamped, second by second, more authoritative than anything said out loud about what the agent should do. A recording captures behavior nobody would think to write down as a rule; a conversation about preferences captures whatever the speaker remembers to say. Own-the-brain includes owning that judgment call about which artifact counts as ground truth, and a rented reflex layer has no opinion on the question at all.
A doctrine is not a deployment
I want to be precise about what’s actually true here, because the honest version is less impressive than the doctrine sounds. The document this whole architecture comes from is marked frozen — superseded as the literal build spec by two successor documents I don’t have visibility into from here, so I can’t tell you whether the dual-plane system it describes has since been built, partially built, or shelved. What I can tell you, because a sibling piece in this same corpus says so directly, is that the voice agent actually running in production as of this writing is still the older pipelined architecture — a fast triage model and a slower response model, not a rented sub-second speech API and five async Claude workers. The dual-plane design names that pipeline explicitly as a fallback rung, not the current state. So the claim in this essay’s title is a real doctrine I hold, with a real reason behind it, applied to a system that exists today on paper and does not yet exist today as a thing you could call.
That’s not a reason to throw the doctrine out. The reasoning holds independent of whether the specific build shipped: sub-second speech-to-speech is a genuinely solved problem being sold by multiple credible vendors right now, and no amount of building it yourself buys you a moat over a company willing to pay the same API bill. Multi-second orchestration, a scoring rubric tuned to one company’s hiring bar, a compliance watchdog tied to specific state law, and an event schema that anticipates a business model that doesn’t exist yet — none of that is for sale anywhere, because it’s specific to what you’re actually trying to build. That’s the whole test: ask which time-constant band the differentiation lives in, then decide who owns which band. The test doesn’t require the dual-plane system to be shipped to be correct. It just means I’m describing an architecture I believe in, not one I’ve watched hold up under a live caller yet.
What I don’t know yet
I don’t know whether the two successor documents this design was frozen in favor of kept the same rent/own split, or changed it. I haven’t read them. I’m reporting what the frozen page says about its own supersession, not what replaced it.
I don’t know if Sentinel, once it’s actually wired into a live transcript stream, catches protected-class drift at a rate that would hold up to an audit, or only catches the cases someone thought to write a redirect for. A compliance worker that’s never run against real adversarial traffic is a well-reasoned plan, not a measured safeguard, and I’m not going to call it the second thing until it’s been tested as the second thing.
And I don’t know how durable “rent the reflex arc” is as a vendor-neutral claim versus a fact about today’s market. The reason the split assigns speech-to-speech to a vendor isn’t philosophical — it’s that no foundation model lab building the deep-plane reasoning also ships a competitive real-time audio model today. If that stops being true — if the lab doing your reasoning also starts renting you sub-second speech in the same API call — the two-plane architecture doesn’t get wrong, but the reason to keep the seam between vendor and owned code gets weaker than “it’s a different company” and starts depending on something closer to what I actually don’t know: whether owning the deep plane stays valuable once owning it and renting the reflex arc stop requiring two separate contracts.
— The Moat Was Never the Audio