Shaurya Singh

← writing

Measured, or UNKNOWN

Agent systems don't fail on bad guesses; they fail when a guess is allowed to render as a fact. The fix is a third truth value the system is forbidden to collapse.

ongoing · doctrine

ongoing · doctrine — running in production across two estates as of 2026-09

I have spent months running two production agent estates — a personal one and a company one. Different stacks, different stakes, same postmortems. Every incident, traced back far enough, has the same shape: something true once got treated as still true. Not a hallucination, not a bad model — a stale fact, or a plain guess, wearing the clothes of a measurement.

The fix I keep landing on isn’t a better model. It’s a third value.

The doctrine, stated plainly

Boolean logic gives you true and false. Production agent systems need a third state — UNKNOWN — and the entire discipline is refusing to let anything collapse into true or false without having just been checked. Not “checked recently.” Not “checked by someone I trust.” Checked, by this session, right now, by an instrument that isn’t just re-reading a note that someone else wrote about a check they did earlier.

The rule, in one line: a fact that wasn’t just measured renders as UNKNOWN, or it doesn’t render at all.

That second clause matters more than it looks. It’s easy to say “flag uncertain things.” It’s a different thing to build a system where an unmeasured fact is structurally incapable of reaching the surface as a plain assertion — where the rendering layer refuses to print a status line unless a measurement fed it. That’s the difference between a style guide and a contract. I mean it as a contract: state doesn’t propagate past a session boundary as fact. It propagates as a claim needing re-verification, or it doesn’t propagate.

Every incident below — the DOWN note, the filler threshold, the /health lie, the retracted eval — is a variant of the same axis, and it isn’t accuracy. Each one was a correct statement at the moment it was made: the DOWN note was true in April, 2500ms felt reasonable when someone typed it, the 200 on /health isn’t lying about the process. None of these is the model saying something wrong. All of them are something true getting repeated past its expiration date, unchecked. That needs a different fix than better answers: a hard refusal to say anything, right or wrong, that wasn’t just measured. A model that’s 100% accurate on stale data is still wrong the moment the data goes stale, and it will never tell you, because accuracy was never the failure mode. Staleness was.

Where the doctrine came from

The clearest origin story is embarrassing, which is why I trust it. A personal voice-agent box of mine went down in April. Someone — a session, a note, doesn’t matter — wrote “[the agent] is DOWN, needs a physical restart.” That sentence was true when it was written. Then the box came back up. The note didn’t.

For weeks, every new session that opened that page inherited the note as ground truth and reported the service DOWN, without checking. Nobody was lying or hallucinating an outage. A true fact from the past was being repeated as a true fact in the present, with no checkpoint that said “re-verify this before you say it out loud.”

When someone finally did re-verify, on 2026-07-19, the service was green on every leg that mattered:

GET /health   -> 200
GET /portal   -> 302
server.py     -> running
ngrok (x2)    -> up

Four legs, four independent checks, all green. The DOWN note had been dead wrong for who knows how long, and it cost nothing to disprove — one round of actual checks. The expensive part wasn’t the outage; it was every session in between that skipped the checks and repeated the note.

That’s the incident the doctrine is named after: don’t assert a state, measure it, then report. Everything else in this piece is a variation on that one sentence.

Case one: the threshold nobody had measured

A voice-recruiting agent I run splits work across two models — a fast one that decides what to do, a slower one that writes the reply. The fast one occasionally needs to say something while the slow one is still thinking, so callers don’t sit in silence. That’s a filler: a “let me check” dropped in mid-turn.

The filler threshold — how long to wait before speaking a filler — had been set to 2500 milliseconds. Nobody remembers exactly why. It felt reasonable. Reasonable is not measured.

Eventually someone pulled the real call log — 173 actual turns, real latencies, no assumptions — and computed the median:

p50 real-call latency: 3760ms
threshold:              2500ms

The threshold was set below the median latency, not above it. That meant the filler wasn’t a rare rescue for slow turns — it was firing on roughly 95% of all turns. It had become a verbal tic: a system that says “let me check” almost every single time it opens its mouth, because someone guessed a number instead of measuring one.

The number “2500ms” had never been false in the sense of being disprovable by inspection. It just had never been measured against the thing it was supposed to bound. It sat there, unchallenged, doing damage every single call, because nobody had run the one query that would have caught it in an afternoon.

Case two: raising the bar and cutting the fat

Fixing the threshold meant treating the same 173-turn dataset as the source of truth instead of the intuition. Raise the threshold to 5500ms, above the measured p50, and the filler drops to firing on roughly 15% of turns — the genuinely slow tail, which is what it was built for. The keyword-based filler matching that had been layered on top of the old threshold got dropped too, once measurement showed it was a predictable pattern doing nothing the raw threshold couldn’t do on its own.

Same week, same agent, a second guess got caught the same way. The persona prompt told the model to be warm — “let it breathe,” “occasionally three sentences when it earns it.” Warm sounds harmless. Measured against real output, it wasn’t:

before:  p50 156 chars   max 492 chars
after:   p50 25 words    max 41 words

The fix wasn’t a better prompt. It was moving the constraint out of prose and into code: a hard two-sentence cap, no preamble, at most two names per turn, enforced structurally rather than requested politely. The lesson generalizes past this one agent: a persona instruction is a request; a measured cap enforced in code is a fact about the system. Only one of those two things can’t drift back to where it started the next time someone edits the prompt.

Health checks as the doctrine’s favorite liar

If there’s one mechanism that keeps re-teaching this lesson, it’s the health check. A green health check feels like a measurement. Often it’s checking the wrong thing.

One inference server in this estate answers its /health endpoint with a 200 while the model behind it is still loading into memory. The process is up. The port is listening. The health check is, technically, not lying about the process — it’s just answering a question nobody actually cares about. The question that matters is “can this thing do the one job I built it for,” and a 200 on /health doesn’t answer that. The fix: readiness has to be a real inference call, not a status code.

POST /completion  {"n_predict": 1}

If that returns real tokens, the service is ready. If it 200s on /health but you skip the real call, you’ll ship traffic to a server that’s still warming up and call the failure a fluke.

The same lie shows up at the config layer, in a worse form, because it’s delayed. On this same estate, a live service tested fully green — every route working, every dependent tool answering — right up until an overnight file copy silently overwrote its config with a stale template. The service kept running on the config it already had loaded in memory. It stayed green. The corruption sat there, armed, invisible, because nothing re-reads its own config while it’s running. The landmine only went off on the next restart, hours later, and by then whoever caused it had moved on and had no reason to suspect their own change.

The doctrine’s answer to both isn’t “check more often.” It’s “check the actual dependency, not a proxy for it,” and “a check that passed an hour ago is not evidence about now” — measured-or-UNKNOWN applied to time instead of state.

Extending the doctrine into what a model claims about itself

The doctrine doesn’t stop at infrastructure. It has to reach claims about capability, because that’s where the incentive to round up is strongest.

I run a pipeline meant to eventually train my own model off real usage: capture interactions, score them by reward, export the good ones as fine-tuning data, retrain. The pipeline itself is real, wired, and running. It is labeled LIVE — as a pipeline. It is explicitly never allowed to claim more than that; the doctrine for it is three words: “never claims a trained model.” The flywheel existing and running is a fact I can assert once I’ve watched it run. Whether a usable trained model exists on the other end is a separate fact, needing its own separate measurement — a real eval against a real checkpoint someone has actually loaded and queried.

I feel the pull to skip that measurement every time I write a status line about this pipeline — “flywheel is live” wants to slide into “we’re training our own model,” because the second sentence is more interesting and I built the first one myself. The only thing stopping the slide is the rule from everywhere else in this piece: check the artifact before you print the claim. Has anyone loaded a checkpoint and run a real eval against it, this session? If not, the honest label is “pipeline live, model unproven,” and the gap between those two sentences renders as UNKNOWN rather than collapsing either direction — one way is a lie, the other is false modesty, and both skip the check that would settle it.

Extending it into the eval itself

The sharpest version of this doctrine isn’t applying it to a claim — it’s applying it to the thing you use to check claims. On 2026-09-07 I retracted a capability result I’d published myself: a small model had scored 10% against a 2% control baseline on a held-out benchmark, margin comfortably over the bar I’d set for “demonstrated.”

The retraction happened because someone went back and checked whether the control could have scored anything other than 2%. It couldn’t. The control was structurally incapable of being right on nine of ten question categories — it could only be correct on the one category where the correct answer happened to be a fixed string it always output. So “2.0%” wasn’t a chance floor at all. It was a property of the answer key, dressed up as a property of chance. Once copied-exemplar answers were also correctly counted as wrong instead of right, the real score fell to 6 out of 100 — under the bar the harness itself had defined as meaningful.

I didn’t quietly edit the number down and move on. I posted the retraction, kept the original claim on the record marked retracted with the reason attached, and said plainly that no replacement capability bar exists yet. The rule behind that is the same rule as everything above: a control that cannot be right is not a floor, it’s decoration, and a retraction that gets buried instead of logged just moves the lie one layer down, into the audit trail instead of the result.

The mechanism underneath: two reads, one instrument

The last piece is subtler and it’s the one I get wrong most often by default. Verification feels satisfying when a second check agrees with the first. But agreement only counts as corroboration if the second check used a genuinely different method. If it used the same method, you don’t have two readings — you have one reading that got asked twice.

I watched this happen live: an agent reported a file transfer as “verified” because it had read a page’s prose description of the file’s hashes and repeated those hashes back. That’s not verification. That’s an echo. The system’s own numbers had come back around and been mistaken for someone else’s numbers agreeing with it — a phantom second source, one source wearing two hats.

The same failure showed up with time itself. A record system’s own “created” timestamp turned out to run about 70 minutes behind the timestamps printed in its own status posts, because of ordinary clock skew between two components. Every duration computed by subtracting one timestamp from the other was fiction — internally consistent, wrong. The fix generalizes past this one bug: cite a timestamp you actually measured on the artifact itself, or say the duration is UNKNOWN. Don’t derive a number from two clocks that were never checked against each other.

The rule that falls out of both incidents: a second read sharing a method with the first read is one instrument, not two. Real corroboration needs a structurally different way of finding out — a different tool, a different code path, a live call instead of a cached description of one.

What I don’t know yet

The doctrine has an obvious cost, and I haven’t found its ceiling. Every measurement is a real check that takes real time — a query against a log, a live health probe, a second instrument brought to bear instead of trusting the first one’s echo. An estate that insists on fresh measurement for everything eventually trades decision throughput for certainty. At some volume of decisions, “measure it” stops being a discipline and starts being a tax nobody can afford to pay on every single fact.

I don’t yet know where that line sits. I don’t know whether it’s a fixed budget (some fraction of total agent time spent re-verifying instead of acting) or whether it scales with the blast radius of being wrong — cheap facts get repeated, expensive facts always get re-measured, and the hard part is classifying which is which before you’re already wrong. I also don’t know whether the discipline degrades gracefully under pressure, or whether — like most safety properties — it holds fine until the one night everyone is tired and in a hurry, which is exactly the night it’s built for.

Until I have a real answer, the honest one is UNKNOWN.

— Shaurya Singh

← back to writing