Steering models

what it actually is

Not vibes. Reproducible control.

The mysticism around “prompt whispering” is mostly noise. The real thing is engineering: a model outputs a probability distribution over the next token, and you shape that distribution on purpose — with role and context that narrow it, negative rules (do not guess, do not apologize) that cut the filler and the confident invention, chain-of-thought that forces the reasoning into the open where a wrong step is visible, and a rigid output contract so a random text generator returns something a program can actually depend on.

The test is reproducibility. If the same steer doesn’t hold across runs and across inputs I didn’t hand-pick, I haven’t engineered anything — I’ve gotten lucky once and written it down. So the craft is really two jobs at once: make it do the exact thing, and prove it keeps doing it when I stop being nice to it.

the two halves

Make it do exact things. Bound what it can do.

Offense is squeezing deterministic, software-shaped output out of a fundamentally random predictor — the JSON that always parses, the format that never drifts, the answer that leads instead of hedging. Defense is the half most people skip: a guardrail around every action the model is allowed to take, so that being wrong is survivable. The correspondent layer is that defense in production — every action passes a wall I wrote, and one of those walls once refused an explicit order from me to bend it. That’s not the system failing; that’s the feature.

The same bet is the whole of CiteGuard: knowing how far to trust a steer — not very — is exactly why its rail sits outside the model.

the third half — steering agents, not just outputs

I don't hand work to a model and hope.

Everything above is about shaping what a model says. The larger job now is directing agents that build — that write the code, run the commands, and touch the repository. That is a different discipline, and it fails in a different place. The generator was never the bottleneck. Review is.

An agent will hand you two hundred plausible lines faster than you can read fifty. If you cannot read them, you have not automated the work; you have automated the appearance of the work and moved the risk somewhere you can't see it. So the whole practice is built around staying able to catch it being wrong.

Decompose so ownership is exclusive. Work is split into bounded tasks where each one owns the files it touches, and parallel waves meet at a barrier before anything merges. Two agents editing the same file is not parallelism, it's a race with extra steps.

Match the model to the consequence. Mechanical sweeps go to the cheap fast tier; the judgment calls and the final review go to the strongest one. The scarce resource is not tokens, it's attention — mine. And fan-out buys coverage, not speed: more agents to see more, not to finish sooner.

Make the verifier hostile. A reviewer told to check something will find it fine. A reviewer told to refute it goes looking for the case that breaks it. A claim that survives that is plausible — still not proven.

Seed controls, or the run means nothing. Every checking pass carries known-good and known-bad plants. If the battery comes back clean without flagging the plants, the correct conclusion is that the battery is broken — the run is void, not clean. A sweep that covers nothing reports exactly the same all-clear as a sweep that found nothing, and telling those two apart is most of the job.

Reproduce the real gate, don't guess at it. Before anything ships, the check gets run locally with the project's own toolchain and its own config — because a tool that passes in isolation and the same tool as the project actually configures it are different tools, and only one of them is the gate.

Draw the irreversible line and never let an agent cross it. Reversible work runs unattended. Publishing, credentials, anything destructive, anything that can't be taken back — those stop and wait for a human. That boundary isn't caution slowing the automation down; it's the thing that makes running the automation fast on the other side of it defensible.

A receipt, from this site, the day I wrote this section. The gate that checks the live site against the build had a hand-written list of pages — and it had quietly drifted to covering five of fourteen. The résumé page and every case study were shipping unchecked while the gate printed OK. Nothing alerted, because a gate that covers less than you think still passes. It now derives its list from the build instead of remembering one, it fails if it derives nothing, and five seeded controls prove it still goes red when something new appears. That's the loop: the tooling caught nothing, so the tooling was the bug.

where it shows up — receipts, not adjectives

Including the words you’re reading.

This isn’t a claim I get to make in the abstract, so here’s where it’s load-bearing on this very site. The adversarial battery behind the agent work exists because a green eval suite once flattered me while the live path was broken — steering isn’t done until the tests try to break the thing instead of confirming it. And the headline on the home page wasn’t typed; it was steered — seven drafts, twenty-eight candidates, four independent adversarial judges scoring each on honesty, voice, typography, and stickiness, one survivor. I point that process at my own writing for the same reason I point it at everything else: I don’t trust a first draft, mine included. The colophon shows the receipt.

What I won’t print is a benchmark score, an “accuracy” percentage, or a before/after lift — none of that is measured cleanly enough to survive this site’s own rules, so none of it appears. The craft is real; the numbers that would flatter it aren’t measured, so they stay off the page.

← back to work or hand me a model that won’t behave →