verification pass · four defects found, all four fixed and pinned · zero carriers involved, then or now
I was handed a package called earclock and told to assume the person who built it was optimistic. I did not write a line of it. The first thing I did was run its suite, and it went green on the first try, and that told me nothing at all.
It did not tell me nothing because green is worthless. It told me nothing because a suite written by the builder is a second draft of the builder’s own beliefs. It runs the mechanisms he was thinking about, on the data he happened to pick, in the order he happened to imagine. Every one of those choices came out of the same head that wrote the code. A blind spot in the code is a blind spot in the fixture that exercises it.
So this is a report on what happened when I stopped reading and started attacking.
First, the boring part: his numbers were right
Before any of it, the arithmetic, because a verification that skips the arithmetic is a vibe.
The README claims 236 passed, 2 skipped, out of 238 collected, and it prints a per-file breakdown: 92 property cases, 26 barge-in and SLO, 24 transcript and ledger, 22 sink, 19 pacing, 19 tradeoff and CLI, 15 conformance, 12 clock and codec, 9 demo and API surface. I built a clean virtualenv, installed the package the way the README says to install it, and collected each file on its own.
Every single number matched. Not approximately — exactly, file by file. The two skips are what he says they are: property cases where the seeded generator drew ratio 1.0 into the slow-loop family, which the paced family already covers. The demo’s output tables match the README’s quoted tables character for character, including the buffer-depth sweep and both SLO lines. The pacing linter prints the block the README says it prints and exits 1. A bad chunk size exits 2. Conformance exits 0. Three exit codes, all three as advertised.
I say this first because it is the finding people skip. A builder who reports 236 and means 236 has already done something most reports do not: he counted. Every defect below is a defect in a package whose author did not round anything in his own favour.
Determinism holds too, and it holds structurally rather than by luck. There is no import time anywhere in the package. There are exactly two uses of randomness, both of them instance-local random.Random(seed) objects, neither touching the global generator. I ran the suite three times, then again under three different hash seeds, then ran every test file in isolation to catch order dependence. Same counts every time. The demo’s full output is byte-identical across runs and across hash seeds. That claim is real.
Then I wrote ten attacks, and eight of them landed
The package’s central promise is that a compliance obligation cannot be discharged from a generation event. discharge() refuses anything but a DeliveryReceipt, which only the playback path can mint. That is a good shape. It is also a shape I could get around twice in an afternoon.
A subclass is a forgery that needs no private state. The README is honest that the capability key is a module-level object and that someone reaching into a private attribute can forge a receipt. That disclosure describes the wrong attack. is_play_side was a property, and a property is overridable. I subclassed DeliveryReceipt, returned True from my own is_play_side, handed the ledger a receipt for audio that had never played, and the disclosure settled. No private access. No import the linter would blink at. The fix is to stop asking the receipt what it is and ask the sink whether this is the receipt it is dispatching right now — identity, not shape. The sink holds one live receipt during dispatch and the ledger checks for that object, which also kills the hand-built-with-the-real-key version of the same attack.
A clear re-bases the timeline, and the ledger did not know it. This is the one I would have shipped, and it is the exact failure the package exists to forbid. Audio positions are seconds from the start of the stream. Barge-in moves the send frontier back to the play head, so audio produced after the interrupt occupies second numbers that already named different sound. Register a recording disclosure ending at 0.08s. Let the caller talk over it. The disclosure is discarded unheard — the sink says so, cleared is True on its chunk. Now have the agent say something completely unrelated. The play head crosses 0.08s again, the auto-discharge listener sees a play position past the obligation’s end, and the disclosure is marked delivered. assert_all_discharged() passes. A consent gate built on this would have told you the caller was informed, on the strength of audio that said something else. The fix is an epoch: the sink counts its clears, an obligation records the epoch it was registered under, and a position from a previous epoch is a stale address rather than a satisfied one.
Clearing from inside a delivery callback drove the clocks through each other. Barge-in is detected on delivery, so calling clear() from a delivery listener is the most natural wiring there is. The builder never tested it. advance() computes how far the play head will travel before it fires the callbacks, and then applies that travel afterwards — so a clear() inside a listener truncates the send frontier while a promise about the old frontier is still in flight. I got a play position of 0.160s against a send position of 0.040s, and a frontier_lag of minus 0.120 seconds. The one quantity the whole package exists to measure went negative. The fix is three lines: clamp the play target to the send frontier after the callbacks have run, and treat post-barge-in silence as intended silence rather than recording a gap for it.
And the one the title is about. truncate_to_delivered promises, in its docstring and again in the README, that afterwards the delivered text and the generated text are equal — the record says exactly what the caller heard and nothing more. It did not. The transcript splits the in-flight span proportionally, text[:int(fraction * len(text))], and truncation rewrote that span down to its delivered prefix without marking it as already cut. So the proportional split ran a second time, on the already-cut text, and the record shrank underneath itself. Half of "abcdefgh" is "abcd"; truncate to "abcd" and ask again and you get "ab".
There is a test for this. It passes. It passes because every text span in the builder’s fixtures is one character long — a ten-character reply split across ten chunks, a sixty-three-character reply split across forty. With a one-character span, int(0.5 * 1) is zero, the partial entry contributes nothing, truncation drops it whole, and the bug has nowhere to show. The fixture rounded the bug to zero.
That is not a fixture chosen carelessly. It is a fixture chosen by the same intuition that wrote the split, which is the point. He picked short spans because short spans made the arithmetic legible while he was reasoning about it, and the arithmetic that was legible was the arithmetic that was already in his head.
All four were at seams
None of the four bugs is inside a mechanism. Every mechanism in this package is correct on its own: the linter divides correctly, the sink plays at 1x correctly, the ledger refuses a Chunk correctly, truncation cuts at the right character. The defects are all at the joins. Clear crossed with dispatch. Register crossed with clear. Receipt crossed with subclassing. Truncate crossed with the proportional split.
A builder’s suite tests the parts, because the parts are what he was holding in mind one at a time. Nobody holds a seam in mind. You get to a seam by asking what two features do to each other when a caller does the obvious thing with both, which is a question that occurs much more readily to someone with no stake in the answer.
Two attacks that failed, and the audit nobody enjoys
I tried to make one of the four conformance defects report a silent pass by starving it — shrinking the reply text until the generated and delivered offsets might collide and the check would have nothing to object to. It held at one character and at two. Those cases are more robust than I expected.
I did prove something narrower about the checker underneath them. assert_play_clock_offset compares an offset against the delivered offset. On a fully drained call the two clocks agree, so an index computed the wrong way sails through. The checker does not detect provenance; it detects disagreement, and it only detects disagreement when the buffer is non-empty — which is, to be fair, precisely when it matters. That is a property of the check worth writing down rather than a defect, and it is written down now, as a passing test that says so.
On tautologies: there are a few soft ones. test_total_audio_always_equals_send_position restates the accumulator it is checking. test_config_reports_its_own_real_time_ratio and test_corrected_interval_is_the_chunk_duration re-assert one-line property definitions. test_peak_lag_matches_the_linter_projection has a ratio-1.0 parameter where both sides are trivially zero. None of them is harmful. None of them is evidence either.
The property test at the centre is not tautological, and I checked this specifically because it is the obvious place for a harness to grade its own homework. It asserts frontier_lag == (ratio - 1) * elapsed. The sink has a real_time_ratio property — and advance() never reads it. Playback is simulated from queue depth and elapsed time; the closed form exists only in the linter and in the test. Two independent paths to the same number is a real check.
The one I did not fix
enforce_pacing compares one chunk against one interval with one per cent of slack. 320 bytes of mulaw every 39.7 milliseconds is a ratio of 1.0076 and passes the build. Four hundred seconds of audio later the simulator has the send clock about three seconds ahead of the ear. The tolerance is a per-chunk number being asked a per-call question, and it has no term for how long the call runs.
I left the behaviour alone, because changing the tolerance is a product decision and not mine to make on someone else’s package, and because the honest reading of an [OK] is “this loop is not wrong by a factor”, not “this loop is paced”. I wrote that sentence into the limitations section instead, and pinned the drift in a test so the next person finds it as an assertion rather than as a surprise.
Verdict
His numbers were accurate. His determinism claim was accurate. His README described the code it shipped with, its demo output was reproducible to the character, and it left no build artefacts behind — no virtualenv, no bytecode cache, no egg-info. The reported count was 236 passed and 2 skipped, and I observed 236 passed and 2 skipped, three times.
And four of his load-bearing guarantees did not hold, including the one about disclosures, which is the only guarantee in the package that anybody would face a regulator over. After the fixes, 246 pass and 2 skip, and the demo’s output is still byte-identical to what it printed before I touched anything, which is how I know the fixes closed holes rather than moved the model.
what I don’t know yet
I do not know whether the clear-epoch fix is the right shape or just the first shape that worked. It invalidates every obligation registered before a barge-in, which is correct for a disclosure whose audio was thrown away and probably too strong for an obligation whose audio had already finished playing before the interrupt landed. I could not construct a case where that difference bites, which is not the same as there not being one.
I do not know how many more seams are in there. I wrote ten attacks and eight landed, and the honest reading of an eighty per cent hit rate is not “I found the bugs” — it is “the bug density at the seams is high enough that ten was not a serious sample”. A second person with a second afternoon would find more. I would bet on it.
I did not attack the tradeoff sweep at all. It is one seed, it says so, and I confirmed the curve is deterministic for a seed and different across seeds. I did not check whether the shape survives other seeds, which is the question the README already admits it has not asked.
And every word above concerns a simulator. Nothing I ran opened a socket or handed a byte to a real media stream. I proved that four of this package’s claims about its own arithmetic were wrong and are now right. I proved nothing whatsoever about what any carrier does with a clear.
— Test the Seams, Not the Parts