Paper 0· Methodology· Pre-registered· Result confirmed (narrow)
A token budget censors a measurement by the cost of the correct answer.
A reasoning model spends more tokens when the right answer is expensive to verify. Cap the tokens and you do not merely truncate the output. You silently drop exactly the answers that were costly to reach, biasing the result by ground truth on a model whose weights never moved.
A fixed output-token cap does not censor a reasoning model's answers at random. It censors them by the correct answer. Because a reasoning model spends tokens in proportion to how far it has to search to verify the true answer, a max_tokens cap truncates the expensive-to-reach class and lets the cheap one through. Statisticians already have a name for missingness that depends on the value that went missing: informative right-censoring, a form of missing-not-at-random. We adopt the name, we did not coin it. A balanced test set does not prevent it, because the bias re-enters after data collection, through truncation, invisibly. This is the standard NL;NL builds before it rates anything: a change on the wire is either the model moving, or your own instrument being censored, and you have to be able to tell which.
Reasoning-token spend tracks the cost of verifying the answer
Ask a reasoning model whether a number is prime. To answer composite it needs one witness: a single factor ends the search. To answer prime it has to eliminate every candidate factor. One answer is cheap to reach, the other is expensive, and the model's reasoning-token count reflects exactly that gap.
Now put a ceiling on the output. If the cap sits below the cost of the expensive class, every expensive-class answer comes back empty with finish_reason: length and a reasoning-token count pinned to the cap, while every cheap-class answer returns fine and correct. The surviving sample is answer-biased by construction. Nothing about the model changed. The measurement was censored by ground truth.
This is precisely informative right-censoring / missing-not-at-random: the probability that an observation goes missing depends on the value it would have had. It is an existing statistical category. What Paper 0 shows is that a benign-looking engineering knob — an output-token cap, present in every harness — puts a model measurement squarely inside it.
What a naive harness publishes instead
From the same bytes on disk, the two default harness behaviours both misreport a model that was actually 100% correct. Drop the empty responses, and it looks like the model only ever answers "no" and never detects a prime. Score the empties as wrong, and it looks like 50% accuracy, indistinguishable from chance. Everything needed to catch the artifact — finish_reason and reasoning_tokens — is returned by the API on every call and discarded by default.
The censoring, then the mechanism
Primality task, balanced four primes and four composites, temperature 0 and 1, three samples each. At an output cap of 300 tokens, four OpenAI reasoning endpoints censor the same way: every prime is truncated, every composite survives and is correct.
| endpoint | prime "yes" — empty | composite "no" — empty |
|---|---|---|
| gpt-5.6-sol | 24 / 24 | 0 / 24 |
| gpt-5.6-terra | 24 / 24 | 0 / 24 |
| gpt-5.6-luna | 24 / 24 | 0 / 24 |
| gpt-5.5 | 24 / 24 | 0 / 24 |
Every empty carries finish_reason: length and reasoning_tokens = 300, exactly the cap. Temperature is irrelevant (identical at 0.0 and 1.0). Every non-empty answer is "no", and every one is correct.
Lift the cap to 4000 on the same items and the reason becomes visible. On Sol the two classes separate perfectly with a gap: the cheapest prime costs more reasoning than the most expensive composite. All sixteen answers are correct. A cap placed anywhere inside that gap censors 100% of primes and 0% of composites. Ours was 300.
| class | min | max | correct |
|---|---|---|---|
| prime (expensive) | 352 | 516 | 8 / 8 |
| composite (cheap) | 0 | 34 | 8 / 8 |
min(prime) = 352 sits above max(composite) = 34. A cap anywhere in [35, 351] reproduces total censoring. Sol is 16/16 correct with nothing truncated.
Flip which class is cheap, and the censoring follows
The primality result alone cannot separate three explanations: that exhaustive search is expensive, that the token "yes" is expensive, or that primality specifically is expensive. A pre-registered second task settles it. Subset-sum — "do any two of these numbers add up to exactly K?" — inverts the asymmetry. Now "yes" is provable by a single witness pair and is cheap, while "no" requires exhausting every pair and is expensive. The expensive answer token flips from "yes" to "no". If cost tracks the answer token or the task, the pattern breaks. If cost tracks the search, it mirrors.
| task | expensive answer | cheap answer |
|---|---|---|
| primality | "yes" — exhaust all factors | "no" — one factor proves it |
| subset-sum | "no" — exhaust all pairs | "yes" — one pair proves it |
The answer token flips; the cost follows the search. Reasoning-token spend is set by the verification cost of the correct answer, not by the answer token and not by primality.
Eight items per class, twenty five-digit numbers each, every label computed and asserted in Python, never eyeballed. On Sol the class means separate cleanly — the "no" class costs 1.86× the "yes" class, a difference of 68 reasoning tokens — and the item means never overlap. Because the observed split is the unique maximum over all label allocations, the item-clustered one-sided permutation test gives an exact p of 1 / C(16,8) = 7.8×10-5, its design floor.
A 50,000-shuffle Monte Carlo returned 6×10-5, below the design's minimum achievable p and therefore noise. The exact combinatorial value is the one reported; a permutation p that beats its own floor is a bug, not a triumph.
At cap 300 nothing was truncated in this run (the expensive class topped out near 190 tokens). The mechanism reproduced; the censoring consequence only appears when the cap falls below the expensive class's cost. Censoring is a consequence, not the finding.
Why balancing the test set is not enough
The 2023 lesson from the demolition of the "ChatGPT is getting worse at primes" result (Narayanan & Kapoor, on Chen et al., arXiv:2307.09009) was: balance your test set, because a set of all primes lets a model that always guesses "prime" score 100%. Our set was balanced four and four. It did not help. The bias re-entered after collection, through truncation. The corrected rule: class-balancing is necessary and not sufficient — you must also show that truncation is uncorrelated with the answer class. For any task where one answer needs a single witness and the other needs exhaustive elimination ("is this code safe", "does a counterexample exist", "is this satisfiable"), it is structurally correlated, and no amount of balancing fixes it.
What is confirmed, and what is not
Stated in full, before anyone asks. The finding survives an adversarial red-team, but only in its narrow form, and the honest boundaries stay in view.
- Confirmed, narrowly. The public claim is not "verification cost sets token spend" as a law. It is the tested association: verification and search cost is a strong driver of token spend on the tasks and endpoints measured, strong enough that a fixed output cap can censor answers by ground truth. It is not the sole determinant.
- The strongest cheap falsifier was run, and the effect held. An independent red-team replaced the trivial composites (all divisible by 3) with hard five-digit semiprimes whose factors sit near their square root, and alternated the answer class on every request so class could not be confounded with request order. The hard composites became roughly 16.5× more expensive by class mean and overlapped the prime distribution. Had they stayed cheap, the mechanism would have been killed. They did the opposite. Cost: $0.31.
- Mostly one lab; the mechanism is isolated on one model. The censoring replicates across four OpenAI endpoints. The inversion control and the phrasing check ran on Sol alone. Cross-provider token scales are not directly comparable (one provider exposes a reasoning effort level, another a token budget), so this is a within-model claim, not a cross-lab law.
- Request order in the original suites was class-blocked, a design defect. The original runs sent one class then the other, confounding class with request time. The interleaved falsifier above addresses it; future suites randomize order by default, and subset-sum should be replicated in randomized order before external release.
- Partial prior art. This was partially anticipated by Chen et al. (arXiv:2307.09009), Fig 4. We did not discover the whole thing from scratch. Paper 0's contribution is naming the mechanism as informative right-censoring, isolating it with the inversion control, and pre-registering the falsification before the data.
- This is a methodology credential, not Report #1. Total spend was under $2. It establishes the standard, it is not a drift finding. Report #1 — behavioural drift across model versions — comes later, once the Meter has accrued its own history.
Pre-registered before the data existed
The prediction, the exact refutation conditions, and the standing constraints were written down and committed to git before the inversion-control suite was run. No suite data existed at that commit. The falsification was fixed in advance so it could not be reinterpreted once the numbers came in, and the result was to be reported whichever way it landed.
- Pre-registration commit
- 15aa98c
committed 10 Jul 2026, before any inversion-control data existed - Ground truth
- computed and asserted in local Python. No language model grades another.
- Re-runnable code
- to be published as a standalone repository, so the finding can be checked, not taken on faith
The re-runnable harness will be published as a standalone public repository. It is not linked here yet, because linking a repository that is not yet public would be exactly the kind of unverifiable claim this paper exists to catch.
The referee is built before the match. Paper 0 is that referee.