NL;NL Labs

NL;NL Labs — Independent instrument

An independent instrument for measuring how AI models actually behave over time.

A provider can quietly quantize, re-route, or downgrade a model while its public label stays the same. NL;NL measures the behaviour itself — on a frozen, pre-registered battery — and keeps the historical record of what changed, and when. A credit bureau for AI models.

Meter — the instrument Research — the papers Studio — the record, in public
01 / METER

The Model Change Ledger

The Meter is the always-on instrument. It probes pinned model versions on a fixed battery and publishes the public record of when a model's behaviour moved — on a label that never changed.

Day-zero state. No accuracy figure, drift finding, or measurement is claimed until the record exists — a rating that reports numbers it has not yet measured is worthless.

02 / METHOD

Why the record can be trusted

The mechanism is the credibility. Nothing here rests on us being believed — every step is fixed in advance and reproducible by anyone.

01

A frozen, pre-registered battery

The probe set is fixed before measurement begins, and its sha256 hash is committed to a public repository up front. It cannot be swapped, tuned, or cherry-picked after the fact — the commit history proves what was asked, and when.

02

Deterministic ground truth

Every probe is graded by local code against a known answer — never by another language model. No LLM judges an LLM. The scoring is a plain function you can re-run on your own machine and get the same result.

03

Provider-pinned measurement

We measure the specific version a provider actually serves through the API, not a friendly averaged label. What the endpoint returns is what we record — because the gap between the label and the behaviour is the whole point.

04

Published either way

Drift or a clean null, the outcome goes on the record. A rating agency that quietly buries the times it was wrong has no standing to be believed the times it is right.

Transparency is not a value statement here. It is the moat.

03 / RESEARCH

Research

NL;NL Research establishes the methodology that every rating must survive. The referee is built before the match.

Paper 0 · Methodology Pre-registered · result confirmed (narrow)

A token budget censors a measurement by the cost of the correct answer.

A reasoning model spends more tokens when the right answer is expensive to verify. Cap the tokens and you do not merely truncate the output — you silently drop exactly the answers that were costly to reach, biasing the result by ground truth. Statisticians already have a name for this: informative right-censoring (missing-not-at-random).

What Paper 0 pins down is that an experimenter's max_tokens cap, on a fixed model with unchanged weights, alone reproduces the exact signature that earlier work read as a model "getting worse." An inversion control — flipping which class is cheap to verify — makes the censoring follow, proving the mechanism is general, not a quirk of one task.

This is the methodology credential, not a drift finding. Before NL;NL publishes any change to a model, it must answer one question: is that the model moving, or your own measurement being censored? Paper 0 is that test. Report #1 — behavioural drift across model versions — follows once the Meter has accrued history.

Every published result carries its pre-registration commit and re-runnable code, so the finding can be checked, not taken on faith.

04 / CONTACT

Talk to the bureau

For downstream parties who need independent, citable reliability evidence for a real decision — routers, insurers and underwriters, procurement, marketplaces, and payment rails pricing model risk.

We do not sell monitoring to the operator of a model we rate. If that is the ask, we are the wrong instrument, by design.

Research & enquiries
research@nlnllabs.com

One instrument, one standard of evidence: a payment, a live integration, or a decision demonstrably changed by what NL;NL measured.