Independent reliability evidence

Independent reliability evidence

The public record
of AI model change.

AI companies change the models behind the names you build on. Sometimes they say so. Sometimes they do not. Either way we write it down.

If a model you depend on starts behaving differently, you usually find out from your users. We measure it ourselves, every day, so your users are not the first to tell you.

Never paid by anyone it rates.

THE METER

What we measure ourselves

A model's name can stay the same while the model behind it changes. So every day we send each model we watch the same fixed set of questions, with the same settings, and we write down what comes back. We never compare one company's model with another's. Each model is compared only with its own earlier days, because that is the only comparison that can show a model moved.

MODELS WE WATCH

Routes we watch

GatewayWatching since 23 July 2026

OpenRouter

A watch begins on the first day a model completes a full day of our checks. If we ever change our own settings, we close that run and start a new dated one. The earlier days stay published.

OPEN THE METER ↗

THE LEDGER

What changed

This record holds changes to deployed AI systems. Some entries use a company's own words, and some use measurements made by us. Every night we check for anything new.

ANTHROPIC

Sonnet 5 sessions now run 33,000 tokens longer before compacting

ANTHROPIC

Sign-in enforcement now exits when it cannot read managed settings

ANTHROPIC

Analytics now stay off from startup, not just after login

Every entry names where it came from: reported by the company, or measured by us.

117 ENTRIES · OLDEST 8 JANUARY 2026 · NEWEST 26 AUGUST 2026 · CHECKED NIGHTLY FROM 27 JULY 2026

OPEN THE LEDGER ↗

PROOF

The questions we ask are fixed. We publish each instrument's fingerprint before we use it and stamp it to Bitcoin.

See the fingerprints ↗HOW THIS WORKS ↗

Research

Paper 0. Our first study tested one way our own measurements could be wrong. Cap how much a model is allowed to write, and it looks worse without having changed at all.

Read Paper 0 ↗

Enquiries

Research and incident
enquiries

If you have found something, or you are making a decision that leans on this record, write to us.

NL;NL Labs is not paid by, and does not sell monitoring to, the operator of any model it rates.research@nlnllabs.com