This paper is written for the people who have to stand behind an AI system. The board member who signs off on it, the risk officer who answers for it, the Chief AI Officer who owns it, and the engineer who keeps it running. It answers one question. You proved the system was fit to deploy on launch day. How do you know it is still fit today.
The Assurance Ladder, Assurance Debt, Proof of Fitness, the Standing Evidence Base, the Drift-to-Evidence Lag, and the Assurance Maturity Index are original constructs of this work, published under the license on the back cover. They rest on published research into how evaluation, contamination, and real-world drift behave [2][3][4]. Every quantitative claim is cited to a named source or labeled directional, without exception.
Every enterprise AI program I see can tell me how it tested a system before launch. Almost none can tell me how it knows the system is still safe to run this morning. The two questions sound alike, and they are not. The first is a photograph. The second is a pulse.
A model you deployed in the spring is not the model you are running now, even if no one touched it. The data it reads has moved, the behavior around it has shifted, and the version behind the interface may have been updated under you. A test it passed in the spring was a true statement about a system that no longer exists. The proof aged the moment the world did, and nobody filed the update.
This paper is my answer to the question I am now asked in every risk review. Not whether to test AI, everyone tests, but how to keep the proof alive so that the confidence on the page matches the system on the wire. The framework in these pages turns assurance from a launch gate into a standing function. The constructs are original work. The numbers are cited or labeled directional, without exception. I hold this to the standard of the rooms it will be read in.
Most enterprises treat proof of fitness as a gate you pass once, at launch. But the system keeps changing after launch, and the proof does not travel with it. The gap between what you proved and what you are running is the subject of this paper. Five findings follow.
A passed test is not a standing proof. A benchmark can be contaminated, so a high score can reflect exposure to the answers rather than skill [3][7]. And a single score is never fitness, because fitness is many things at once, accuracy, safety, cost, and behavior under stress [2]. The launch result is a start, not a settlement.
Confidence decays, and the decay is invisible. Real-world conditions drift, and models lose accuracy when they do [4]. Nothing on a dashboard turns red when a proof goes stale, so the belief on the page stays green while the system quietly moves out from under it. We name that widening gap the Assurance Debt.
Fitness is a standing claim, not a launch event. The fix is Proof of Fitness, evidence that is refreshed as fast as the system drifts and kept in a Standing Evidence Base that any decision can resolve to. Assurance becomes a function that runs, not a document that is filed.
The debt has a formula, and it is simple. Assurance Debt grows whenever drift outruns your evidence. When you refresh proof slower than the world changes, your confidence is already stale, and the lag between the two is the exposure. Exhibit 5 makes the mechanism explicit.
Continuous assurance is reachable in one quarter. The Assurance Ladder gives five rungs from a one-time launch gate to a system that produces its own evidence. A single program can climb from a stale launch gate to standing, monitored assurance in about one quarter with a small team.
Enterprises test AI before they ship it, and then the testing mostly stops. That would be fine if the thing being tested held still. It does not. A benchmark score, the usual proof of fitness, is fragile in two ways that matter to anyone signing off. First, the test may be contaminated. Public benchmarks leak into the data models train on, so a high score can measure exposure to the answers rather than the skill you meant to buy [3][7]. Second, a single score is not fitness. Fitness is accuracy and safety and cost and behavior under stress, all at once, and no one number carries all of it [2].
Even a clean, broad test only describes the system on the day it ran. Models lose accuracy when real-world conditions move away from the test conditions [4], and in a live deployment they always do. The proof does not travel with the system. It stays frozen at the launch value while the system drifts, and the distance between them is pure, unmeasured risk.
A launch test proves a system was fit on one day. Keeping it fit is a different job, and it never ends.
Adoption is climbing, so the number of systems running on one-time proof is climbing with it. Exhibit 2 shows what happens to each of them after launch day.
Assurance is not on or off. It is a ladder. The Assurance Ladder places any AI system on one of five rungs by how its fitness is known, from proven once at launch to proving itself continuously. The higher the rung, the smaller the Assurance Debt the system carries.
Assurance Debt. The growing gap between what you last proved about a system and what it is actually doing now. It accrues silently, because nothing signals that a proof has gone stale, which is what makes it dangerous.
Proof of Fitness. Standing, refreshed evidence that a system is still fit for its purpose across accuracy, safety, cost, and behavior, not a one-time score. A Proof of Fitness is only ever as current as its last refresh.
No rung is wrong for every system. A low-stakes tool can live comfortably on rung two. But a system that acts in the world, or that carries real consequence, cannot sit on the bottom rungs without building a debt that someone will eventually have to pay, usually at the worst possible moment.
Continuous assurance has four moving parts. Evidence sources that watch the live system, a Standing Evidence Base that holds the current proof and its age, a sign-off gate that decides whether to keep running, and a feedback path so that drift and incidents trigger a refresh. The loop is the product.
Standing Evidence Base. The living record every deploy or keep-running decision resolves to. It holds the current Proof of Fitness for each system and the date each piece of evidence was last refreshed, so a reviewer sees not only what was proven but how fresh the proof is.
| Evidence type | What it proves | Refresh cadence | Goes stale when |
|---|---|---|---|
| Live metrics | Behavior in production right now | Continuous | The metric stops tracking real outcomes |
| Scheduled re-tests | Accuracy on a fixed task | Monthly or quarterly | The task stops reflecting real use |
| Adversarial probes | Resistance to misuse and edge cases [5] | Each release and on cadence | New attack patterns appear |
| Owned benchmark | Fitness under your real constraints [6] | Each release | Contamination creeps in [3] |
Assurance Debt is not a vague worry. It behaves like a measurable quantity, and the rule that governs it is one line. Debt accrues at the rate of drift minus the rate of evidence refresh. When you refresh proof slower than the world changes, the debt grows. When you refresh at least as fast, it stays near zero. Everything else in this paper is a way of keeping the second condition true.
The lag between a change in the world and the moment your evidence catches up is where the exposure lives. Shorten the lag and the debt stays small, no matter how fast the world moves. Let the lag grow and a single stale proof can be carrying a system that no longer does what you signed off on.
Drift-to-Evidence Lag. The time between the world changing and your evidence catching up. During the lag the system runs on a proof that is already out of date. It is the single number continuous assurance exists to shrink.
Cost of standing still. A stale proof is a live liability. The system keeps acting on a sign-off that no longer describes it, and the first time anyone checks is usually after something has already gone wrong.
Cost of assurance. Continuous assurance is a standing function, not a second build. Most of it is instrumenting what you already run and scheduling what you already know how to do.
Payback logic. Assurance pays back as avoided incidents and as speed. A system you can prove is still fit is a system you can keep shipping, instead of stopping to re-argue whether it is safe.
You do not assure the whole estate at once. You start with what would hurt most if it failed, and each gate is a measurable exit, not a status meeting.
Can a wrong output be undone? An irreversible, high-consequence system cannot run on a proof you cannot see the age of.
How fast do the data, the model, and the world around it move? Fast drift demands fast refresh, or the debt opens quickly.
Does it act, or only advise? A system that acts carries its mistakes into the world, so it needs standing proof, not a launch note.
Must you be able to defend the system to an outside party? High scrutiny demands a standing evidence base anyone can read.
The closing argument. The assurance gap is a habit problem, not a tooling problem. Most enterprises prove fitness once and then trust the proof long after it has expired. The organizations that make assurance a standing function will deploy faster, because they can keep proving a system is safe instead of stopping to argue about it. The ladder is in these pages. The first rung above a one-time gate is one quarter away.
Answer honestly for your most consequential deployed AI system. Count the YES answers to find your Assurance Maturity Index and the rung it places you on. Designed to be completed in pen.
| No. | Probes | Question | Yes | No |
|---|---|---|---|---|
| 1 | PROOF | Can you state the date your most important AI system was last proven fit? | ||
| 2 | DRIFT | Do you watch for drift in the data and behavior of each deployed system? | ||
| 3 | METRIC | Do live metrics tell you when behavior drops, not just whether the system is up? | ||
| 4 | RETEST | Is every high-consequence system re-tested on a schedule, not just at launch? | ||
| 5 | PROBE | Have you adversarially probed each acting system in the last two quarters? | ||
| 6 | BASE | Is there one record holding the current proof of fitness and its age for each system? | ||
| 7 | GATE | Does a keep-running decision read current evidence, not the launch report? | ||
| 8 | OWNER | Does every deployed system have a named owner accountable for its assurance? | ||
| 9 | CADENCE | Do you refresh proof at least as fast as each system drifts? | ||
| 10 | ACTION | When evidence shows a system is no longer fit, can you pause or roll it back fast? |
This paper is a conceptual contribution. The Assurance Ladder, Assurance Debt, Proof of Fitness, the Standing Evidence Base, the Drift-to-Evidence Lag, and the Assurance Maturity Index are original to this work. They apply published findings about how evaluation and deployment actually behave, that one metric is not fitness [2], that benchmarks can be contaminated [3][7], that real-world drift degrades performance [4], that adversarial probing surfaces failures static tests miss [5], and that evaluation under real constraints is its own discipline [6], to the specific question of how an enterprise keeps a proof current.
Quantitative claims follow one discipline throughout, made visible in every exhibit. Solid fills carry cited data. Hatched fills and curves labeled directional exist to make the structure of an argument legible, not to report measurements. The debt formula is labeled a formally argued result, not an observed series. No statistic in this paper is attributed to a source that cannot be independently verified.
Arjun Jaggi writes and advises at the intersection of enterprise AI strategy, governance, and assurance. His published body of work, spanning original frameworks, concept papers, and books, is read by executive teams navigating AI adoption and is available in full, without a paywall, at arjunjaggi.com.