The Problem Belongs to the CTO
Every enterprise that would never ship a line of software without a test suite is shipping AI systems with zero behavioral specifications. The same organization that requires 80% code coverage in CI/CD has deployed an LLM-powered customer escalation system with no documented behavioral claims and no eval to verify them. When a model update arrives, nobody knows what changed because nobody ever specified what it was supposed to do.
This is not a quality assurance problem. It is a governance problem. You cannot audit a system whose behavior was never specified. You cannot demonstrate regulatory compliance through a behavior that was never tested. You cannot confidently update a model when you have no mechanism to detect whether the update changed anything. Eval-Driven Development is the practice that closes this gap: write the evaluations before you ship the system, not after something breaks.
The CTO and Chief AI Officer own this problem. The cost of not addressing it is not visible until a model update silently changes a behavior that was carrying a compliance control, a customer promise, or a risk decision, and nobody finds out until a regulator, a customer, or a competitor does.
Dark Behavior is any output, decision, or response pattern of a deployed AI system that was never formally specified, never tested with a runnable evaluation, and cannot be audited against a documented behavioral claim. Dark Behavior is not a bug: the system may be performing exactly as it was informally intended to. The risk is that it is performing without proof, without version history, and without any mechanism to detect when that behavior changes. In a high-risk AI deployment under In any deployment, it is an ungoverned business decision running at scale.
Eval Surface is the total set of behavioral claims about a deployed AI system that are backed by runnable, versioned evaluations. An organization's Eval Surface defines how much of its AI system's behavior it can actually prove, audit, and monitor. The complement of the Eval Surface is Dark Behavior. Most enterprise AI systems today have an Eval Surface that covers fewer than a third of the behavioral claims their stakeholders believe the system makes, based on structural inference from how AI systems are built and deployed without formal specification disciplines. Organizations that have achieved Eval-Driven Development operate with an Eval Surface that expands with every new system capability, ensuring behavioral claims and the tests that verify them grow together.
Why Evals Are Not a Quality Step: They Are the Specification
The practitioner community has been circling this insight for several years. The clearest version of it: if you cannot write an eval for a behavior, you do not actually know what that behavior is. A prompt that instructs a model to "respond professionally and helpfully" is not a behavioral specification. It is an aspiration. A behavioral specification says: for this input class, the system must always return a response that satisfies these measurable criteria, and here is a runnable test that verifies it.
This reframes the question from "how do we measure quality after deployment" to "how do we define what we're building before we build it." The CheckList framework from Ribeiro et al. [3] demonstrated in 2020 that behavioral testing of NLP models: testing capability types, not just aggregate accuracy, surfaces failure modes that benchmark metrics consistently miss. That insight applies with even greater force to enterprise AI systems, where the behavioral claims are not academic but operational: the system must not recommend competitor products, must escalate certain keywords to compliance queues, must refuse to answer out-of-scope questions. These are testable claims. They are almost never tested.
The behavioral contract between an AI system and its enterprise context (what the system will always do, what it will never do, and what it must do consistently under edge conditions) lives in the prompt. The same Prompt Debt dynamic documented in The Prompt Is Business Logic means that when the prompt is ungoverned, the behavioral contract is ungoverned. An eval suite is the executable form of that contract.