Trust, then Verify

Treat every diff as a conjecture: a claim that this code satisfies some intent, offered without proof. That framing is not a metaphor borrowed for color — it changes what you do. If a diff is a conjecture, your job is not to admire it, agree with it, or feel good about it. Your job is to try to break it, and to build machinery that keeps trying after you have stopped looking. Confidence in the code should be proportional to the strength of the refutation attempts it survived, and nothing else.

Human code came with a free signal you have been reading for years without noticing. Sloppiness was evidence of inattention, and inattention was evidence of defects, so "this reads carefully" was weak but real evidence of correctness. Generated code destroys that inference. The model's objective is to produce text that looks like correct code for this context; producing text that is correct is a frequent side effect, not the target. So plausibility saturates — every diff reads like the work of a careful author — while correctness stays a coin flip weighted by how well-represented your exact problem was in training data. Plausibility and correctness come apart, and every heuristic you had was built on their correlation.

The practical consequence is uncomfortable: your sense that a diff is fine carries almost no information, and it feels exactly as strong as it did when it was informative. That feeling is the thing to distrust. Everything in this guide is machinery for replacing it with evidence you can name — a criterion checked, a property that held, a gate that passed, a mutant that died.

A hallucinated API is an invented method, parameter, endpoint, or configuration key: something that fits the shape of the library's design so well that the model produced it by interpolation rather than recall. These are the most-discussed AI failure and the least dangerous variety — when they are loud. A call to a method that does not exist raises AttributeError the first time the line runs, which is to say the first time anyone touches the feature. It dies at import or in the first smoke test, and nobody's money moves.

The dangerous variety is silent. Consider what happens when the model decides that the payment-processor client takes a timeout:

processor = ProcessorClient(
    api_key=os.environ["PROCESSOR_KEY"],
    timeout_seconds=5,          # plausible. also: not a real parameter.
)

# the client's actual signature:
class ProcessorClient:
    def __init__(self, api_key, **options):
        self.api_key = api_key
        self.timeout = options.get("timeout", 30)   # note the key it looks for

Nothing raises. **options absorbs timeout_seconds, the client uses its 30-second default, and the code reads as if a five-second timeout is enforced. Six weeks later the processor gets slow, your requests hang for thirty seconds each, the Gunicorn workers saturate, and the support console goes down during a billing incident — the exact moment agents need it. The reviewer who read that line saw a timeout being set. It was, in a real sense, there: it just was not doing anything.

The generalization worth carrying: absence of an error is not presence of a behavior. An exception is a gift, because it is the dependency refusing to pretend. The failures that hurt are the ones where the dependency shrugs, and those are found by checking the interface — reading the signature, the type stub, or the vendor's documentation — not by checking that the program still starts.

A subtler cousin is the interface that exists but does not mean what the model assumed. processor.refund(charge_id, amount) is a real method in our running system; whether amount is dollars or cents, and whether omitting it refunds the full charge, are facts the model guessed. That guess is unfalsifiable by reading the calling code, and it is off by a factor of one hundred if it is wrong.

The second signature is the one the whole guide is really about: logic that is correct-shaped and incorrect. Inverted conditions, boundaries off by one comparison operator, defaults that are reasonable in general and wrong here, an early return that skips a step the author would not have skipped. These are not sloppy — they are the outputs of a process that pattern-matched a common shape onto your specific problem and got the shape right.

Here is the version that runs through the rest of this course. Our system is Ledgerline, a subscription-billing support console: a React front end for support agents, a Flask support-api service in a container on AWS, Postgres behind it, deployed with CDK. Support agents issue refunds. Tier-1 agents have a refund cap of $50.00; anything above that needs a supervisor. The first generated handler contains no cap check at all — that omission is the subject of the next section — so this is the follow-up patch, produced when the model was asked to add one:

REFUND_CAP_CENTS = {"tier1": 5000, "tier2": 25000}

cap = REFUND_CAP_CENTS.get(session["role"], 5000)
if amount > cap:
    return jsonify({"error": "refund_exceeds_cap"}), 403

Read it the way you would read a contract clause rather than a line of code, and three separate questions appear, none of which the code answers. First: is the cap inclusive? amount > cap permits exactly $50.00; amount >= cap forbids it. Both are defensible policies; only one is yours, and the code does not tell you which was intended — only which was generated. Second: the .get(..., 5000) fallback silently assigns tier-1 privileges to any role the dictionary does not know, including a future "contractor" role that should have a cap of zero. Failing open on an unknown role is a permission decision made by a default argument. Third: amount is the requested refund, but the policy a lawyer would write says an agent may not refund more than $50.00 on a charge — a cap on the running total, not on any single request. Three requests of $30.00 against charge ch_7c2m pass this check individually and refund $90.00 in aggregate.

Notice the common structure. In each case the model produced the modal implementation — the one that appears most often in code that looks like this — and the modal implementation encodes assumptions that were never yours to begin with. The defect is not in any line. It is in the gap between the specification you did not write and the distribution the model fell back to.

The third signature has a mechanical explanation that is worth stating plainly, because it predicts what you will see: a model reproduces the median public example, and the median public example is insecure. Tutorials optimize for the shortest path to a working demo. Stack Overflow answers optimize for answering the question asked, not the question unasked. Neither audience is a production system handling other people's money. So generated code arrives carrying the security posture of a blog post: string-built SQL, CORS(app) with no origin list, secrets read from a literal because the example did, verify=False because the tutorial's certificate was self-signed, and — most consequentially — authentication mistaken for authorization.

Two of these are planted in the Ledgerline refund endpoint and will be caught by two different layers later in this course. Preview them now:

cur.execute(f"SELECT amount, customer_id FROM charges WHERE id = '{charge_id}'")

That is a request-controlled value interpolated into SQL. It is the most mechanically detectable defect in the entire codebase — a fixed syntactic pattern, matched by static analysis in milliseconds — which is exactly why module 5 assigns it to a CI gate and forbids you from spending review attention on it.

session = verify_session(request.headers.get("Authorization"))
if not session:
    return jsonify({"error": "unauthenticated"}), 401
# ... and then straight to the refund. no role check. no tenant check.

That one no tool will ever catch. The code is locally impeccable: a session is verified, an unauthenticated request is rejected with the correct status code. It is missing the question "is this agent permitted to refund this charge for this amount" — and a missing question has no syntax. This is the single most common serious defect in generated backend code, and it is the reason human review keeps its seat at the table.

The last two signatures are one failure in two costumes: the form of rigor without its content. The model has learned what careful engineering looks like — error handling exists, tests exist — and produces the visual signature of both without the substance of either.

Phantom edge-case handling is the block that appears to handle a failure and actually deletes it:

try:
    result = processor.refund(charge_id, amount)
except Exception as e:
    logger.warning("refund failed: %s", e)
    result = {"status": "pending"}

Read what this does rather than what it looks like. The refund did not happen. The caller is told pending, which is a lie with a plausible shape. The support agent tells the customer it is processing. The warning lands in a log stream nobody alerts on. In three days a chargeback arrives and the audit trail says the system believed a refund was in flight. The except block did not handle the edge case; it converted a loud failure into a quiet wrong answer, which is a strict downgrade. A reviewer scanning for robustness sees error handling and moves on — the block is defensive-looking, and that appearance is the whole problem.

A test-shaped test is the same trick applied to verification:

def test_refund_succeeds(monkeypatch):
    monkeypatch.setattr(processor, "refund",
                        lambda cid, amt: {"status": "succeeded"})
    resp = client.post("/refunds", json={"charge_id": "ch_7c2m", "amount": 8400})
    assert resp.json["status"] == "succeeded"

This test cannot fail for any reason related to your refund logic. It stubs the processor to return success, then asserts that success came back — it verifies that the stub works and that the handler can copy a string. Delete the cap check, delete the authorization check, break the retry loop: still green. Yet it adds a line to the coverage report, it appears in the PR as diligence, and it is indistinguishable at a glance from a test that means something. The tell is structural and quick to check: does the assertion follow from anything other than the setup? If the value asserted was placed there by the test itself, the test asserts nothing.

Both signatures matter beyond their local damage because they defeat the layers above them. Phantom handling makes runtime monitors blind — errors never reach them. Test-shaped tests make the test layer report health it does not have. A failure that disables your detection is worse than one that merely breaks something.

Start with the resource, because every review methodology that ignores it fails the same way. Careful reading — the kind where you hold the system's invariants in your head and ask whether this change violates one — is expensive, degrades fast, and is not fungible with skimming. Call it ninety minutes a day of real capacity, on a good day, with interruptions.

Against that budget you have a queue that no longer scales with human typing speed. Diff size has decoupled from authorship effort: a model produces four hundred lines in the time it takes to write a sentence describing them. The traditional implicit allocator — read the whole diff, spend time roughly proportional to its length — now hands your best attention to whatever happens to be longest. Longest usually means most mechanical: a generated test file, a component extraction, a migration touching forty call sites.

Meanwhile the nine-line diff that changes which role may issue a refund gets the tail end of your attention, at 5:40 p.m., after four hundred lines of React. That is the allocation you must invert, and inverting it requires an explicit rule, because the default is not a decision anyone made — it is what happens when you read diffs in the order they arrive.

The protocol is an ordered hunt, and the word hunt is doing work. You are not reading the diff from top to bottom looking for problems. You open the diff and search it five times, in this order, for five specific things:

1. Authorization and authentication semantics. Who is allowed to do this, and does the code ask? Not "is there a decorator" — does the check compare the acting principal against the specific resource and the specific action? This step is first because the failure is invisible (nothing is locally wrong), unbounded (an authorization gap is exploitable by anyone who finds it), and undetectable by every other layer in the stack.

2. Money movement and irreversible effects. Charges, refunds, payouts, emails to customers, external notifications, anything with a real-world counterpart that cannot be rolled back by reverting the deploy. Second because irreversibility means no later layer gets a second attempt.

3. Data mutation and deletion. Writes, migrations, cascades, bulk updates, anything with a DELETE or a schema change. Third because damage is bounded by backups — real, but recoverable at a cost.

4. External calls and their failure behavior. Every network boundary, and specifically what happens when it times out, returns an error, or succeeds slowly. This is where retries, idempotency, and partial failure live, and it is where generated code is weakest, because correct failure handling is underrepresented in the tutorials the model learned from.

5. Everything else. Timeboxed. A skim, honestly labeled as a skim.

The order tracks irreversibility × reach, and it is a priority queue rather than a checklist: if you run out of attention at step 3, you have spent it on the three zones that most deserved it, and you can say so in the review. A checklist that you abandon halfway is a failure; a priority queue that you abandon halfway did its job.

Here is the artifact, as generated. Thirty-four lines, idiomatic Flask, consistent naming, a retry loop with exponential backoff, correct HTTP status codes throughout. It passed review at a company you have heard of.

@app.route("/refunds", methods=["POST"])
def create_refund():
    session = verify_session(request.headers.get("Authorization"))
    if not session:
        return jsonify({"error": "unauthenticated"}), 401

    body = request.get_json()
    charge_id = body["charge_id"]
    amount = body["amount"]                     # cents

    cur = db.cursor()
    cur.execute(f"SELECT amount, customer_id FROM charges WHERE id = '{charge_id}'")
    row = cur.fetchone()
    if row is None:
        return jsonify({"error": "charge_not_found"}), 404

    for attempt in range(3):
        try:
            result = processor.refund(charge_id, amount)
            break
        except ProcessorTimeout:
            if attempt == 2:
                return jsonify({"error": "processor_unavailable"}), 502
            time.sleep(2 ** attempt)

    refund_id = new_id("rf")
    cur.execute(
        "INSERT INTO refunds (id, charge_id, amount, agent_id) "
        "VALUES (%s, %s, %s, %s)",
        (refund_id, charge_id, amount, session["agent_id"]),
    )
    db.commit()
    return jsonify({"refund_id": refund_id, "status": result["status"]}), 201

Step 1 — authorization. Search the diff for the authorization decision. There is a session check on line 3: verify_session returns a session or nothing, and nothing produces a 401. That is authentication — it establishes who is calling. Now ask the authorization question: where is the comparison between this principal and this action on this resource? Read the handler again looking only for that. It is not there.

The consequences, stated concretely with our cast: agent agt_d4k1 holds role tier1, whose policy cap is $50.00. She can refund the full $84.00 on charge ch_7c2m — the role is read from the session for the audit insert on line 30 and never consulted for a decision. Worse, no line ties charge_id to the agent's tenant: the handler will happily refund a charge belonging to a completely different customer of a completely different Ledgerline client, if the agent guesses or enumerates an ID. Call this D1.

Step 2 — money and irreversibility. processor.refund(charge_id, amount) moves real money out and cannot be reverted by rolling back the deploy. Read the loop around it. Three attempts, backoff, correct-looking. Now ask the step-4 question early, because it is the same call: what does ProcessorTimeout mean? It means the request did not come back. It does not mean the processor did not act. If the refund succeeded and the response was lost, attempt two issues a second real refund — the customer receives $168.00 against an $84.00 charge, and the refunds table records one row, so the discrepancy surfaces at reconciliation, days later, as a number nobody can explain. There is no idempotency key anywhere in the call. Call this D2.

Step 3 — data mutation. One bounded insert with bound parameters. Note in passing that the insert happens after the money moves and outside a transaction with it, so a crash between them loses the audit record for a completed refund — worth a comment, and a smaller radius than D1 or D2.

Step 4 — external calls. Already covered by D2. Also note the read on line 12 while you are here: f"...WHERE id = '{charge_id}'" interpolates a request-controlled string into SQL. Call this D3.

Run that review again and sort the findings by who could have made them. D3 was found by pattern recognition: a fixed syntactic shape, no context required, catchable by a regular expression a decade old. D2 required knowing that timeouts are ambiguous — general engineering knowledge, and mechanizable as a property once someone writes the invariant down. D1 required knowing that Ledgerline has tiered refund caps, that tier1 means $50.00, that the console is multi-tenant, and that the business considers an agent refunding another client's charge a serious incident. None of that is in the repository.

That difference is your entire comparative advantage, and it is worth stating as a rule: you are the only layer that carries intent. Humans catch divergence between what the system does and what the business meant, authorization semantics, business invariants, and the question of whether this feature should exist in this shape at all. Machines catch patterns, types, known-bad constructs, regressions, and — once you write the invariant down — violations of it across an input space you could never enumerate by hand.

The operational consequence is sharper than it first sounds. Every review finding that a machine could have made is a process bug, not an achievement. When you catch string-built SQL by eye, the correct emotional response is mild alarm: the pipeline should have caught that, your attention was the backstop, and backstops made of human attention fail silently on the Friday you are tired. Module 5 is the discipline of converting these findings into gates, and the review is where the gate backlog is generated.

This also settles what a good review looks like when you are pressed for time. A review that says "I ran steps 1 and 2; D1 blocks this; steps 3–5 unread" is more useful to the author and more honest to the record than a full-length review that visited every line and found the nested ternary. The first is a position. The second is coverage without detection — Figure 1.2's failure mode, performed by a person.

The mechanism is worth stating precisely, because the popular version — "garbage in, garbage out" — is both true and useless. Here is what actually happens.

A prompt is a constraint set. "Add an endpoint so support agents can refund a charge" constrains perhaps a dozen decisions and leaves fifty open: the boundary of the refund cap, what happens on an unknown role, whether refunds are per-request or per-charge, what a timeout means, how the query is built, whether an audit record is required, whether it is transactional with the money movement, what the retention period is. The model must resolve all fifty to emit code. It resolves them by sampling the training distribution — that is, by choosing what code like this usually does.

Now the diff arrives, and consider the reviewer's epistemic position. She sees amount > cap. Was $50.00-inclusive a decision the author made and she should defer to, or an accident of sampling she should challenge? She cannot tell from the code, because a decision and an accident produce byte-identical output. There is no comment saying "chose exclusive after asking legal," and there would not be even if someone had. So she has three options: approve on faith, ask the author (who does not know either, having not made the decision), or re-derive the intended behavior from first principles — for all fifty open questions, in every diff, forever.

This is why the specification layer is not a productivity nicety. It is the artifact that makes review possible: a diff plus a spec can be evaluated against something, and a diff alone can only be evaluated against the reviewer's memory of a conversation. It also explains a phenomenon you have probably felt without naming — that reviewing generated code is more tiring than reviewing human code of the same size. You are not just checking the work; you are reconstructing the requirements it was supposedly built against.

An acceptance criterion is a falsifiable statement of done: concrete inputs, an observable outcome, and no adjectives doing load-bearing work. The operational test is a person, not a rule — could a competent stranger with no access to me turn this into a test? If she would have to ask a question, the criterion is not finished.

Compare. "The endpoint should handle refund limits appropriately" — appropriately is a placeholder for a decision nobody made, and it will be resolved by sampling. "Refunds respect agent role caps" is better and still fails the stranger test: which roles, what caps, inclusive or exclusive, capped on what quantity. Here is the same requirement as criteria, drafted for Ledgerline:

AC-1  A request from an agent with role tier1 to refund an amount such that
      (already refunded on the charge + requested amount) <= 5000 cents
      succeeds and returns 201.

AC-2  The same request where the sum would be 5001 cents or more is rejected
      with 403 and error code refund_exceeds_cap. No processor call is made.

AC-3  Exactly 5000 cents in total on a charge is PERMITTED for tier1.
      (The cap is inclusive. Decided with Billing Ops on 2026-03-04.)

AC-4  A request whose role is not present in the role-cap table is rejected
      with 403. Unknown roles fail closed.

AC-5  A request for a charge whose tenant_id differs from the session's
      tenant_id is rejected with 404 (not 403 — existence is not disclosed
      across tenants). No processor call is made.

Five things happened there that are worth naming. AC-1 states the quantity the cap applies to — the running total on the charge — which was the ambiguity that let three $30.00 refunds through in module 1. AC-3 pins the boundary as its own criterion and records who decided, so it reads as a decision rather than an accident. AC-4 converts the fail-open default into an explicit fail-closed requirement. AC-5 specifies the status code and gives the reason, because "404 not 403" looks like a bug to anyone who does not know it is a cross-tenant disclosure choice. And every one of them says what happens to the side effect — "no processor call is made" — which is the part a test can actually assert.

Acceptance criteria describe what the system does. Invariants describe what it must never do, across all inputs, all sequences, and all implementations. That difference is not stylistic — it changes what the statement can catch.

A criterion is a point: this input produces this output. It is refuted by one failing example, and it says nothing about the inputs you did not enumerate. An invariant is a universally quantified claim, so it is refuted by any counterexample anywhere in the input space, including sequences no one thought to write down. D2 lives precisely there: no reasonable person writes an example test for "two refund requests where the first times out after the processor already succeeded," because that is not a feature, it is an interleaving. But it is trivially inside the region an invariant covers.

Ledgerline's invariants, stated in the spec, above the criteria:

INV-1  The total amount refunded on a charge, counted at the processor,
       never exceeds the original charge amount.
       (ch_7c2m: $84.00 in, never more than $84.00 out — under any
        sequence of requests, retries, timeouts, or concurrent agents.)

INV-2  No refund is issued without a recorded authorization decision naming
       the acting agent, the charge, and the amount. The audit row and the
       money movement are committed together or not at all.

INV-3  No SQL statement is constructed by string interpolation of any value
       derived from a request.

INV-4  No agent observes or affects a charge outside their tenant.

Note the phrase counted at the processor in INV-1. That is the difference between an invariant that catches D2 and one that does not: counted in our own refunds table, the double refund is invisible, because the handler only wrote one row. The invariant has to be stated against the place where the real-world effect lands. This is the kind of precision that survives being handed to a property test in module 4 — and INV-1 becomes that test almost verbatim.

Two loops are available when generated code is wrong, and they differ in whether information accumulates.

In the first, you read the diff, dislike something, and re-prompt: "that's not right, the cap should apply per charge." The model regenerates. Something else is now wrong. You re-prompt again. On the fourth attempt the tests pass and you ship. This is prompt-and-pray, and its defining feature is that the three failures taught you three real things — about the cap semantics, about the retry, about the query — and all three evaporated into a conversation that will be closed by Thursday. The next feature starts from the same zero, and so does the next engineer, and so does the model.

In the second, each failure is diagnosed for its cause: the code is wrong because the spec was silent. You write the missing clause into the spec, then regenerate against the amended spec. The spec is now permanently stronger. The next generation cannot make that mistake, because the constraint is in the input. The failure was converted into a durable asset rather than a conversational one.

When to amend versus when to patch. Not every failure is a spec gap, and treating them all as one produces a spec bloated with implementation trivia. The discriminator: would a competent engineer, given this spec, have made this same choice? If yes, the spec is silent on something load-bearing — amend it and regenerate, because the gap will otherwise be re-sampled every time. If no — the spec was clear and the output contradicted it — that is a local slip, so patch it by hand and add a test. A dozen local slips in a row is itself a signal, but about the model or the task decomposition, not the spec.

The unit-versus-integration argument is usually conducted as a matter of taste, speed, or file layout. Reframe it as refutation power and it becomes decidable. A unit test isolates a component and refutes claims about that component's logic, cheaply and fast. An integration test runs a component against real collaborators — a real database, a real HTTP boundary — and refutes claims about the seams: what happens across a transaction, whether the ORM emits the query you think, whether the retry and the idempotency mechanism compose.

Here is the fact that should drive your allocation: AI-written code fails disproportionately at seams. The reason is structural rather than mystical. Each component in isolation is a well-represented pattern the model has seen thousands of times, and it produces those correctly. The joint behavior — this ORM's transaction semantics combined with this retry policy combined with this processor's timeout contract — is specific to your assembly, appears nowhere in training data as a unit, and must be inferred. Inference is where the plausible-but-wrong lives. D2 is the canonical case: the retry loop is textbook, the processor call is textbook, and the composition double-refunds.

A working allocation rule for a system like Ledgerline: unit-test the pure decision logic where the branch space is large and the collaborators are irrelevant — cap arithmetic, role resolution, boundary conditions. Integration-test every place where a belief about a collaborator is load-bearing — the transaction spanning the audit row and the processor result, the tenant-scoped query actually filtering. And reach for a property whenever the claim is universally quantified, which is where we go next.

An example test asks: for this input, is the output right? A property-based test asks: for inputs drawn from a description of the whole space, does this claim hold? You supply generators that describe the space and an assertion that must hold everywhere in it; the framework searches for a counterexample, and when it finds one it performs shrinking — reducing the failure to its minimal form so what you read is the smallest input that still breaks.

The crucial move for verifying AI code is that the generated space includes fault schedules, not just values. Real systems fail in interleavings, and interleavings are precisely what nobody writes example tests for — not out of laziness, but because "the second request arrives after the first timed out following a successful processor call" is not a scenario anyone lists under a feature. Take INV-1 from module 3 and make it executable.

from hypothesis import given, strategies as st

CHARGE_ID = "ch_7c2m"
CHARGE_TOTAL_CENTS = 8400          # $84.00, customer cus_9x4t


class RecordingProcessor:
    """Processor-side ground truth. Performs the side effect FIRST, then
    raises if this attempt is scheduled to time out — modelling the case
    that matters: the refund happened and the response was lost."""

    def __init__(self, fault_schedule):
        self.faults = list(fault_schedule)
        self.performed = []            # what money actually moved

    def refund(self, charge_id, amount, idempotency_key=None):
        if idempotency_key is not None:
            for key, _cid, _amt in self.performed:
                if key == idempotency_key:
                    return {"status": "succeeded", "id": "rf_3n8p"}
        self.performed.append((idempotency_key, charge_id, amount))
        if self.faults and self.faults.pop(0):
            raise ProcessorTimeout("response lost after the refund succeeded")
        return {"status": "succeeded", "id": "rf_3n8p"}


@given(
    requests=st.lists(st.integers(min_value=1, max_value=CHARGE_TOTAL_CENTS),
                      min_size=1, max_size=4),
    faults=st.lists(st.booleans(), max_size=12),
)
def test_total_refunded_never_exceeds_the_charge(requests, faults):
    processor = RecordingProcessor(faults)
    client = make_client(processor=processor,
                         charge=(CHARGE_ID, CHARGE_TOTAL_CENTS, "cus_9x4t"))

    for amount in requests:
        client.post("/refunds",
                    json={"charge_id": CHARGE_ID, "amount": amount})

    moved = sum(amt for _key, cid, amt in processor.performed
                if cid == CHARGE_ID)
    assert moved <= CHARGE_TOTAL_CENTS, (
        "refunded %d cents against an %d cent charge"
        % (moved, CHARGE_TOTAL_CENTS)
    )

Four design decisions carry the test. The stub records processor-side truth, because module 3 established that counting in our own table shows one refund while two occurred. The side effect is appended before the exception is raised, which encodes the only failure mode that matters — success followed by a lost response. The generators cover both request amounts and a fault schedule, so the search space is sequences × failure patterns rather than values alone. And the assertion is the invariant, in cents, unchanged from the spec.

Run it against the handler from module 2 and it fails on roughly the first attempt. Shrunk, the counterexample is two lines long:

Falsifying example: test_total_refunded_never_exceeds_the_charge(
    requests=[4201],
    faults=[True],
)
AssertionError: refunded 8402 cents against an 8400 cent charge

Read what shrinking produced. One request. One timeout. Not a stress test, not a concurrency scenario — the minimum possible failure. The amount shrank to 4201 because that is the smallest value whose doubling exceeds the charge, which tells you the framework understood the shape of the violation. And the mechanism is legible in the output: the handler asked for 4201 cents, the processor performed it, the response was lost, the retry asked again with no key to recognize the resend, and 8402 cents left the building. This is a bug report you could hand to anyone.

The fix is to derive the key once per logical operation, outside the retry loop — one refund request, one key, however many attempts:

idem_key = f"rf:{session['agent_id']}:{body['request_id']}"   # once, before the loop
for attempt in range(3):
    try:
        result = processor.refund(charge_id, amount,
                                  idempotency_key=idem_key)
        break
    except ProcessorTimeout:
        ...

You now have a suite. What refutes the claim that the suite refutes anything? Mutation testing answers it directly: the tool seeds small defects into your source — flip a comparison, delete a call, negate a branch, replace a return with a constant — and runs the suite against each mutant. A mutant the suite fails on is killed. A mutant the suite passes is a survivor, and a survivor is a proof: there exists a defect in your code that your tests do not detect. The mutation score is the kill rate, and unlike coverage it measures detection rather than execution.

Run it on the Ledgerline cap module, whose fixed logic is one function:

def within_cap(already_refunded, amount, cap):
    return already_refunded + amount <= cap

That is the property worth internalizing: a survivor is a named missing assertion, not a vague coverage gap. The corrective is mechanical — add a test asserting within_cap(0, 5000, 5000) is True and its neighbours at 4999 and 5001 — and then the mutant dies and the suite has demonstrably gained the power to detect that defect class. Compare the alternative feedback you would otherwise have received: a coverage report showing 100%, because every one of those lines executed on the happy path.

Mutation testing is expensive — it runs your suite once per mutant — so treat it as a scheduled audit on your highest-radius modules rather than a per-commit gate. Nightly on the money and authorization paths is a defensible policy; on the whole repository per pull request, it will be turned off within two weeks.

Put the pieces into a division of labor you can state to a team, because the point is not that you personally test more carefully — it is that verification stops depending on anyone's care.

The human writes the invariants and the acceptance-level tests. These are module 3's spec made executable, and they must come from outside the generator, because they are the standard the generator is being measured against. The model may write example tests — it is fast and genuinely good at enumerating cases you would skip — but generated tests are treated as an artifact under verification, not as verification. Mutation testing audits the boundary between those two categories, and it is what makes the arrangement enforceable rather than aspirational.

Two repositories run identical static analysis on every pull request. In the first, the tool posts its findings as a comment. In the second, a failure sets the check to red and branch protection prevents merge. The analysis is the same; the control is not remotely the same, and the difference shows up in production defect rates rather than in any dashboard.

The reason is that an advisory finding creates a decision, and decisions get made by tired people under deadline. "Is this f-string SQL actually exploitable here? The charge id comes from our own front end. It's Thursday and this unblocks support." Each individual override is defensible; the aggregate is a control that fires exactly when nobody is under pressure, which is to say when it is not needed. A required check removes the decision. There is nothing to weigh, nobody to persuade, and — this is the underrated part — nobody to be the person who said no.

There is a human argument alongside the technical one, and it is worth stating because it changes how teams behave. Depersonalizing enforcement is a gift to your reviewers. When the standard lives in a person, holding the line costs social capital, and people spend social capital carefully — which means the standard is enforced most weakly against the most senior author and on the most urgent change. When the standard lives in the pipeline, the conversation shifts from "James is blocking my PR" to "the SAST gate is red," and the fix is to fix it. The gate never gets tired, never gets lobbied, and never wants to ship on Friday.

A gate set is not a list of tools; it is a claim that each failure class has an owner. Build it by starting from module 1's taxonomy and asking, for each signature, what mechanism catches it. What follows is a defensible default for a Python service like support-api, stated technique-first — the technique is the gate, and the named tools are instances that will be replaced within a few years.

Two design notes about that table. Type checking earns its slot for a reason specific to generated code: in a typed codebase, a hallucinated method or an invented attribute on a typed object is a type error, so the most-discussed AI failure mode is largely mechanized — the exception being a parameter a **kwargs-accepting client silently absorbs, which stays a review concern. That is a strong argument for type annotations in a codebase written by models even if you found them unnecessary when it was written by you.

Coverage delta rather than absolute coverage is a deliberate choice. An absolute threshold punishes whoever touches a legacy module and can be satisfied by tests that assert nothing; a delta gate asks only that new lines arrive with tests, which is a floor worth having. It is still gameable — module 1's test-shaped tests satisfy it perfectly — which is why the mutation spot-check exists alongside it rather than instead of it.

Return to D3 and watch it end. The line, unchanged from module 2:

cur.execute(f"SELECT amount, customer_id FROM charges WHERE id = '{charge_id}'")

The gate run produces:

>> Issue: [B608:hardcoded_sql_expressions] Possible SQL injection vector
   through string-based query construction.
   Severity: Medium   Confidence: Low
   Location: support_api/refunds.py:12
11    cur = db.cursor()
12    cur.execute(f"SELECT amount, customer_id FROM charges WHERE id = '{charge_id}'")
13    row = cur.fetchone()

Code scanned: 1 file, 34 lines
Total issues by severity: Medium: 1

Exit code: 1  —  required check "sast" failed

The fix is thirty seconds of work and the same in every language:

cur.execute("SELECT amount, customer_id FROM charges WHERE id = %s", (charge_id,))

Now account for what just happened at the level of process rather than code. Zero seconds of reviewer attention were spent. Nobody had to know about SQL injection, remember to look for it, or be alert at 5:40 p.m. The finding arrived before a human read the diff, phrased as an exit code. And the rule persists: the next generated handler, and the one after a model upgrade, and the one written by a contractor in eight months, all hit the same wall.

Note the Confidence: Low in that output, because it is where these programs usually die. Static analysis produces false positives; a rule that fires on safe code trains people to add suppression comments, and a codebase full of suppressions is a codebase with no rule. The discipline is to tune the rule until its findings are trustworthy, require a written justification on each suppression, and review the suppression list quarterly the way you would review a control exception register. A noisy required gate does not survive contact with a deadline.

Gates fail in one characteristic way, and it is not that they are too strict. A required check that fails intermittently for reasons unrelated to the change teaches the team a lesson very efficiently: red does not mean broken, red means re-run. Once that lesson is learned it generalizes, and it generalizes to the gates that are working. This is gate erosion, and its cost is not the flaky suite — it is that a genuine failure in a different gate no longer stops anybody.

Second element of hygiene: overrides are logged exceptions, not conversations. There will be a genuine emergency, and a control with no exception path invites a worse workaround — someone will push to the branch directly or disable the check globally. So define the path in advance: an override requires a named approver who is not the author, a written reason, an expiry, and an entry in a register that gets reviewed. You have run this exact process for control exceptions in a compliance program; import it wholesale, including the part where a persistently-renewed exception is escalated rather than rubber-stamped, because a permanent exception is a policy nobody agreed to.

Third: branch protection is what makes any of this real. Required checks that can be bypassed by pushing directly to the protected branch are advisory checks with extra steps. Configure the protection, include administrators, and accept the small friction — the alternative is that the control is strongest against exactly the people least likely to need it.

Put the two artifacts side by side. The refund handler's D1 lets an authenticated support agent exceed their refund cap and reach other tenants' charges — serious, bounded by what the endpoint can do. An IAM statement granting s3:* on * lets anything holding that role read, overwrite, and delete every object in every bucket in the account, including the export bucket, the backups, and the CloudTrail logs that would tell you it happened. It is D1 at cloud scale, and it fits on two lines.

That compression is the first reason infrastructure review is different. The second is that the blast radius is often not visible in the change's stated purpose. Nobody opens a pull request titled "grant delete access to all buckets." They open one titled "add refund-history export," and the wildcard is inside it, added at 11 p.m. to unblock a test run.

The third reason is the generation dynamic. When a model hits an AccessDenied, it has two paths: work out which specific action on which specific resource was denied and grant exactly that, or widen until the error stops. The second path is shorter, far more common in the training data, and — this is what makes it dangerous — indistinguishable from the first in a green build. Both make the error go away. Only one of them is a permission decision anyone would have approved.

CDK's value is that it hides CloudFormation. That is also its review hazard: the source states intent, and the synthesized template states what was actually promised to AWS. The gap between them is where findings hide, because L2 and L3 constructs and grant* helpers expand into statements the source never displays.

Here is the export feature's source, in full, as it appeared in the pull request:

const exportBucket = new s3.Bucket(this, "RefundExportBucket", {
  encryption: s3.BucketEncryption.S3_MANAGED,
});

const exportRole = new iam.Role(this, "ExportRole", {
  assumedBy: new iam.ServicePrincipal("lambda.amazonaws.com"),
});

exportBucket.grantReadWrite(exportRole);

// unblock the AccessDenied we hit during testing
exportRole.addToPolicy(new iam.PolicyStatement({
  actions: ["s3:*"],
  resources: ["*"],
}));

Twelve lines, encrypted bucket, service principal correctly scoped, a named grant helper. It reads like careful work, and a reviewer moving quickly registers "grantReadWrite — sensible, least privilege, someone thought about this." The comment on the last statement even sounds diligent. Now run cdk diff and read what is actually being promised:

Resources
[+] AWS::S3::Bucket RefundExportBucket
[+] AWS::IAM::Role ExportRole
[+] AWS::IAM::Policy ExportRole/DefaultPolicy
     Statement 1:
       Action:   s3:GetObject, s3:GetObjectVersion, s3:GetBucket*,
                 s3:List*, s3:PutObject, s3:PutObjectLegalHold,
                 s3:PutObjectRetention, s3:PutObjectTagging,
                 s3:PutObjectVersionTagging, s3:Abort*, s3:DeleteObject*
       Resource: arn:aws:s3:::RefundExportBucket, .../*
     Statement 2:
       Action:   s3:*
       Resource: *

Two things the source did not show. Statement 1: grantReadWrite expanded to eleven actions including s3:DeleteObject* — defensible for this feature, and worth knowing, because "read write" in English does not obviously include delete. Statement 2 is the finding: every S3 action on every resource in the account. That includes s3:DeleteBucket on the backups bucket, s3:PutBucketPolicy on any bucket (which is how a private bucket becomes public), and read access to whatever else lives in this account.

Read a policy statement the way you would read an indemnity clause: every position is a term, and a wildcard in any position is an unbounded term. Three positions matter.

Action — what may be done. s3:* is not "S3 access"; it is roughly one hundred and thirty operations, including DeleteBucket, PutBucketPolicy (grant public access to anything), PutBucketVersioning (disable the versioning that makes deletion recoverable), and PutBucketLogging (turn off the record of what happened). The last two are why a wildcard on action is worse than the sum of its obvious parts: it includes the operations that disable the controls detecting its abuse.

Resource — what it may be done to. * means every bucket now and every bucket created in future, by anyone, forever. This position ages worst: the grant was written when the account had three buckets and silently expands as the account grows.

Condition — under what circumstances. Usually absent, which is fine for a service role, and load-bearing whenever a third party can assume the role (the confused-deputy conditions) or the grant should be limited by source, tag, or MFA state.

Now the corrective for statement 2, derived properly. Ask what the export feature actually calls: it reads refund rows, writes one CSV per night, and lists the prefix to find prior exports. That is three actions on one resource:

exportRole.addToPolicy(new iam.PolicyStatement({
  actions: ["s3:PutObject", "s3:GetObject", "s3:ListBucket"],
  resources: [
    exportBucket.bucketArn,
    exportBucket.arnForObjects("refund-history/*"),
  ],
}));

Better still, delete the hand-written statement entirely and use the grant helper scoped to the prefix — exportBucket.grantPut(exportRole, "refund-history/*") — because helpers are reviewed, versioned, and consistent, whereas hand-written statements are where wildcards go to hide. The general rule: prefer grants over hand-written policy statements, and treat a hand-written statement in a diff as a finding until justified.

Some infrastructure changes are mistakes you fix, and some are events you have. The line between them is removal policy and exposure, and both classes are quiet in a diff.

In CDK, RemovalPolicy.DESTROY on a stateful resource means the data goes when the resource goes — on a stack teardown, on a resource replacement triggered by an unrelated property change, on a bad rollback. It is the correct setting for a scratch bucket in a development stack and a data-loss incident waiting for a trigger on anything that stores facts.

The export diff flips RefundAuditLog from Retain to Delete, and it does so inside a pull request about exports — almost certainly because the model saw a teardown fail on a retained table during testing and removed the obstacle. Look at what that table is. It records which agent authorized which refund for which customer at what time. It is the evidence trail for every money movement in the system: the artifact you produce when a customer disputes a refund, when an auditor samples authorization controls, and when the question is whether agt_d4k1 was permitted to refund ch_7c2m.

Exposure is the second irreversible class, and it is irreversible in a different sense: the resource can be closed in a minute, but you cannot un-disclose what was read. A security group opening port 22 to 0.0.0.0/0 "for debugging," a bucket made public to make a front-end work, a database subnet made routable — for each of these, the question is not "can we revert it" but "between deploy and revert, what could have been taken, and would we know?" With versioning disabled and logging off, the honest answer is often no.

Practical discipline for both classes: they get a different gate, not just a more careful reader. A policy scan that fails the build on any removal-policy change to a stateful resource, on any ingress rule with a 0.0.0.0/0 source, and on any public-access-block removal, converts "we should have noticed" into "it could not merge." This is module 5 applied to infrastructure, and it is where the majority of the value is — the reader's job is to catch what the scan cannot, which is whether a correctly-formed grant is the right grant.

Cost belongs in this module because it is a blast-radius dimension with the same two variables as everything else: magnitude and reversibility. Treating it as a finance concern rather than a review concern is how a diff about a nightly export acquires a permanent monthly bill.

The export diff adds provisioned concurrency of 50 to ExportHandler. Provisioned concurrency keeps execution environments warm and bills for every second they are warm, whether or not anything invokes them. Do the arithmetic in the review, because a number changes the conversation:

50 environments × 1 GB × 730 h/month × 3600 s/h  =  131,400,000 GB-seconds
131,400,000 × $0.0000041667                      ≈  $548 / month

That per-GB-second figure is a published price, and published prices move — look up the current one before you quote a number in a review. The arithmetic is the durable part. The function runs once a night for about four minutes. The team is proposing roughly $6,500 a year to eliminate a cold start on a batch job that nobody waits for. Stated that way the finding needs no security framing to be obvious — but nobody states it that way unless someone in review does the multiplication, and the diff line itself reads as an innocuous performance setting that a model added because performance settings are what you add when asked to make something faster.

The cost surfaces worth memorizing, because they recur: provisioned concurrency and reserved capacity of any kind; NAT gateways (per-hour plus per-gigabyte, and the classic surprise in a private-subnet design); log retention set to "never expire" on a high-volume group; cross-availability-zone and egress data transfer; per-request-priced services multiplied by a retry loop; and anything with the word "provisioned" or "reserved" in its name, which is the general form of "you are now paying for idle."

A dependency is not a library. It is a standing agreement with a party you have not met, under which their code executes inside your process, with your environment variables, your database connection, and your cloud role, whenever they publish a new version and you install it.

Stated that way the risk profile is obvious, and it is worth restating because familiarity has worn it smooth. Your direct dependencies are counterparties you chose. Their dependencies are counterparties chosen for you, by people optimizing for their own convenience. In a graph of four hundred packages, the number you have evaluated is roughly the number you typed, and the number that can execute code at install time or import time is all of them.

What AI assistance changes is who makes the onboarding decision. When a model writes import redis, the choice to trust that project was made by a token sampler. Most of the time it picks the obvious, widely-used package and nothing is wrong. The failure mode is not that models pick bad packages on average — it is that the decision has moved somewhere with no notion of counterparty risk, and unless you take it back deliberately, it will be exercised hundreds of times without ever being made.

Models hallucinate package names, and they do it with statistical regularity rather than randomly. Asked repeatedly for a library that solves a common problem, a model will propose the same plausible-but-nonexistent names — flask-idempotency-shim, boto3-retry-helpers, requests-retry-adapter — because those names sit where the naming conventions of the ecosystem say such a package would sit. Regularity is the vulnerability: a hallucination that repeats is a hallucination an attacker can predict.

The chain, with our running example. A model is asked to fix D2 and suggests pip install flask-idempotency-shim — a name that does not exist, invented by convention-matching. An attacker, having mined thousands of model outputs for exactly such names, has already registered it on the public index, with a plausible README, a version history, and a setup.py whose install hook posts the contents of the environment to a collection endpoint. A developer or an agent runs the install. It succeeds — of course it succeeds, that is the attack — and the payload runs with whatever the shell had: PROCESSOR_KEY, the database URL, and any AWS credentials in scope.

Two control points break the chain, and they are different in kind — one is a judgment, one is a mechanism, and neither substitutes for the other.

CP1 — vet before add. A human verifies a package before its name enters the dependency set. Not after the install; before, because install-time code has already run by then. The check takes under five minutes: does the project exist on its own terms (a repository, an issue tracker, a history), who publishes it and what else have they published, when was it last updated and by how many people, and does the download pattern look like adoption or like a spike from nowhere. A package registered three weeks ago, by an account with no other work, with a version history that starts at 1.0.0, is the signature — and it is visible in thirty seconds on the index page.

CP2 — deterministic installs. CI and agent sandboxes install only from the lockfile, with hash verification, and never resolve a new name. This is the mechanical control, and it is the one that holds when nobody is paying attention: an agent that decides mid-task to install something helpful simply cannot, because the install command is constrained to the recorded set. CP1 depends on a person doing a thing; CP2 does not, which is why the pair is stronger than either.

The distinction between pinning and a lockfile is the most commonly misunderstood thing in this module, and CP2 rests on it entirely.

Pinning fixes versions in the manifest — the file you write. flask==3.0.2 is a pin. It is a policy about your direct dependencies and says nothing whatsoever about theirs. Flask depends on Werkzeug, Jinja2, itsdangerous, click, and blinker, each with a version range; those resolve at install time, to whatever satisfies the range today. Install the same pinned manifest in March and in July and you get different code.

A lockfile records the resolution: every package in the transitive closure, its exact version, and a cryptographic hash of the artifact. It is not a policy about the future — it is a record of the present, an answer rather than a question. Installed with hash checking enabled, it guarantees that what runs in CI, on your laptop, and in production is byte-identical, and that any artifact whose hash does not match the record fails the install rather than proceeding.

That last clause is the security property, and it is what makes CP2 mechanical rather than aspirational: a package that is not in the lockfile cannot arrive as a side effect of anything. Not a transitive bump, not a compromised upstream release, not an agent deciding a helper library would be useful. The install command has no discretion left.

Dependency audit tools match your resolved graph against databases of known-vulnerable versions. They are cheap, they belong in your required checks, and they have a coverage boundary that is routinely misunderstood.

An audit answers: does my graph contain a package-version known to be bad? A slopsquat is not known to be bad. It was registered three weeks ago, nobody has reported it, and there is no advisory to match against — the database describes the past, and this attack is designed to be new. So a green audit is compatible with having installed a malicious package this morning, which is a fact worth saying out loud on any team that treats the audit gate as "supply chain: handled."

Update pull requests are code changes with a blast radius, and the automation that opens them makes them feel like a chore rather than a decision. When a bot proposes bumping a package, what is actually being proposed is that a third party's new code gets your credentials on the next deploy. Apply module 2's protocol: the interesting artifact is the lockfile diff, not the version number in the manifest, because that is where new transitive packages appear — a patch bump on a direct dependency can introduce three packages you have never seen, and each is a new counterparty.

A practical cadence that works: update on a schedule rather than continuously, so the changes arrive as a reviewable batch rather than as constant background noise you learn to click through; keep the audit gate required so known-vulnerable versions cannot linger; and treat any package whose maintainership changed since your last update as a new counterparty requiring the CP1 check, because a transfer of publishing rights is the most common way a trusted package becomes an untrusted one.

Five rungs, each a strictly larger grant of blast radius, each with a prerequisite that is not optional.

Rung 1 — autocomplete. The agent proposes; you accept keystroke by keystroke. Radius: nothing reaches disk without your hands. Prerequisite: none.

Rung 2 — diff for review. The agent produces a change set; you read it before it exists anywhere durable. Radius: your attention is the only control. Prerequisite: the module 2 protocol — without an ordering rule, more diffs simply means less attention per diff, and the rung silently degrades into rubber-stamping.

Rung 3 — branch with a pull request. The agent works independently and opens a PR. Radius: still bounded by review, but now the code exists, CI runs, and volume goes up. Prerequisite: the module 5 gate set actually required, because at this volume gates do the work review used to.

Rung 4 — merge on green. The agent merges when checks pass, without a human read. Radius: your main branch. Prerequisite: a suite with demonstrated refutation power — module 4's properties on the invariants and a mutation score you have actually measured. "CI is green" means nothing until you know what green can detect; merge-on-green with a 40% kill rate is merge-on-nothing.

Rung 5 — deploy on green. The agent ships to production. Radius: customers. Prerequisite: everything above, plus canary deployment with automatic rollback, plus the eval gates in section 4.

Reviewing a plan is the cheapest review you will ever perform, and the leverage comes from a simple asymmetry: a plan is two hundred words and a wrong plan becomes four hundred lines. Intent divergence — the class module 2 established is human-only — is fully visible in the plan and expensively buried in the diff.

A reviewable plan is short and states four things: the files it intends to touch, the approach in a sentence or two, the invariants it will honor, and what it is explicitly not doing. That last item does most of the work. A plan that says "not modifying the authorization middleware, not changing the refunds schema" is falsifiable at execution time and tells you immediately when scope has drifted.

PLAN — fix the non-idempotent retry in POST /refunds

Touching:  support_api/refunds.py, support_api/processor.py,
           tests/test_refunds_property.py
Approach:  derive an idempotency key once per request from
           (agent_id, request_id), pass it on every retry attempt;
           processor client forwards it as Idempotency-Key.
Honors:    INV-1 (total refunded at the processor never exceeds
           the charge), INV-2 (audit row transactional with result).
Not doing: no change to the cap logic, the authorization checks,
           the refunds schema, or the retry count.

Sixty seconds to read. Compare the alternative — inferring that same intent from a diff, where an unrequested change to the cap logic looks exactly like a requested one, and where you would have to reconstruct "was this in scope" from a chat transcript.

Two gates make the loop work, and both are load-bearing. The approval gate sits after the plan and before execution; rejection loops back with an amended spec rather than a conversational correction, which is module 3's tightening loop embedded in the agent workflow. The CI rejection path catches what the plan review cannot: the agent submits, a gate fails, and the agent repairs against the failure output and resubmits — a loop that can run without you, because the gate's output is machine-readable and unambiguous. That second loop is the mechanism that makes higher rungs economical: the agent handles its own mechanical failures, and you see only what needs judgment.

An agent's blast radius is not a property of its instructions. It is the set of things reachable from inside its execution environment — and prompts do not constrain that set, because a prompt is a request and the environment is a fact.

This matters more for agents than for other software because of the input surface. An agent reads issue text, code comments, web pages, error messages, and dependency README files, and any of those can contain instructions. Prompt injection is not exotic; it is the normal consequence of a system that treats retrieved text as input. So the design question is not "how do we stop the agent from being tricked" — assume it will be — but "when it is, what can it reach?"

Sandbox design, concretely, for the Ledgerline agent: a container built from a pinned image with the locked dependency set already installed and no egress to the package index (module 7's CP2 made structural); network egress limited to an allow-list — the source host, the model API, nothing else; credentials issued per run with a one-hour expiry, scoped to the support-api repository and a non-production AWS account; no access to the production database URL, the processor API key, or any secret that is not required by the task class; and the filesystem confined to the checkout.

Note what the third item rules out, because it is the mistake people make after doing everything else right: putting the production database URL in the sandbox's environment defeats the sandbox entirely. The container is isolated; the credential in it is not. A sandbox is defined by what is reachable from inside, and an environment variable is very much inside.

Required checks verify the work. At higher rungs you also need to verify the worker, because the thing you granted autonomy to is not a fixed artifact: the model changes under you, your prompts and tool definitions change, and a system that was safe at rung 4 in March can be a different system in June with nothing in your repository having changed.

An eval-style gate is a golden-task suite the agent must pass to hold its rung. Concretely for Ledgerline: fifteen tasks drawn from real history, each with a known-good outcome and a scored check — three where the correct behavior is to refuse or escalate (a task requiring an IAM change while the agent sits at diff-for-review for that class), three seeded with the module 1 signatures to confirm the agent's own review catches them, and one containing an injection attempt in an issue comment. It runs on a schedule, on every model version change, and on every change to the agent's prompts or tools. A drop below threshold demotes the agent a rung automatically — which is the point, since the alternative is discovering the regression through its consequences.

Now the deploy button. The question "when may an agent deploy unattended" feels like a judgment about trust, and the answer is that it is a question about policy encoding. An agent may deploy when the pipeline encodes the entire deployment decision: required checks that refute the failure classes you care about, eval gates confirming the agent still holds its rung, a canary that exposes a fraction of traffic first, and automatic rollback on defined error and latency thresholds.

At that point notice what has happened to the button. If all of those conditions are encoded and enforced, a human pressing deploy adds nothing — they are not re-deriving the checks, they are observing that the checks passed. And if they are not all encoded, a human pressing deploy adds only their mood: their sense that today feels fine, which is exactly the miscalibrated instinct module 1 spent thirty minutes dismantling. The button was never load-bearing. The human mood was, and that is the thing to replace.

Every anti-pattern named in this course, in one table. The organizing insight is in the last column: most of these are misallocations — work being done at a layer structurally incapable of doing it, usually by a person doing a machine's job or a machine being asked for judgment.