Workflow · last verified 2026-07-18

The AI quality stack: how you know the LLM is delivering

The job this workflow does

The senior trap is reaching for one control — "we run evals" or "we added a guardrail" — and calling output quality solved. A single gate catches a single failure class and lets the rest through. The evidence is blunt: AI-generated code security pass rates have held flat at roughly 55% since 2025 (Veracode, Spring 2026 update), with Java the worst-performing language at about 29% secure and cross-site scripting at about 15% secure. The model's raw output is, on the security axis, close to a coin flip. The gates are not optional polish; they are how a coin flip becomes something you can ship.

The threat catalog the gates map against is the OWASP Top 10 for LLM Applications (2025 edition) — prompt injection (LLM01), improper output handling (LLM05), excessive agency (LLM06), sensitive information disclosure (LLM02), and the rest. No one gate answers the whole list. Order them cheap-and-early to expensive-and-late so the fastest checks reject the most before you spend a human's attention or a red-teamer's hour on what a linter would have caught.

We run this chain on this platform, and the proof panel below is our own gate output, not a vendor diagram. Each step names what it catches, the one check that proves the gate is live, and where to go to install it properly.

The nine gates, in order

1. Spec-first framing — catches building the wrong thing

The most expensive defect is a feature that works perfectly and solves the wrong problem, because no test suite flags it. The gate is a reviewed specification that exists before the code — a written statement of the business objective, the user outcome, and the acceptance criteria, signed off while changing it is still cheap. Make its presence a checklist item on every pull request, so "there is a spec, and a human reviewed it before code" is a state you can verify, not a habit you hope for.

# PR checklist gate: the spec for THIS feature exists (and was reviewed) before its code
test -f "tasks/${FEATURE}-SPEC.md"

Deep page: spec-first development — the brainstorm to spec to plan to implement workflow, with the human checkpoints and the deterministic hooks that enforce them.

2. Context engineering — catches wrong-convention output

An LLM with no project context writes code that is correct in the abstract and wrong for your repo: the wrong naming convention, the wrong test runner, an HTTP framework you banned two years ago. The gate is a context file (a CLAUDE.md, an agent instruction set, a retrieved convention pack) that is present and current. Stale context is worse than none, because it looks authoritative — so the check is freshness, not mere existence.

# is the context file current? a commit date older than the stack it describes = stale
git log -1 --format=%cs CLAUDE.md

Deep page: context engineering — how to build, layer, and keep the context current so this gate stays green.

3. Structured output — catches shape drift

The moment an LLM response feeds another system, its shape becomes a contract, and "mostly the right JSON" is a production incident waiting for the one field the model dropped. The gate is a schema, validated on every response in CI — not a prompt that asks nicely for JSON, but a validator that fails the build when the shape drifts. If the output does not match the schema, it never reaches the code that assumed it would.

# schema-validate every LLM response in CI; a non-conforming response fails the build
npx ajv-cli validate -s schema.json -d response.json

4. Machine gates — catches defects and vulnerabilities before a human looks

Never spend human review on what a machine finds for free. Before a person reads a line of AI-generated code, run the deterministic gate: type-check, lint, static application security testing, infrastructure-as-code scanning, and a dependency audit — chained so the first failure stops the pull request. Given the ~55% security pass rate above, this layer is where most of the model's introduced vulnerabilities get caught cheaply.

# one chained machine gate: type + lint + SAST + IaC scan + dependency audit
npx tsc --noEmit && npx eslint . && semgrep scan --config auto && checkov -d infra/ && npm audit --audit-level=high

Each tool is in-stack and free to run locally: semgrep scan --config auto is Semgrep's no-account local invocation (the free open-source CLI; the paid AppSec Platform adds supply-chain and secrets scanning); checkov -d scans Terraform and is maintained by Prisma Cloud (Palo Alto Networks); npm audit --audit-level=high exits non-zero at or above the given severity so CI can gate on it. On pnpm, the equivalent is pnpm audit --audit-level high (space-separated value, narrower set: low, moderate, high, critical).

Deep page: AI-assisted code review — the diff-discipline workflow this machine pass front-loads, now with the SAST, IaC, and dependency-audit steps installed.

5. Evals — catches quality regression

Machine gates prove the code is well-formed and free of known vulnerability patterns; they say nothing about whether the LLM feature is any good. That is what evals measure, and an eval suite is only a gate when it runs as a release gate with a numeric threshold — not a dashboard someone checks when they remember.

# eval suite as a release gate with a numeric threshold
npm run eval   # this repo additionally gates coach changes at 80% pedagogy adherence

Deep pages: eval-first development — build the harness before the feature; and the eval-harness gate pattern — wire the threshold into CI so the suite blocks deploys.

6. Guardrails — catches unsafe I/O

Evals measure quality on the inputs you thought of; guardrails constrain behavior on the inputs an adversary picks. The gate sits on the live I/O path — screening prompts for injection (OWASP LLM01) and responses for improper output handling (LLM05). The check that keeps it honest is a regression set of previously blocked prompts, replayed against the gateway, asserting that every one still flags. A guardrail that silently stopped catching yesterday's attack is worse than none.

# replay the blocked-prompt regression set; exits non-zero if any probe stopped flagging
fail=0
for p in redteam/blocked/*.txt; do
  curl -s "$GATEWAY" --data-binary @"$p" | grep -q '"flagged":true' || { echo "REGRESSED: $p"; fail=1; }
done
exit $fail

Deep page: defense in depth — layering guardrails with the gates above and below so no single control is load-bearing.

7. Human-in-the-loop — catches irreversible actions

Some actions cannot be graded after the fact because they cannot be undone: a payment, a deletion, an email to a customer. For these, the gate is a human approval step wired in front of execution — a structural guarantee, not a prompt asking the model to be careful (excessive agency is OWASP LLM06 precisely because models are not careful on their own). Prove it by inspecting the tool definitions: every irreversible tool must carry an approval gate.

# gate: fails if any irreversible tool definition lacks an approval gate
test -z "$(grep -L 'requireApproval' tools/irreversible/*.ts)"

Deep page: the HITL approval pattern — where to place the checkpoint, how to make it non-bypassable, and how to keep it from becoming a rubber stamp.

8. Observability — catches silent production drift

Every gate so far runs before or at deploy. Production is where the model, the traffic, and the costs drift out from under all of them, silently, until a bill or an incident makes it loud. The gate is per-turn cost and latency, logged and alarmed — and the spot-check is that no production turn is missing its record, because an unmonitored path is a blind spot by definition.

# spot-check gate: fails if any turn log file carries no cost record
test -z "$(grep -L 'cost_usd' logs/turns/*.jsonl)"

Deep pages: LLM observability and AgentOps for senior engineers — tracing, cost accounting, and drift alarms for LLM and agent systems in production.

9. Red-teaming — catches what the gates missed

The final gate assumes the previous eight are imperfect and goes looking for the gap. A scheduled adversarial suite generates attacks — jailbreaks, prompt extraction, injection — and runs them against the live target on a cadence, so a regression in any earlier gate surfaces as a failed probe instead of a breach. promptfoo's red-team runner is the in-ecosystem option: MIT-licensed, with red-teaming in the free Community tier (up to 10,000 probes per month), driven by a promptfooconfig.yaml that names the target, the attack plugins, and the delivery strategies.

# scheduled adversarial suite; reads promptfooconfig.yaml in the working directory
npx promptfoo@latest redteam run

Deep page: red-teaming LLM systems — the attack surface, how to build the suite, and the cadence that keeps it useful.

Governing the stack: the AWS Well-Architected Generative AI Lens

Nine gates are an implementation; an auditor and a review board want a framework to hold them against. AWS published the Well-Architected Generative AI Lens on 19 November 2025 for exactly that: it maps generative-AI concerns onto the six standard AWS Well-Architected Framework pillars, covering the lifecycle from scoping through model selection, customization, development, deployment, and continuous improvement (scoped to foundation models on Amazon Bedrock or customer-managed models on Amazon SageMaker AI, and explicitly excluding model training).

The gates above fall out of its focus areas cleanly. Operational Excellence asks for consistent model output quality, traceability, and monitored operational health — gates 5 and 8. Security asks you to protect the endpoints, mitigate harmful-output and excessive-agency risks, and secure prompts — gates 4, 6, 7, and 9. Reliability asks for observability and graceful failure handling — gate 8. Performance Efficiency and Cost Optimization ask you to hold acceptable performance and cost — the per-turn cost record from gate 8 again. And the lens frames Responsible AI as a shared responsibility across model producers, providers, and consumers — the reason a consumer of a foundation model still owns every gate on this page. Use the lens as the review scaffold; use the nine gates as what you actually install. See also AI agent governance for the policy and audit layer around them.

As-built: this platform's own gate chain

This library's differentiation is that we run what we document. The chain that ships this platform's coach agent, stated as evidence rather than advice:

• The test suite runs green without spending a cent. The full suite runs keyless — OPENROUTER_API_KEY= npm test — with the one live-spend eval skipped unless a real key is present. A sub-ten-second run is the proof no real tokens were burned.
• Prompt changes clear a live 80% gate. Any change to the coach's prompt or model must pass a live pedagogy-adherence gate at 80% (median of three runs, with early exit), on top of the free offline npm run eval suite that runs on every change.
• Guardrails run in FLAG mode, deliberately. The gateway's guardrails flag rather than block, because block mode was verified to trip on the platform's own prompt-injection lesson — the security curriculum would have been censored by the security control. The right setting was an evidence-backed call, not a default.
• Public copy passes a claims-reviewer before ship. A dedicated review agent checks every member-facing claim — pricing, access state, positioning — against the product's commercial facts before anything is committed or deployed.
• The reliable gates are deterministic, not prompt instructions. When a prompt-only guardrail failed three times to hold, the fix was a deterministic hook that fires on what changed — backed by a release gate that holds even when everything upstream misses. A gate you can bypass by rephrasing is not a gate.

Artifacts this workflow produces

• A reviewed spec file per feature, checked before code (gate 1).
• A current context file whose freshness is verifiable (gate 2).
• A response schema validated in CI, plus the failing-build wiring (gate 3).
• The chained machine gate — type, lint, SAST, IaC scan, dependency audit — as one CI step (gate 4).
• An eval suite with a numeric release threshold that blocks deploys (gate 5).
• A guardrail regression set replayed on every change (gate 6).
• Approval gates on every irreversible tool, inspectable in the tool definitions (gate 7).
• Per-turn cost and latency records with a drift alarm (gate 8).
• A scheduled red-team config and its probe history (gate 9).

When the full stack is overkill

You do not always need all nine. If the output is deterministically checkable end to end — a schema validator and a compiler already tell you pass or fail — gates 3 and 4 are your eval, and the judge machinery of gate 5 is ceremony. A genuinely throwaway prototype whose every output a human reads individually can skip guardrails and red-teaming until it stops being throwaway. The honest move is to name which gates you are skipping and why, not to skip them silently: the moment a change can degrade something a user depends on without anyone noticing, you are past the exemption and the missing gate is a decision you now own.

Changelog

• 2026-07-18 — initial version. Tool invocations, OWASP LLM Top 10 categories, the AWS Well-Architected Generative AI Lens structure, and the Veracode figures verified against current primary sources on 2026-07-18.