The control layer

AI guardrails: enforcing safe, valid LLM and agent behaviour

What AI guardrails are

A guardrail is a check that runs outside the model, on the model's input or output, and can pass, block, rewrite, or escalate. The model itself is non-deterministic and persuadable; a guardrail is the deterministic-or-classifier boundary you put around it so its behaviour stays inside a defined envelope. Two properties define one: where it runs (before the model sees input, or after it produces output) and how it decides (a fixed rule like a regex or schema check, or a model-based classifier that scores the content).

Guardrails are not the same as the model's own alignment or a system prompt. A system prompt is an instruction the model may ignore or be talked out of; a guardrail is code that runs regardless of what the model decided. That separation is the point — controls you can audit and test, sitting between the model and the user.

Input vs output guardrails

Guardrails come in two positions, and most production systems run both:

• Input guardrails run on the request before it reaches the model: detecting prompt-injection and jailbreak attempts, stripping or flagging PII, blocking off-topic or disallowed requests, and enforcing length or rate limits. They are the cheapest place to stop a bad interaction — the model never runs.
• Output guardrails run on the model's response before it reaches the user or a downstream tool: filtering unsafe content, checking the answer is grounded in the retrieved source, validating that structured output matches a schema, and inspecting tool calls before they execute. They are the last line — what catches a model that was successfully manipulated despite the input check.

The asymmetry matters: input guardrails protect the system from the user; output guardrails protect the user (and your downstream systems) from the model. Skipping either leaves a gap — input-only misses a hijacked generation, output-only wastes a model call on a request you could have refused up front.

Types of guardrail

Across frameworks the same families recur. Knowing them by what they enforce — not by a vendor's product name — is what lets you compose the right set.

These compose: a single request might pass through a PII redactor and an injection classifier on the way in, then a safety filter, a schema validator, and a tool-call check on the way out. Each is independent, so you add or remove one without rewriting the others.

Where guardrails sit (and the trade-offs)

Architecturally, guardrails are part of the operational plane — the layer that wraps the model and its tools. They are not free, and the trade-offs are the real design work:

• Latency. A model-based guardrail is itself an inference call. Run two on input and three on output and you have added five round-trips around every turn. Deterministic checks (regex, schema) are cheap; classifier-based checks are not — reserve them for risks a rule can't catch.
• False positives. Too strict and the guardrail blocks legitimate requests, degrading the product; too loose and it misses real ones. This threshold is a tuning problem with no universal setting — it depends on your risk tolerance and is something you pin with evals, not vibes.
• Determinism vs coverage. A rule is fast, auditable, and predictable but only catches what you enumerated; a classifier generalises to novel inputs but is itself a model that can be wrong or evaded. Most systems layer both — rules for the known, a classifier for the rest.
• Defence in depth, not a wall. No single guardrail is complete. They reduce the probability and blast radius of a bad outcome; they do not eliminate it. Treat them as one layer of the threat model in agent security, paired with least-privilege tools and bounded loops.

What the guardrail does on a match is a third setting, and it is usually three-way, not two. The instinct is to think in block-or-nothing terms, but the products themselves are more granular. Cloudflare's AI Gateway Guardrails is the explicit case: each hazard category is set to flag, ignore, or block, and prompts and responses are configured separately, so one deployment can block a category on the way in and only flag it on the way out. Block stops the interaction. Flag lets it through and logs it — and because logs are written for every request, including the ones that pass clean, flag mode is how you learn what your real traffic looks like before you commit to blocking any of it. Ignore switches a category off entirely, which is the right answer for a hazard that structurally cannot apply to the surface: an unevaluated category costs you no latency and no false positives.

The rule of thumb: block where a match is unambiguously abuse and a false positive is cheap; flag where the same content is legitimate in one context and hostile in another, or where you are still learning your traffic; ignore what cannot apply. Check the failure behaviour too, because it follows the mode rather than being a separate setting — Cloudflare documents that when the evaluation is unavailable, a category set to block blocks the request, while one set to flag lets it proceed without evaluation. Fail-closed and fail-open are not a choice you make independently of the mode; they come with it.

Scope is the trade-off that gets missed, because it does not look like a trade-off: every placement covers some surfaces and not others. A response check that only sees buffered responses does not see streamed ones; an input classifier reading the user's message does not see a retrieved document, a tool-call argument, or the system prompt unless it was wired to. The gap between the surfaces you assume are covered and the surfaces the guardrail actually inspects is where the incidents live. Write the covered surfaces down per guardrail and test each one — coverage is a property you verify, not one you infer from the feature being switched on.

The main guardrail frameworks

Several open-source toolkits and platform services implement these controls. They overlap heavily; the choice is mostly about where your stack already lives and whether you want a library you host or a managed service. Verify exact capabilities against each vendor's live docs before building — this space moves quickly.

The pattern across all six is identical to the types table above: input checks, output checks, deterministic rules, and model-based classifiers. Pick by integration cost, not by feature lists that mostly converge. This site's own coach runs the same idea in miniature — a bounded loop with input and tool-call checks, built across Anthropic, AWS, and Cloudflare.

As-built: one control, three separate qualifications

This platform routes its coach calls through an authenticated Cloudflare AI Gateway with Guardrails enabled. Describing that honestly takes three qualifications, and the fact that it takes three is the lesson.

The mode is not one mode. We ran block mode and it flagged our own security curriculum against us: a lesson question about prompt injection, legitimate teaching content, returned a 424 and kept a learner out of a lesson about the exact attack. So that one hazard category is set to flag. The other thirteen are still block, on both the prompt and the response side. "We run it in flag mode" is the sentence that sat in our own decision record from June until we re-read the dashboard in July, and it was wrong — a per-category setting had been compressed into a per-gateway one. The failure semantics diverge with it: the flagged category proceeds without evaluation if the evaluator is unavailable, the blocked thirteen refuse the request. One control, two opposite behaviours under the same outage.

The coverage is not what the toggle implies. Cloudflare documents that streaming is not supported when using Guardrails, and the coach streams its replies token by token (streamComplete in src/lib/llm.ts). We measured rather than inferred: across four live authenticated-gateway calls, the first token arrived 1ms after the request against 2032ms of total generation — nothing buffered the response to evaluate it. So we do not count the gateway guardrail as a control over the coach's streamed output. The prompt side still fires — a hazard match returns 424 before generation, which we confirmed identically on paired streaming and non-streaming calls — but on that path the response side inspects nothing. Coverage split down the middle of a single request. What Guardrails does cover end to end is every non-streaming model call we make — grading, the practical judge, the guides answerer, the injection judge's own call. The dashboard read "on" throughout. Configuration is not coverage.

And it was never the injection defence. Guardrails evaluates hazard categories with a content-safety classifier; it is not a substitute for application-level reasoning about your own logic, and the Cloudflare WAF-side detection that does score prompt injection is an Enterprise-tier paid add-on, not part of the gateway feature we run. The control is our own code. An OutputScanner (src/lib/promptSafety.ts) scans the coach's stream chunk by chunk against a fixed canary list — plus a key-leak list on turns where an answer key is still unrevealed — holding back the last few characters of its buffer on every push so a canary split across two chunks is caught before either half is emitted; on a hit the loop discards the reply and signals the client to clear what it already displayed, rather than keeping the safe-looking prefix. The tool boundary works the same way: the coach's learning-state write tools latch off for the remainder of any turn that has ingested an external doc-search result, checked before dispatch regardless of the order the model asks for its tools in (src/lib/tools.ts). Both are deterministic and in the request path. Each such decision also fires a durable audit row through src/lib/audit.ts — best-effort, so a failed write degrades the record and never the control — because a decision that only reached a log line is one you cannot prove ran afterwards.

The newest layer needs stating most carefully of all. An input-injection judge (src/lib/injectionJudge.ts, added 2026-07-25) resolves sequentially before the tutor is called and classifies only the learner's own recent turns — never the system prompt, the retrieved curriculum, or tool results, which is precisely what keeps it from reintroducing the block-mode false positive on our own security lessons. Its own call runs through the same gateway, which constrains how it can be written: its policy text may never quote a verbatim attack phrase, because a literal one returns a 424 and the policy ships on every call — the judge would block itself on every turn rather than only on hostile ones. It is deliberately fail-open: on a timeout, an unparseable verdict, a gateway block on its own call, or an open circuit breaker it returns unavailable and the turn proceeds unjudged. Note what the third of those means: the gateway's effect on the judge is to switch it off, not to protect it. So it lowers the odds that an override attempt reaches the tutor; it is not a control that provably ran on any given message, and we do not treat it as one. The two deterministic controls above are what we lean on. Which of your controls degrade quietly, and what is still standing when they do, belongs in the same document as the list of surfaces each one covers — and if that is too much nuance to compress into a sentence, write less rather than write it loosely. A compressed description that reads as a guarantee is how we got the wrong sentence in the first place.

Frequently asked questions

What are AI guardrails?

AI guardrails are programmatic controls that run around a model, on its inputs and outputs, to keep its behaviour safe, on-topic, and valid. They pass, block, rewrite, or escalate content using deterministic rules (regex, schema checks) or model-based classifiers — independently of what the model itself decided, which is why they hold even when a system prompt is ignored or subverted.

What is the difference between input and output guardrails?

Input guardrails run on the request before the model sees it — detecting injection, stripping PII, blocking off-topic asks. Output guardrails run on the response before it reaches the user or a tool — filtering unsafe content, checking grounding, validating schema, and inspecting tool calls. Input guardrails protect the system from the user; output guardrails protect the user and downstream systems from the model. Production systems generally run both.

What are LLM guardrails?

LLM guardrails are the same controls applied specifically to large language model applications: the input and output checks — content, topic, format, PII, grounding, and tool validation — that wrap a single model call or an agent's loop. The term is used interchangeably with AI guardrails; LLM guardrails just names the model class they are most often built for.

Do guardrails stop prompt injection?

They reduce it, but do not eliminate it — and the first limit is scope, not strength. A guardrail can only evaluate the surfaces it actually inspects, and those are usually narrower than the deployment assumes: a streamed response may never be buffered for the output check, and tool-call arguments, retrieved documents, and the system prompt may be outside the check's remit entirely. On those surfaces the guardrail is not wrong about the attack; it never sees it. Confirm which surfaces yours inspects before you count it as a control — an enabled toggle is a configuration state, not evidence of coverage. The second limit is evasion: even on a surface it does inspect, a determined attacker can craft inputs that slip past a classifier. Guardrails are one layer of defence in depth, paired with least-privilege tools; see prompt injection for the attack and why no single control fully closes it.

Do guardrails work with streaming responses?

Often not, and it is the coverage gap worth checking first. Many response-side guardrails need the complete response before they can classify it, so a reply streamed token by token can bypass the output check entirely, depending on the implementation. Cloudflare AI Gateway is the case measured on this site's own deployment: with Guardrails enabled, the dashboard reported the feature as on while the response-side check had never inspected a single streamed reply a user saw. Across four live gateway calls the first token arrived 1ms after the request against 2032ms of total generation — only possible if nothing held the response back to evaluate it. Cloudflare's own documentation is unambiguous that streaming is not supported with Guardrails, so we treat the response side of that path as uncovered. The prompt-side check does still fire before generation — we confirmed that half on paired streaming and non-streaming calls rather than assuming it. The general test is latency: if the response check adds no delay before the first token, it is not inspecting the response. The mitigation is a first-party incremental scanner over the stream, which must hold back a tail of its buffer so a match split across two chunks is not missed.

What are the best guardrail frameworks?

There is no single best — they converge on the same controls. Open-source options include Guardrails AI (Python validators), NVIDIA NeMo Guardrails (programmable rails in Colang), and Meta's Llama Guard (a classifier model). Managed services include Amazon Bedrock Guardrails and Azure AI Content Safety. Choose by where your stack already lives and whether you want a self-hosted library or a managed service, not by feature lists that mostly overlap.

What are the downsides of guardrails?

Four main ones: latency (model-based guardrails are extra inference calls on every turn), false positives (too strict and they block legitimate requests; too loose and they miss real ones — a threshold you must tune with evals), limited scope (a guardrail only evaluates the surfaces it actually inspects, and the ones it does not are invisible in the dashboard), and a false sense of completeness (no guardrail is a wall — they lower probability and blast radius, not to zero). They are a layer of risk reduction, not a guarantee.