Guardrails are the controls that sit around an artificial intelligence (AI) system and decide what it is allowed to receive, say and do. In practice they are software layers, separate from the model itself, that inspect a user's prompt before it reaches a large language model (LLM), inspect the answer before it reaches the user, and, for AI agents, inspect each action before it executes. They exist because training cannot fully fix a language model's behaviour: the model that drafts a product description can be talked into inventing a refund policy or leaking personal data. Guardrails enforce limits from outside.

The road-safety metaphor is used loosely. In advertising it also covers brand controls on AI-written ads and budget caps on buying agents.

How guardrails work

Most guardrail systems check content at up to five points, a structure NVIDIA's open-source NeMo Guardrails toolkit makes explicit in its documentation.

  • Input rails screen the user's message. They look for prompt attacks such as jailbreaks, banned topics and personal data that should not be sent to the model.
  • Dialog rails steer the conversation itself, for example forcing a scripted answer on a medical question.
  • Retrieval rails filter documents pulled from a knowledge base in retrieval-augmented generation (RAG) before the model reads them, a defence against prompt injection hidden in third-party content.
  • Execution rails check what an agent sends to a tool, such as an API call, and what comes back.
  • Output rails screen the model's reply before display, blocking or rewriting harmful, off-topic or ungrounded text.

The checks themselves come in three broad kinds. Deterministic rules are the oldest: keyword lists, regular expressions for card numbers or email addresses, allowlists of permitted domains. Classifier models are now the core of most products. Meta's Llama Guard, described in a paper submitted on December 7, 2023, is a Llama 2 model with 7 billion parameters fine-tuned to label both prompts and responses against a safety risk taxonomy that operators can adjust. A third kind uses one LLM to judge another, which is flexible but slow and itself open to manipulation.

Commercial services package these into configurable policies. Amazon Bedrock Guardrails shows the typical menu. At general availability on April 23, 2024, Amazon Web Services (AWS) offered four policy types: denied topics written in natural language; content filters for hate, insults, sexual content, violence, misconduct and prompt attacks, each with an adjustable strength; word filters holding up to 10,000 phrases of no more than three words; and sensitive information filters that block or redact personal data in inputs and outputs. On July 10, 2024, AWS added a fifth safeguard, the contextual grounding check, which scores each answer for whether it is supported by the source material and whether it is relevant to the question. Its worked example set the grounding threshold at 0.85 and the relevance threshold at 0.5 on a 0-to-1 scale; answers below either threshold are blocked. The same release introduced the ApplyGuardrail API, which lets the filters run on any model, not just those hosted on Bedrock.

Agent frameworks treat guardrails as tripwires. In OpenAI's Agents SDK, a triggered guardrail raises an exception that halts the run. By default input checks run in parallel with the agent to save time, so, according to OpenAI's documentation, the tripwire may fire after tokens have been spent or tools have already run. Tool guardrails can block individual function calls, but not handoffs or hosted, computer and shell tools.

Origin and evolution

Filtering predates generative AI, in spam filters and safe search modes, but the modern sense took shape in 2023, after ChatGPT's November 2022 launch. Shreya Rajpal released Guardrails AI, an open-source Python library for validating LLM output against defined structures and rules, in early 2023. NVIDIA announced NeMo Guardrails on April 25, 2023, introducing Colang, a modelling language for specifying permitted dialogue flows. As of November 2026, the toolkit's GitHub page lists version 0.24.1 as the latest release.

Cloud providers followed. AWS previewed Guardrails for Amazon Bedrock on November 28, 2023, initially in English only and in two US regions, with denied topics and content filters. Meta released Llama Guard in December 2023 as part of its Purple Llama safety project and has since issued newer versions alongside later Llama models. The Open Worldwide Application Security Project (OWASP) Top 10 for LLM Applications, first published in 2023, ranks prompt injection as risk LLM01, a position it kept in the 2025 edition.

The next step moved from screening words to checking facts and actions. AWS made Automated Reasoning checks generally available on August 6, 2025, a feature that tests answers against formal logic rules derived from a policy document; AWS says it delivers "up to 99% accuracy at detecting correct responses from LLMs", a vendor figure without independent audit. Model developers also built guardrails into their own serving stacks. Anthropic described its Constitutional Classifiers, input and output classifiers trained on a written list of permitted and forbidden content, in February 2025.

Why guardrails matter for marketing

Brands deploying chatbots and agents carry the liability when guardrails fail: a Canadian tribunal held Air Canada responsible in February 2024 for a refund policy its website chatbot had invented. As advertisers, they also depend on platforms' guardrails around generated content and placements.

The agent wave has made guardrails a product category in ad tech. The PPC Land explainer on agentic AI put it bluntly in August 2026: guardrails are the current product cycle. On August 5, 2026, PubMatic added a five-step governance layer to its AgenticOS platform, combining platform-wide constraints with buyer-set budget thresholds, inventory allowlists, pre-approved asset libraries, approval workflows in which a human authenticates with a one-time passcode, and audit trails with drift detection. The checks run as an independent system check on traffic using the Ad Context Protocol, regardless of which agent sends it. Six days later, Fluency said it withholds execution authority from LLMs across about $3 billion in annual managed spend, leaving live budget changes to deterministic software that follows advertiser rules.

Brand safety and spending senses

Two narrower uses travel under the same word. In brand safety, guardrails are the instructions that keep AI-written ads on message. On October 28, 2026, Microsoft Advertising opened a pilot allowing up to 40 messaging guidelines for generated copy in Performance Max and AI Max search campaigns, with example rules such as "Never imply that products are low cost". Microsoft said they "help reduce the risk", not that they guarantee compliance.

In spending, guardrails are hard limits on what an agent may commit. Walmart's Agent Profiles, announced on October 21, 2026, let shoppers require approval for every purchase or set a limit such as $50, $100 or $200, without saying whether a limit applies per order, day or month. Olyzon's CTV agents move budgets across four demand-side platforms (DSPs) "inside caps and approval thresholds the buyer controls". Such deterministic limits are easier to trust than a classifier, but only as good as the thresholds people set.

Limitations and disputes

Guardrails are probabilistic filters guarding a probabilistic system, and they can be evaded. A study by William Hackett, Peter Garraghan and colleagues at Lancaster University and Mindgard, first submitted in April 2025, tested six prompt injection and jailbreak detectors, including Microsoft's Azure Prompt Shield and Meta's Prompt Guard. Using invisible characters, emoji and other character-level tricks, the authors reported evasion success of up to 100% in some cases. Simpler bypasses work too. A December 2024 lawsuit against Character.ai alleged its filters could be circumvented by regenerating responses until one slipped through, and users of xAI's Grok bypassed image safeguards with descriptive phrasing from December 25, 2025.

The opposite failure is over-blocking. Strict filters refuse legitimate requests, and every added check costs latency and money. Vendor performance claims are hard to compare: AWS's statement that Bedrock Guardrails block "as much as 85% more harmful content" than some models' native protection rests on its own testing against an unnamed baseline.

There is also a structural argument that output filters cannot solve the agent problem. A SUSTech preprint posted on October 30, 2026 argues that drift from an agent's goal can be impossible to certify from the system's own records, so a drift detector reading only an agent's logs, like the one PubMatic describes, cannot resolve every case. Its figures are illustrative and not peer reviewed.

The word also often names a promise rather than a mechanism: announcements rarely say which actions need human sign-off or what a filter's miss rate is.

Not the same as

  • Alignment shapes the model's own behaviour through training, such as reinforcement learning from human feedback. Guardrails act outside the model at run time and can be swapped without retraining.
  • Red teaming is adversarial testing that finds where guardrails fail. It measures; guardrails enforce.
  • Content moderation reviews what humans post on a platform, often after publication. AI guardrails screen machine input and output in real time, though the classifiers increasingly overlap.
  • Brand suitability controls decide where ads appear. Brand guardrails on AI copy decide what generated ads say.

Recent developments

Regulators are now asking for guardrails by name. On October 8, 2026, the UK Information Commissioner's Office opened a call for evidence on AI agents, closing on November 20, 2026, which says autonomy should be a deliberate design decision supported by "guardrails, permissions and approval thresholds" and lists least-privilege access and human approval for high-risk actions among emerging controls. On October 5, 2026, OpenAI said its ChatGPT ad placement guardrails assess "whether a conversation is appropriate for advertising at all", and opened brand suitability pilots with DoubleVerify and IAS in which the partners test those safeguards without seeing user conversations. As of November 2026, outside firms can evaluate those guardrails but not watch them on live traffic.

Timeline

  • November 30, 2022 - OpenAI launches ChatGPT, bringing LLMs into consumer products.
  • Early 2023 - Shreya Rajpal releases the open-source Guardrails AI library.
  • April 25, 2023 - NVIDIA announces NeMo Guardrails and the Colang language.
  • 2023 - OWASP publishes its first Top 10 for LLM Applications, with prompt injection as LLM01.
  • November 28, 2023 - AWS previews Guardrails for Amazon Bedrock.
  • December 7, 2023 - Meta researchers publish the Llama Guard paper.
  • February 2024 - A Canadian tribunal holds Air Canada liable for its chatbot's invented policy.
  • April 23, 2024 - Bedrock Guardrails reaches general availability with four policy types.
  • July 10, 2024 - AWS adds contextual grounding checks and the ApplyGuardrail API.
  • February 2025 - Anthropic describes Constitutional Classifiers.
  • April 15, 2025 - Hackett and colleagues report up to 100% evasion of guardrail detectors.
  • August 6, 2025 - AWS makes Automated Reasoning checks generally available.
  • December 25, 2025 - Grok safeguard bypasses begin producing sexualised images of minors.
  • August 5, 2026 - PubMatic launches a five-step guardrail layer for AgenticOS.
  • August 11, 2026 - Fluency restricts live budget actions to deterministic agents.
  • October 5, 2026 - OpenAI announces brand suitability pilots with DoubleVerify and IAS.
  • October 8, 2026 - UK ICO opens its AI agent call for evidence.
  • October 21, 2026 - Walmart announces spending limits for shopping agents.
  • October 28, 2026 - Microsoft Advertising pilots up to 40 messaging guidelines for AI-written ads.
  • November 20, 2026 - ICO call for evidence closes.

Summary

Who: Model developers such as Meta, Anthropic and OpenAI, cloud providers such as AWS and NVIDIA, open-source projects such as Guardrails AI, ad tech firms such as PubMatic and Fluency, and the brands and agencies that configure them.

What: Guardrails are controls outside an AI model that screen inputs, outputs and agent actions against defined rules, using keyword lists, classifier models, grounding checks and hard limits such as budget caps and approval steps.

When: The modern sense emerged in 2023, with NeMo Guardrails in April, Bedrock Guardrails in preview in November and Llama Guard in December. Agent-focused guardrails became an ad tech product category in 2026.

Where: Between users and models in chatbots and search, inside cloud AI services, in agent frameworks, and in ad buying platforms where agents spend money or generate ad copy.

Why: Training cannot guarantee an LLM's behaviour, so operators add external checks to limit harmful output, data leaks and unauthorised actions, while accepting that those checks can be bypassed or can block legitimate use.