Create and Use Guardrails in Gateways

This guide walks you through configuring guardrails for traffic flowing through an Agentic Layer gateway using a Presidio-based PII guard on an AiGateway as the working example.

Before following this guide, make sure you have the Agent Runtime Operator and a gateway operator that supports guardrails installed. Today, guardrails are implemented by the LiteLLM Gateway Operator (Presidio-based PII guardrails on an AiGateway) and the Tool Gateway Agentgateway Operator (Presidio-based PII guardrails on a ToolGateway).

Quick Start: PII Guardrail on an AiGateway

This example masks common PII entities (names, phone numbers, email addresses, credit cards) in every request sent through an AiGateway.

Prerequisites

  • A Presidio Analyzer service reachable from the cluster. For local experimentation you can use the Microsoft Presidio Helm chart or the presidio manifests in the agentic-layer/presidio repository.

  • An installed Agent Runtime Operator and AI Gateway LiteLLM Operator.

Step 1: Create a GuardrailProvider

The GuardrailProvider points at the Presidio backend. It is referenced by one or more Guard resources.

apiVersion: runtime.agentic-layer.ai/v1alpha1
kind: GuardrailProvider
metadata:
  name: presidio
  namespace: guardrail-providers
spec:
  type: presidio-api
  presidio:
    baseUrl: "http://presidio.guardrail-providers:80"
The baseUrl must be reachable from the gateway’s namespace. Use a fully qualified DNS name (service.namespace) when the provider and gateway live in different namespaces.

Step 2: Create a Guard

The Guard resource binds a provider to a concrete policy: when it runs (mode), which entities it cares about, and what to do with them.

apiVersion: runtime.agentic-layer.ai/v1alpha1
kind: Guard
metadata:
  name: pii-guard
  namespace: guards
spec:
  mode:
    - pre_call
  description: Detect and mask PII in user inputs before they are sent to the LLM.
  providerRef:
    name: presidio
    namespace: guardrail-providers
  presidio:
    language: en
    scoreThresholds:
      ALL: "0.5"
    entityActions:
      PHONE_NUMBER: MASK
      EMAIL_ADDRESS: MASK
      CREDIT_CARD: MASK
      PERSON: MASK

For the full field list see Guardrails Reference.

Step 3: Attach the Guard to an AiGateway

Reference the Guard from AiGateway.spec.guardrails. The LiteLLM operator resolves the reference, fetches the referenced GuardrailProvider, and writes the appropriate guardrail block into the LiteLLM configuration.

apiVersion: runtime.agentic-layer.ai/v1alpha1
kind: AiGateway
metadata:
  name: ai-gateway-pii
  namespace: default
spec:
  aiGatewayClassName: litellm
  aiModels:
    - provider: openai
      name: gpt-3.5-turbo
  guardrails:
    - name: pii-guard
      namespace: guards

Apply the three manifests:

kubectl apply -f guardrailprovider.yaml
kubectl apply -f guard.yaml
kubectl apply -f aigateway.yaml

Step 4: Verify the Guardrail Is Active

Inspect the generated LiteLLM ConfigMap — the operator writes a guardrails section derived from every resolved Guard:

kubectl get configmap ai-gateway-pii-config -n default -o yaml

You should see a guardrails: list whose entries reference the pii-guard Guard name, the Presidio base URL from the GuardrailProvider, and the score thresholds and entity actions from the Guard.

Send a request through the gateway that contains PII, then check the response. PII entities listed in entityActions with action MASK will be replaced with placeholders before the prompt reaches the LLM.

Multiple Guards on a Single Gateway

AiGateway.spec.guardrails is an ordered list, so you can chain multiple concerns — for example PII masking and toxic-language detection. The gateway operator evaluates them in the order they appear.

spec:
  guardrails:
    - name: pii-guard
      namespace: guards
    - name: toxic-language-guard
      namespace: guards

Each referenced Guard can use a different provider and mode. The LiteLLM implementation requires every provider referenced to have type: presidio-api; other provider types are accepted by the CRDs but not yet consumed by the LiteLLM operator.

Cross-Namespace References

The providerRef on a Guard and the ObjectReference entries in spec.guardrails both support explicit namespace fields. This lets you centralize guardrail policy:

  • Put shared GuardrailProvider resources in a guardrail-providers namespace.

  • Put reusable Guard resources in a guards namespace.

  • Reference them from application AiGateway resources in their own namespaces.

The gateway operator is granted the RBAC permissions it needs to read Guard and GuardrailProvider resources across namespaces.

Related