Install the AI Gateway Operator for LiteLLM

This guide walks you through installing the LiteLLM AI Gateway Operator on your Kubernetes cluster.

Dependencies

The LiteLLM AI Gateway requires the following dependencies:

  • Kubernetes: A running Kubernetes cluster

  • Agent Runtime Operator: Includes the required CRDs for AI Gateways. See Install the Agent Runtime Operator for installation instructions.

Installation

Install with Kubernetes YAML

Install the LiteLLM AI Gateway

# Install the operator
kubectl apply -f https://github.com/agentic-layer/ai-gateway-litellm-operator/releases/download/v0.2.0/install.yaml
# Wait for the operator to be ready
kubectl wait --for=condition=Available --timeout=60s -n ai-gateway-litellm-system deployment/ai-gateway-litellm-controller-manager

Find the latest version at https://github.com/agentic-layer/ai-gateway-litellm-operator/releases

Install with Flux

If you’re using Flux for GitOps, you can install the operator using an OCI repository:

apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
  name: ai-gateway-litellm-operator
  namespace: flux-system
spec:
  interval: 5m
  url: oci://ghcr.io/agentic-layer/manifests/ai-gateway-litellm-operator
  ref:
    semver: ">= 0, < 1"
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: ai-gateway-litellm-operator
  namespace: flux-system
spec:
  sourceRef:
    kind: OCIRepository
    name: ai-gateway-litellm-operator
  interval: 10m
  path: ./default
  prune: true
  wait: true
  dependsOn:
    # Ensure cert-manager is installed first (if you install it with Flux as well)
    - name: cert-manager

Verify the installation

# Check that the operator is running
kubectl get pods -n ai-gateway-litellm-operator-system

You should see the operator pod running.