Skip to content

Proxy Gateway

Deploy OxideShield™ as a transparent HTTP proxy between your application and LLM APIs. The proxy intercepts all traffic, applies security guards, and provides enterprise features like rate limiting, user tracking, and real-time alerts.

License Tiers

Professional (£149/month): Core proxy gateway - ideal for Molt and chat bot deployments. Includes request/response interception, all guards, and basic configuration.

Enterprise (POE): Advanced features - webhook alerts (Slack, Discord), rate limiting, streaming guards, and user tracking with strike system.

Architecture

Proxy Gateway Architecture

Quick Start

# Start proxy with Anthropic upstream
oxideshield proxy \
  --listen 0.0.0.0:8080 \
  --upstream anthropic=https://api.anthropic.com \
  --guards PatternGuard,PIIGuard

# With full configuration file
oxideshield proxy --config proxy.yaml

# Validate configuration
oxideshield proxy --validate --config proxy.yaml

# Print example config
oxideshield proxy --example-config

Configuration Reference

Complete Configuration

# proxy.yaml - Complete reference configuration
proxy:
  # Network settings
  listen: "0.0.0.0:8080"

  # TLS configuration (optional)
  tls:
    enabled: false
    cert_path: "/etc/ssl/certs/oxideshield.crt"
    key_path: "/etc/ssl/private/oxideshield.key"

  # Upstream LLM APIs
  upstreams:
    anthropic:
      url: "https://api.anthropic.com"
      timeout_ms: 60000
      retry_attempts: 2
      retry_delay_ms: 1000
    openai:
      url: "https://api.openai.com"
      timeout_ms: 30000

  # Request routing
  routing:
    - path: "/v1/messages"
      upstream: anthropic
    - path: "/v1/chat/completions"
      upstream: openai
    - path: "/*"
      upstream: anthropic  # Default

  # Security guards
  guards:
    input:
      - name: length
        type: length
        config:
          max_chars: 10000
          max_tokens: 4000
        action: block

      - name: encoding
        type: encoding
        config:
          detect_invisible: true
          detect_homoglyphs: true
          normalize: true
        action: sanitize

      - name: pattern
        type: pattern
        config:
          categories:
            - prompt_injection
            - jailbreak
            - system_prompt_leak
        action: block

      - name: pii
        type: pii
        config:
          redaction: mask
          categories:
            - email
            - phone
            - ssn
            - credit_card
        action: sanitize

    output:
      - name: pii_output
        type: pii
        config:
          redaction: mask
        action: sanitize

      - name: toxicity
        type: toxicity
        config:
          threshold: 0.7
        action: block

  # Guard pipeline settings
  pipeline:
    strategy: fail_fast  # fail_fast, all, or custom
    on_error: block      # block or allow

  # Streaming response handling
  streaming:
    strategy: periodic
    eval_interval_chars: 500
    eval_interval_tokens: 100
    max_eval_interval_ms: 2000
    early_termination: true
    max_buffer_chars: 10000

  # Rate limiting
  rate_limit:
    enabled: true
    requests_per_minute: 60
    requests_per_hour: 1000
    tokens_per_hour: 100000
    burst: 10
    limit_by: ip_address
    whitelist: []
    custom_limits: {}
    response:
      status_code: 429
      retry_after_seconds: 60
      message: "Rate limit exceeded"

  # User tracking
  tracking:
    enabled: true
    max_strikes: 3
    strike_window_seconds: 3600
    block_duration_seconds: 86400
    track_by: ip_address
    strike_actions:
      - Block
    blocklist: []
    allowlist: []

  # Webhook alerts
  alerts:
    destinations: []
    events:
      - block
      - high_severity
    rate_limit_per_minute: 60
    include_request_details: false
    retry:
      max_retries: 3
      initial_backoff_ms: 1000
      max_backoff_ms: 30000

  # Metrics
  metrics:
    enabled: true
    endpoint: "/metrics"

  # Health checks
  health:
    enabled: true
    endpoint: "/health"

Streaming Protection

The proxy handles streaming SSE responses with real-time guard evaluation.

Streaming Strategies

Strategy Description Latency Security
end_only Evaluate only after stream completes Lowest Basic
periodic Evaluate at intervals (chars/time) Low Good
sentence_boundary Evaluate at sentence ends Medium Better
continuous Evaluate every chunk Highest Maximum

Configuration

streaming:
  # Evaluation strategy
  strategy: periodic

  # Trigger evaluation every N characters
  eval_interval_chars: 500

  # Trigger evaluation every N estimated tokens
  eval_interval_tokens: 100

  # Maximum time between evaluations (ms)
  max_eval_interval_ms: 2000

  # Terminate stream immediately on threat detection
  early_termination: true

  # Force evaluation if buffer exceeds this size
  max_buffer_chars: 10000

How Streaming Evaluation Works

Streaming Evaluation Flow

Early Termination Response

When a threat is detected mid-stream, the proxy sends an SSE error event:

event: error
data: {"type":"security_violation","guard":"pii","message":"Potential credential leak detected","request_id":"req-123"}

Rate Limiting

Protect against abuse and control costs with per-client rate limiting.

Configuration

rate_limit:
  enabled: true

  # Request limits
  requests_per_minute: 60
  requests_per_hour: 1000

  # Token limits (estimated from request size)
  tokens_per_hour: 100000

  # Burst allowance for short spikes
  burst: 10

  # What to rate limit by
  limit_by: ip_address  # ip_address, api_key, header, combined

  # Clients that bypass rate limiting
  whitelist:
    - "admin-api-key"
    - "192.168.1.100"

  # Custom limits for specific clients
  custom_limits:
    "premium-api-key":
      requests_per_minute: 200
      requests_per_hour: 5000
      tokens_per_hour: 500000

  # Response configuration
  response:
    status_code: 429
    retry_after_seconds: 60
    message: "Rate limit exceeded. Please slow down."

Rate Limit Keys

Key Description Example
ip_address Client IP address 192.168.1.1
api_key API key from Authorization header sk-abc123
header Custom header value X-Client-ID: client1
combined IP + API key 192.168.1.1:sk-abc123

Rate Limit Response

HTTP/1.1 429 Too Many Requests
Retry-After: 60
Content-Type: application/json

{
  "error": {
    "type": "rate_limit_exceeded",
    "message": "Rate limit exceeded: 60 requests/minute",
    "retry_after": 60
  }
}

Checking Rate Limit Status

curl http://proxy:8080/_oxideshield/rate-limit/my-api-key

Response: 

{
  "requests_minute": 45,
  "requests_minute_limit": 60,
  "requests_hour": 500,
  "requests_hour_limit": 1000,
  "tokens_hour": 75000,
  "tokens_hour_limit": 100000,
  "burst_available": 8
}

User Tracking

Track user behavior and automatically block repeat offenders.

Strike System

Users accumulate strikes for security violations. After reaching the maximum, they're automatically blocked.

tracking:
  enabled: true

  # Maximum strikes before block
  max_strikes: 3

  # Time window for counting strikes (seconds)
  strike_window_seconds: 3600  # 1 hour

  # How long to block after max strikes
  block_duration_seconds: 86400  # 24 hours

  # What to track by
  track_by: ip_address  # ip_address, user_id, channel, channel_user, api_key

  # Actions that count as strikes
  strike_actions:
    - Block

  # Permanently blocked users
  blocklist:
    - "192.168.1.50"
    - "bad-api-key"

  # Users that are never tracked/blocked
  allowlist:
    - "admin-key"
    - "trusted-ip"

Tracking Keys

Key Format Use Case
ip_address 192.168.1.1 Web applications
user_id user_123 Authenticated apps
channel discord Multi-platform bots
channel_user discord:user_123 Chat bots
api_key sk-abc123 API consumers

User Stats API

# Get user statistics
curl http://proxy:8080/_oxideshield/tracking/user_123

Response: 

{
  "key": "user_123",
  "strikes": 2,
  "max_strikes": 3,
  "blocked": false,
  "first_seen": 3600,
  "last_activity": 120,
  "total_requests": 150,
  "blocked_requests": 5,
  "triggered_guards": {
    "pattern": 3,
    "pii": 2
  }
}

Management APIs

# Get all blocked users
curl http://proxy:8080/_oxideshield/tracking/blocked

# Get top offenders
curl http://proxy:8080/_oxideshield/tracking/top-offenders?limit=10

# Manually block a user
curl -X POST http://proxy:8080/_oxideshield/tracking/user_123/block \
  -d '{"duration_seconds": 86400}'

# Unblock a user
curl -X POST http://proxy:8080/_oxideshield/tracking/user_123/unblock

Webhook Alerts

Receive real-time notifications when security events occur.

Supported Destinations

Slack

alerts:
  destinations:
    - type: slack
      webhook_url: "https://hooks.slack.com/services/T00/B00/XXX"
      channel: "#security-alerts"  # Optional override
      username: "OxideShield™"       # Optional
      icon_emoji: ":shield:"        # Optional

Slack alert format: 

🚨 Security Alert
━━━━━━━━━━━━━━━━━
Guard: pattern
Action: Block
Severity: high
Request ID: req-abc123
━━━━━━━━━━━━━━━━━
Reason: Prompt injection detected

Discord

alerts:
  destinations:
    - type: discord
      webhook_url: "https://discord.com/api/webhooks/123/abc"

Discord embeds with color-coded severity: - Critical: Red (#FF0000) - High: Orange (#FF8C00) - Medium: Gold (#FFD700) - Low: Turquoise (#00CED1)

Generic Webhook

alerts:
  destinations:
    - type: webhook
      url: "https://your-siem.example.com/api/events"
      method: POST  # POST or PUT
      headers:
        Authorization: "Bearer ${SIEM_TOKEN}"
        Content-Type: "application/json"

Webhook payload: 

{
  "timestamp": "2024-01-15T10:30:00Z",
  "request_id": "req-abc123",
  "guard": "pattern",
  "action": "Block",
  "reason": "Prompt injection detected",
  "severity": "high",
  "channel": "discord",
  "user_id": "user_123",
  "context": {}
}

Alert Events

Event Description
block Any blocked request
high_severity High severity detections
critical Critical severity detections
pii_detected PII found in request/response
jailbreak Jailbreak attempt detected
rate_limit_exceeded Rate limit hit
all All events (for debugging)

Retry Configuration

alerts:
  retry:
    max_retries: 3
    initial_backoff_ms: 1000    # Start at 1 second
    max_backoff_ms: 30000       # Cap at 30 seconds

Retry uses exponential backoff: 1s -> 2s -> 4s -> ... -> 30s (max)

API Endpoints

Endpoint Method Description
/* * Proxy to upstream
/health GET Health check
/metrics GET Prometheus metrics
/_oxideshield/config GET Current configuration
/_oxideshield/rate-limit/{key} GET Rate limit status
/_oxideshield/tracking/{key} GET User tracking stats
/_oxideshield/tracking/{key}/block POST Block user
/_oxideshield/tracking/{key}/unblock POST Unblock user
/_oxideshield/tracking/blocked GET List blocked users
/_oxideshield/tracking/top-offenders GET Top offenders

Deployment

Docker

# docker-compose.yml
version: '3.8'
services:
  oxideshield:
    image: oxideshield/proxy:latest
    ports:
      - "8080:8080"
    environment:
      - OXIDESHIELD_PROXY_LISTEN=0.0.0.0:8080
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
      - SLACK_WEBHOOK_URL=${SLACK_WEBHOOK_URL}
    volumes:
      - ./proxy.yaml:/etc/oxideshield/config.yaml
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3

docker compose up -d

Kubernetes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: oxideshield-proxy
spec:
  replicas: 3
  selector:
    matchLabels:
      app: oxideshield-proxy
  template:
    metadata:
      labels:
        app: oxideshield-proxy
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "8080"
        prometheus.io/path: "/metrics"
    spec:
      containers:
      - name: proxy
        image: oxideshield/proxy:latest
        ports:
        - containerPort: 8080
        env:
        - name: ANTHROPIC_API_KEY
          valueFrom:
            secretKeyRef:
              name: llm-secrets
              key: anthropic-api-key
        volumeMounts:
        - name: config
          mountPath: /etc/oxideshield
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 30
        readinessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 10
        resources:
          requests:
            memory: "128Mi"
            cpu: "100m"
          limits:
            memory: "512Mi"
            cpu: "500m"
      volumes:
      - name: config
        configMap:
          name: oxideshield-config
---
apiVersion: v1
kind: Service
metadata:
  name: oxideshield-proxy
spec:
  selector:
    app: oxideshield-proxy
  ports:
  - port: 80
    targetPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: oxideshield-config
data:
  config.yaml: |
    proxy:
      listen: "0.0.0.0:8080"
      upstreams:
        anthropic:
          url: "https://api.anthropic.com"
      # ... rest of config

AWS ECS

{
  "family": "oxideshield-proxy",
  "containerDefinitions": [
    {
      "name": "proxy",
      "image": "oxideshield/proxy:latest",
      "portMappings": [
        {
          "containerPort": 8080,
          "protocol": "tcp"
        }
      ],
      "environment": [
        {
          "name": "OXIDESHIELD_PROXY_LISTEN",
          "value": "0.0.0.0:8080"
        }
      ],
      "secrets": [
        {
          "name": "ANTHROPIC_API_KEY",
          "valueFrom": "arn:aws:secretsmanager:..."
        }
      ],
      "healthCheck": {
        "command": ["CMD-SHELL", "curl -f http://localhost:8080/health || exit 1"],
        "interval": 30,
        "timeout": 5,
        "retries": 3
      }
    }
  ]
}

Monitoring

Prometheus Metrics

# Request metrics
oxideshield_requests_total{guard="pattern",action="block"} 150
oxideshield_requests_total{guard="pii",action="sanitize"} 2341

# Latency metrics
oxideshield_guard_duration_seconds_bucket{guard="pattern",le="0.001"} 9500
oxideshield_guard_duration_seconds_bucket{guard="pattern",le="0.01"} 9800

# Rate limiting
oxideshield_rate_limit_exceeded_total{key_type="ip_address"} 45

# User tracking
oxideshield_users_blocked_total 12
oxideshield_strikes_recorded_total{guard="pattern"} 89

# Streaming
oxideshield_streams_terminated_total{reason="pii_detected"} 5
oxideshield_stream_chunks_processed_total 125000

# Alerts
oxideshield_alerts_sent_total{destination="slack",status="success"} 150
oxideshield_alerts_sent_total{destination="slack",status="failure"} 2

Grafana Dashboard

Import the OxideShield™ dashboard from oxideshield.ai/downloads.

Key panels: - Requests per second by guard/action - P50/P95/P99 latency - Rate limit utilization - Blocked users over time - Alert delivery status

Log Output

{
  "timestamp": "2024-01-15T10:30:00Z",
  "level": "info",
  "request_id": "req-abc123",
  "guard": "pattern",
  "action": "block",
  "reason": "Prompt injection detected",
  "duration_ms": 2,
  "user_key": "192.168.1.1"
}

Security Best Practices

1. Enable All Layers

rate_limit:
  enabled: true

tracking:
  enabled: true

alerts:
  destinations:
    - type: slack
      webhook_url: "..."

2. Defense in Depth

guards:
  input:
    - length      # Fast rejection
    - encoding    # Normalization
    - pattern     # Known attacks
    - semantic    # Novel attacks
    - pii         # Data protection

3. Fail Secure

pipeline:
  strategy: fail_fast
  on_error: block

4. Monitor Everything

metrics:
  enabled: true

alerts:
  events:
    - block
    - high_severity

5. Regular Updates

# Update attack patterns
oxideshield update-patterns

Troubleshooting

Proxy Not Starting

# Check port availability
lsof -i :8080

# Run with debug logging
oxideshield proxy --config proxy.yaml --log-level debug

Connection Refused to Upstream

# Test upstream connectivity
curl -I https://api.anthropic.com/v1/messages

# Check TLS certificates
openssl s_client -connect api.anthropic.com:443

High Latency

# Check guard latency
curl http://proxy:8080/metrics | grep duration

# Consider switching to fail_fast
pipeline:
  strategy: fail_fast

False Positives

guards:
  input:
    - name: pattern
      config:
        confidence_threshold: 0.9  # Increase threshold
        exclude_patterns:
          - "legitimate_pattern"

Next Steps