Documentation

#Getting Started

#Quick Start

Integrate PromptUnit in two steps. No new SDKs, no refactoring. just change your base URL and add two headers.

from openai import OpenAI

client = OpenAI(
    api_key="your-openai-key",
    base_url="https://www.promptunit.ai/api/proxy/openai",
    default_headers={
        "x-promptunit-key": "pu_live_xxxxxxxxxxxx",
        "x-promptunit-feature": "customer-support",
    }
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
)

import OpenAI from "openai"

const openai = new OpenAI({
  apiKey: "your-openai-key",
  baseURL: "https://www.promptunit.ai/api/proxy/openai",
  defaultHeaders: {
    "x-promptunit-key": "pu_live_xxxxxxxxxxxx",
    "x-promptunit-feature": "customer-support",
  },
})

const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Hello" }],
})

#What happens after you connect

The learning window, the nightly adjustments and the optional side-by-side checks: How your savings grow over time.

#Authentication

Two headers authenticate your requests. The PromptUnit key identifies your account; the provider key is forwarded to the upstream model provider.

* Provider keys can be configured once in the PromptUnit dashboard instead of passing them per-request. Per-request keys always take precedence.

#Feature Tagging

The x-promptunit-feature header tags each request with the product feature that made the call. This powers per-feature cost breakdowns in your dashboard. the most valuable insight for understanding where your AI spend actually goes.

Pick descriptive, consistent kebab-case names:

You can pass any string up to 64 characters. Tags appear immediately in the dashboard after the first tagged request is received.

#Supported Providers

Your app connects through one of six: OpenAI, Anthropic, Google, Groq, DeepSeek or OpenRouter (your own OpenRouter key, which reaches hundreds of models). Mistral, Together AI, Perplexity, xAI and Cohere are extra options: add their keys and PromptUnit can switch a request to one of their models, but your app can't send its requests to them through PromptUnit directly yet. Add provider keys in the dashboard under API Keys.

CPR (Cross-Provider Routing) evaluates all connected providers on every call and routes to the globally cheapest model that clears your quality threshold. You get routing across your entire provider stack with zero extra code.

#OpenAI Integration

#Base URL

Replace the OpenAI base URL with the PromptUnit proxy endpoint. Every other part of your SDK usage stays identical.

#Anthropic Integration

#Base URL

Point your Anthropic SDK at the PromptUnit proxy endpoint to get routing, prompt caching optimization, and cost tracking.

#Response Headers

Every proxied response is augmented with x-promptunit-* headers that expose routing decisions, costs, quality scores, and optimization results. Your application can read these headers directly, or they are automatically ingested into your dashboard.

#Inferio™ Engine

Inferio™ is the inference optimization engine that processes every request before it reaches the upstream model provider. It operates as a transparent pipeline: your request goes in, an optimized request goes out to the best-fit model, and the original response format is returned to your application unchanged.

#How Routing Works

For each request, Inferio™ computes a score across 10 dimensions:

The score vector is matched against a continuously-updated capability matrix of available models. The cheapest model whose capability profile meets your configured quality threshold receives the request. If no cheaper model qualifies, the originally requested model is used as a passthrough.

#The 27 Layers

Inferio™ runs each request through up to 27 processing layers. Not all layers fire on every request. The pipeline is conditional based on request characteristics and your configuration. Layers 23-27 are the advanced intelligence algorithms that compound in value as traffic grows.

Layer numbers reflect internal pipeline position. Non-sequential numbers indicate reserved slots for layers in private beta or upcoming release.

#Logging Modes

PromptUnit stores per-request metadata to power your dashboard analytics. You control how much metadata is retained. Switch modes any time from your dashboard settings. changes take effect on the next request.

Routing is unaffected by logging mode. Classification and model selection happen in memory during the request. Logging mode only controls what gets written to the database after the response is returned.

In Privacy mode, the Spend by Feature breakdown in your dashboard will show Unknown for feature names and the task type column in your logs will be empty. Cost and savings data remain fully accurate.

#GitHub Action

The PromptUnit AI Cost Analyzer is a GitHub Action that scans every pull request for expensive AI model usage and posts a comment with routing savings estimates. It runs automatically on PR open and sync with no build step required.

Add to your repo

Create .github/workflows/ai-cost-analyzer.yml in your repository:

name: AI Cost Analyzer

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  analyze:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
    steps:
      - uses: promptunit/sdk@main
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}

That's the entire setup. The action uses only the standard GITHUB_TOKEN, no extra secrets required.

The action is self-contained and uses only Node.js built-in modules. It detects usage of GPT-4o, GPT-4 Turbo, o1, o3, Claude Opus, and Gemini 2.5 Pro in PR diffs. If any expensive model is found and PromptUnit is not already integrated, it posts a comment showing which calls could cost less and the two settings (base URL and PromptUnit key) that connect PromptUnit.

Source and full documentation are available at github.com/promptunit/sdk.

#Limits & Billing

PromptUnit charges only on verified savings: the measurable difference between what you would have paid and what you actually paid through optimized routing.