Catch expensive AI model usage in every pull request
The PromptUnit GitHub Action scans your PR diff for expensive models like GPT-4o and Claude Opus, and posts routing savings estimates automatically.
When a developer adds a new gpt-4o call, nobody gets a notification. The cost lands silently in the monthly invoice. By then the pattern is set, the feature is shipped, and changing it means a refactor discussion nobody wants to have. The hidden cost of defaulting to GPT-4o compounds faster than most teams realize.
The PromptUnit AI Cost Analyzer posts a comment the moment a PR introduces expensive AI model usage, before the code merges.
What it detects
The action scans the added lines in every PR diff for references to:
- GPT-4o, GPT-4 Turbo, o1, o3
- Claude Opus
- Gemini 2.5 Pro
If it finds any of those models and PromptUnit is not already integrated, it posts a cost comparison comment on the PR. If PromptUnit is already in the diff, it stays quiet.
Setup
Create .github/workflows/ai-cost-analyzer.yml in your repository:
name: AI Cost Analyzer
on:
pull_request:
types: [opened, synchronize]
jobs:
analyze:
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- uses: promptunit/sdk@main
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
That is the entire configuration. No API keys, no build step, no npm install. The action uses only Node.js built-in modules and the standard GITHUB_TOKEN that GitHub provides automatically to every workflow.
What the PR comment looks like
When expensive models are detected, the action posts a table showing what PromptUnit routing would cost compared to routing everything through the flagship model:
| Without routing | With PromptUnit | |
|---|---|---|
| Simple tasks (classification, extraction, summarization) | Full model price | Up to 94% cheaper |
| Complex tasks (reasoning, code gen) | Full model price | Unchanged |
It also shows the one-line SDK change that enables routing:
// Before
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })
// After
import { createPromptUnit } from "@promptunit/sdk"
const openai = createPromptUnit({
promptunitKey: process.env.PROMPTUNIT_API_KEY,
openaiKey: process.env.OPENAI_API_KEY
})
If the developer pushes another commit, the action updates the existing comment rather than posting a duplicate.
Why at PR time
Code review is the cheapest moment to change a routing decision. Once the pattern is in production and copied across the codebase, it requires a deliberate effort to change. A PR comment costs nothing and takes ten seconds to read.
The action does not block merges, does not fail CI, and does not require any configuration on the developer's side. It is purely informational.
No build step
Most GitHub Actions that post useful PR comments depend on @actions/core and @actions/github, which require npm install at runtime or a pre-built dist file. The PromptUnit action uses only the Node.js built-in https module. There is nothing to install, nothing to cache, and no ncc build pipeline to maintain.
The action source is at github.com/promptunit/sdk under github-action/index.js.
14-day observation period
If a developer adds PromptUnit after seeing the PR comment, routing does not start immediately. PromptUnit runs in shadow mode for 14 days first: it logs every call, classifies each request, and projects your exact savings in the dashboard. No routing until you click Enable.
For teams already spending $3K or more per month on AI APIs, the observation period typically shows 40-70% reduction available, the same range documented in how to reduce your OpenAI API costs by 50-70%. The action puts that number in front of the right person at the right time.
For the complete routing strategy the action supports, see LLM Model Routing: The Complete Guide. For the hidden cost problem the action surfaces, see The Hidden Cost of Defaulting to GPT-4o. For LLM cost tracking beyond PR-time detection, see LLM Cost Tracking: The Complete Guide.
Try It Free
See exactly where your AI budget is going. PromptUnit's 14-day observation period shows you the savings before you commit to anything.
Try the live demo, no API key needed. Or talk to us if you want a walkthrough.