Catch expensive AI model usage in every pull request
The PromptUnit GitHub Action scans your PR diff for expensive models like GPT-4o and Claude Opus, then posts routing savings estimates before the code merges.
When a developer adds a new gpt-4o call, nobody gets a notification. The cost lands silently in the monthly invoice. By then the pattern is set, the feature is shipped, and changing it means a refactor discussion nobody wants to have. The hidden cost of defaulting to GPT-4o compounds faster than most teams realize.
The PromptUnit AI Cost Analyzer posts a comment the moment a PR introduces expensive AI model usage, before the code merges.
What it detects
The action scans the added lines in every PR diff for references to:
- GPT-4o, GPT-4 Turbo, o1, o3
- Claude Opus
- Gemini 2.5 Pro
If it finds any of those models and PromptUnit is not already integrated, it posts a cost comparison comment on the PR. If PromptUnit is already in the diff, it stays quiet.
Setup
Create .github/workflows/ai-cost-analyzer.yml in your repository:
name: AI Cost Analyzer
on:
pull_request:
types: [opened, synchronize]
jobs:
analyze:
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- uses: promptunit/sdk@main
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
That is the entire configuration. No API keys, no build step, no npm install. The action uses only Node.js built-in modules and the standard GITHUB_TOKEN that GitHub provides automatically to every workflow.
What the PR comment looks like
When expensive models are detected, the action posts a table showing what PromptUnit routing would cost compared to routing everything through the flagship model:
| Without routing | With PromptUnit | |
|---|---|---|
| Simple tasks (classification, extraction, summarization) | Full model price | Up to 94% cheaper |
| Complex tasks (reasoning, code gen) | Full model price | Unchanged |
It also shows the two settings that connect PromptUnit, the base URL and the PromptUnit key:
// Before
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })
// After
const openai = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
baseURL: "https://www.promptunit.ai/api/proxy/openai",
defaultHeaders: { "x-promptunit-key": process.env.PROMPTUNIT_API_KEY },
})
If the developer pushes another commit, the action updates the existing comment rather than posting a duplicate.
Paying full price on every request?
PromptUnit checks each request your app sends and moves it to a cheaper model when one is just as good for that request. You change two settings, and your own provider key stays. Free until it has saved you money, then 20% of what it saves.
Try it free →Why at PR time
Code review is the cheapest moment to change a routing decision. Once the pattern is in production and copied across the codebase, it requires a deliberate effort to change. A PR comment costs nothing and takes ten seconds to read.
The action does not block merges, does not fail CI, and does not require any configuration on the developer's side. It is purely informational.
No build step
Most GitHub Actions that post useful PR comments depend on @actions/core and @actions/github, which require npm install at runtime or a pre-built dist file. The PromptUnit action uses only the Node.js built-in https module. There is nothing to install, nothing to cache, and no ncc build pipeline to maintain.
The action source is at github.com/promptunit/sdk under github-action/index.js.
After you add PromptUnit
If a developer adds PromptUnit after seeing the PR comment, savings start with the first request. Each request is checked, and a cheaper model answers only when it is just as good for that request. The dashboard shows every request with the model that answered, what it cost and what it saved.
For teams already spending $3K or more per month on AI APIs, a 40-70% reduction is our estimate for a workload that is mostly simple requests. It is not a guarantee, and real savings depend on your task mix (see what we measured and OpenAI API cost too high? Six fixes). The action puts that number in front of the right person at the right time.
For the complete routing strategy the action supports, see LLM Model Routing: The Complete Guide. For the hidden cost problem the action surfaces, see The Hidden Cost of Defaulting to GPT-4o. For LLM cost tracking beyond PR-time detection, see LLM Cost Tracking: The Complete Guide.
Try It Free
See exactly where your AI budget is going. PromptUnit shows every request with the model that answered, what it cost and what it saved, starting with your first one.
Start free, no credit card needed. Or talk to us if you want a walkthrough.