AI Cost Optimization Checklist 2026: 15 Steps to Lower Your API Bill

Teams that reduced their AI API costs by 40-70% in the past year did not do it with a single clever optimization. They worked through a list. Model selection, caching, prompt compression, routing, batching, context pruning. None of these techniques is complicated individually. The savings compound when you apply them in combination, and most teams have only done two or three of them.

This is a 15-point checklist written for engineering and platform teams managing production LLM workloads. The items are ordered roughly by typical savings impact, with the highest-leverage actions first. Each item includes an estimated savings range, though your actual results will depend on your current setup.

1. Audit Which Model You Are Defaulting To

Before anything else, find out what model your application is actually calling in production. Teams regularly discover they are running on a flagship model because it was the default in a tutorial or set during prototyping and never revisited. If you are running GPT-5.4 at $2.50 per million input tokens and $15 per million output tokens on tasks a smaller model handles equally well, the savings from a model change alone are 50-80%. Check your API usage dashboard, match model names to current pricing, and establish your baseline before moving on.

2. A/B Test Cheaper Models on Your Actual Production Tasks

Do not assume the quality gap between models is meaningful for your use case. The only reliable test is running both models on a representative sample of your real production inputs and scoring the outputs using a rubric that matches your actual quality requirements. Take 100 examples from your production logs. Run both the current model and a cheaper candidate. Score them. If the cheaper model performs within 5% of the current model on your rubric, you have a clear routing decision. The measurement takes a day to set up. Skipping it means either missing savings or discovering quality regressions through user complaints. GPT-4o-mini handles 60-70% of production tasks with quality indistinguishable from GPT-4o in A/B tests. The question is whether your specific tasks are in that majority.

3. Enable Provider-Side Prompt Caching

Anthropic offers 90% off cached token reads when prompt caching is enabled on requests with repeated system prompts or context. OpenAI automatically caches the prefix of prompts over a certain length, providing 50% off for cached input tokens. For applications with large, static system prompts, the savings on input costs are significant. A system prompt of 2,000 tokens sent 100,000 times per day costs $30 per day in input tokens at Claude Sonnet 4.6 pricing without caching. With caching, after the first call, those reads cost $3 per day. Estimated savings: 20-60% on input costs for any application with repeated system prompt content. The implementation requires a small change to how you structure requests, and the savings begin immediately.

4. Tag API Calls by Feature

This item does not directly reduce costs, but it enables every other optimization on this list. Without per-feature attribution, you cannot know which tasks to target, which optimizations have the highest impact, or whether your changes are working. Tag every API call with at minimum the feature name and user tier. Store these alongside the token counts from API responses. Build a weekly report sorted by total spend per feature. The first time you run this report, you will almost certainly find that one or two features account for the majority of your AI spend, and at least one of them will be a candidate for immediate optimization. The complete guide to LLM cost attribution by feature covers implementation approaches in detail.

5. Compress System Prompts

System prompts grow over time. Developers add instructions to handle edge cases, include examples to improve output quality, and paste in context that seemed useful during testing. Few people go back to remove content that is no longer needed. Audit your system prompts for redundancy, filler phrases, and instructions that are not actually changing model behavior. A disciplined compression pass typically achieves 30-50% token reduction on system prompts without any quality change. At high call volumes, the savings from prompt compression alone can be substantial. A 1,000-token system prompt compressed to 600 tokens saves 400 tokens on every call, and those savings multiply directly against your call volume and input token rate.

6. Add Output Length Constraints to Prompts

Output tokens cost more than input tokens on every major provider. Claude Sonnet 4.6 charges $3 per million input tokens and $15 per million output tokens. Unconstrained output length means you are paying for however many tokens the model decides to generate, which is often more than you need. Adding explicit length guidance to prompts, such as "respond in under 100 words" or "return a JSON object with exactly these fields," reduces output length without reducing output quality for most structured tasks. Estimated savings: 20-40% on output costs for tasks with unconstrained generation where the output length is not inherently variable.

7. Implement Application-Level Caching for Repeated Identical Queries

If users in your application ask the same questions, and they do, those are opportunities for full response caching at the application layer. A Redis or Memcached store keyed by the hash of the user query and system prompt context can return instant, free responses for repeat queries without any API call. Savings are up to 100% of cost for cached queries. For products with shared corpora, high user overlap, or predictable query patterns, cache hit rates of 20-50% are realistic. The engineering investment is low. The savings begin immediately on deployment.

8. Move Batch Workloads to the Batch API

OpenAI and Anthropic both offer 50% off standard token rates for requests submitted through their Batch APIs. The tradeoff is latency. Batch API requests are processed asynchronously, typically within a few hours rather than in real time. For any workload that is not user-facing and does not require immediate responses, this is a straightforward 50% cost reduction. Document processing, embedding generation, content classification, offline analysis, and evaluation runs are all strong candidates. If you are currently running these synchronously because nobody set up the batch pipeline, the setup cost is a few hours of engineering work. The ongoing savings are immediate. See the full batch API decision framework for a routing guide.

9. Implement Cross-Provider Model Routing

Different providers offer different price-performance points for different task types. Claude Haiku 4.5 at $1 per million input tokens and $5 per million output tokens handles many classification and extraction tasks at lower cost than some OpenAI alternatives. Gemini 3 Flash at $0.25 per million input tokens and $1.50 per million output tokens is competitive for high-volume simple tasks. Routing simple tasks to the cheapest capable model while reserving more capable models for complex requests can reduce blended cost by 40-70% on routed requests. PromptUnit provides routing infrastructure for cross-provider dispatch so teams can implement this without custom routing code. The cross-provider routing guide covers the provider comparison in depth.

10. Prune Conversation Context

Multi-turn applications that send the full conversation history on every API call are paying for context that is often irrelevant to the current user request. A conversation that has been going for ten turns before reaching the current question includes early context, clarifications, and prior outputs that may have no bearing on the next response. Summarize older turns rather than sending them verbatim. Limit context to the most recent N turns plus the original task context. For most conversation types, the most recent 3-5 turns plus the system context is sufficient. Estimated savings: 30-60% on input costs for multi-turn applications. This is one of the highest-impact optimizations for chat-based products.

11. Reduce Few-Shot Examples

System prompts that include multiple few-shot examples are often carrying more examples than necessary. The marginal quality improvement from a fifth example over a second example is typically small, while the token cost is linear. Test 2-shot versus 5-shot on your task. Run the comparison on 50-100 examples and measure output quality. For most classification and extraction tasks, 2-shot achieves 90% or more of the quality improvement available from 5-shot, at 60% lower cost for the example portion of the prompt. The exact savings depend on the length of your examples, but for prompts where examples represent 30-50% of total input tokens, this test is worth running.

12. Use Structured Output Modes

For structured extraction or transformation tasks, JSON mode produces denser outputs than conversational responses. A preamble like "Sure! Here is the information you requested in JSON format: {..." wastes tokens. A clean JSON object with no framing is more useful and cheaper. Structured output modes also reduce malformed response rates, meaning fewer retries and lower total cost per successful call. Estimated savings: 20-40% on output tokens for extraction and transformation tasks.

13. Set Per-Feature and Per-User Cost Budgets in Code

Implement cost guardrails in your application that stop or throttle API calls when a user or feature exceeds a defined spend threshold within a billing period. This prevents cost anomalies from compounding undetected. A bug that causes a loop to generate thousands of API calls, a user who constructs adversarial prompts that produce long outputs, or a spike in traffic that triggers unexpectedly high consumption, all of these can be caught early if you have budget limits in code rather than discovering them in the next billing statement. This is not primarily an optimization. It is cost governance, and it prevents the kind of five-figure surprise bills that have hurt real companies.

14. Monitor Your Retry Rate

High retry rates silently multiply your API costs. If your application retries on rate limit errors, timeout errors, or output validation failures, each retry is an additional API call. A 20% retry rate effectively increases your cost per successful call by 20%, before accounting for the output tokens spent on failed calls that still counted toward your bill. Pull your retry rate from application logs. If it is above 5%, investigate the root cause rather than absorbing the cost. Common causes include prompts that reliably produce outputs failing your validation schema, rate limit misconfiguration, and inconsistent output length causing timeout errors. Fixing the root cause saves costs and improves latency simultaneously.

15. Re-Evaluate Model Choices Quarterly

Model pricing and capabilities change fast. A routing decision that was optimal three months ago may not be optimal today. In 2026, multiple providers have released new model tiers with meaningfully lower pricing at comparable quality levels. A quarterly review of your routing configuration against current model offerings is low effort and can identify opportunities that did not exist when your configuration was last set. Run a quick A/B test on any new model that looks competitive. For a current view of model pricing across providers, see the OpenAI API cost calculator and pricing guide.

The Compounding Effect

These fifteen techniques are additive. A team that applies model selection, prompt caching, system prompt compression, and batch routing to eligible workloads sees the product of several independent multipliers applied to their baseline cost. A 30% reduction from model selection combined with a 40% reduction from caching combined with 50% off batch workloads produces a combined reduction substantially larger than any single item.

The teams that have achieved 60-80% reductions in AI API spend worked through a list like this one and compounded the savings. Most of the work is measurement and configuration, not fundamental engineering changes.

PromptUnit automates prompt caching, prompt compression, cross-provider routing, and context pruning, handling several of the highest-impact items on this list as infrastructure-layer features rather than application code changes.

Start reducing your AI API costs systematically at www.promptunit.ai.