Three habits cause almost every runaway OpenAI bill: no per-request ceiling, no caching, and paying for retries that didn’t need to happen. Fix those three and you’ve fixed 90% of it.

1. Set a hard token ceiling, not a soft one

max_tokens caps the response, not the whole request. If you’re not setting it, a model that starts rambling — which happens more than you’d expect on open-ended prompts — silently burns tokens until it hits its own limit, not yours.

response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    max_tokens=600,
)

Pick a number based on what a correct response actually needs, not a round number that feels safe. If your outputs are typically under 200 tokens, capping at 600 still gives headroom without leaving the door open to a 4000-token runaway.

2. Watch input tokens, not just output

Output tokens cost more per token, but input tokens are usually where the bill actually comes from, because context grows every turn in a chat loop. A conversation that resends the full history on every message costs O(n²) tokens over a session, not O(n). Two fixes:

• Truncate or summarize history past a fixed turn count instead of sending everything back every time.
• Don’t resend static content — system prompts, tool definitions, retrieved documents — as fresh text if it hasn’t changed. That’s exactly what prompt caching is for, and if you’re not using it, you’re paying full price for the same tokens on every request.

3. Stop paying for retries that shouldn’t exist

Two retry patterns quietly double your bill:

• Retrying on rate limits with no backoff. A tight retry loop against a 429 doesn’t just fail again, it can trip a stricter limit and start failing everything. Use exponential backoff, and respect the Retry-After header when present.
• Retrying on malformed output because you didn’t ask for structured output in the first place. If you need JSON back, use response_format={"type": "json_object"} (or a tool/function-call schema) instead of parsing free text and retrying on json.loads failures. This alone eliminates most of the “the model didn’t follow the format” retries.

Track it before you cap it

Set a hard organizational spend limit in the OpenAI dashboard (Settings → Limits) as a backstop, but don’t rely on it as your only control — it’s a circuit breaker for when something is already badly wrong, not a cost management tool. Instead, log token usage per request from the response object:

usage = response.usage
# usage.prompt_tokens, usage.completion_tokens, usage.total_tokens

Pipe that into whatever metrics system you already have, tagged by endpoint or feature. Within a week you’ll see which one feature is 80% of your bill — it’s almost always concentrated, not evenly spread across the product.

The model choice most people skip

gpt-4o-mini (or whatever the current small model is) is dramatically cheaper than the flagship model and is good enough for classification, extraction, and short structured tasks — the bulk of what most apps actually call the API for. Reserve the expensive model for the specific calls where quality genuinely moves the needle, and measure that difference on real outputs before assuming you need it everywhere. A lot of “we need GPT-4o for everything” turns out to be “we never tried mini on the easy 80% of our calls.”

Gotcha

Streaming responses still bill the full completion, not what the user actually read before closing the tab. If your UI lets people bail early, that doesn’t reduce your bill — factor that into max_tokens, not into hoping people stop reading.