Per-Customer LLM Billing: Key per Tenant, Caps, Real Cost

The short answer: the simplest reliable way to bill each customer of a multi-tenant AI product for model usage is to give every tenant (or tenant tier) its own API key. Set a USD spend cap and a concurrency cap on that key, and record the exact charge the gateway returns for each request instead of estimating it from token counts. On HopBase, each billed non-streaming response carries X-HopBase-Cost-USD and usage.cost_usd. Each finished video or image task reports usage.cost_usd, and GET /v1/usage returns what a key has left to spend. When a tenant's cap runs out, its requests get 402 insufficient_quota while other tenants keep working. A key over its concurrency cap gets 429 apikey_concurrency_limit, so one noisy customer cannot exhaust your capacity. Streaming responses carry no cost header; reconcile them from the per-key CSV export, where every row can be recomputed.

The problem: one provider bill, many customers

If you run an AI agent platform, a vertical SaaS with AI features, or a white-label AI service, your model provider sends you one bill. Your customers each expect their own. Between those two sits a set of questions every team ends up answering:

• How much did tenant A cost us this month, by model?
• How do we stop one tenant's runaway agent loop from spending everyone's budget?
• How do we show a customer their remaining credit in our own dashboard?
• When finance asks, can we prove each line of an invoice?

Token counts alone don't answer these. Prices differ by model, cached input is cheaper, some models bill reasoning tokens as output, some charge more above a context threshold, and video bills per second or per video token. If you multiply tokens by a price table yourself, that table drifts out of date with every model launch.

Three architectures, compared

Approach	How tenants are separated	Where the cost number comes from	Enforcement
Tag requests, keep your own ledger	A tenant ID in your app; one shared provider key	Your own price table × token counts	Your code, before each call
Self-hosted proxy	Virtual keys or a customer ID field in the proxy	The proxy's price table	The proxy
Key per tenant on a gateway that reports actual charges	One gateway key per tenant or tier	The gateway's charge for each request	The gateway: spend cap and concurrency cap per key

All three work. The rest of this guide covers the third, because it removes the two hardest parts: keeping a price table correct and enforcing limits before a request goes out.

Step 1: one key per tenant, with caps

In the HopBase console, open API Keys and create a key for each tenant. Keys are created and edited in the console rather than through a public API, so make key creation a step in your tenant onboarding. Three settings per key do most of the work:

• Groups decide which models the key can call. A key includes 1 to 10 groups across families, so a "Basic" tenant can get only DeepSeek and Qwen while a "Pro" tenant also gets Claude and video. A model outside the key's groups returns 404 model_not_found.
• Quota is a USD spend cap for the key. When it is used up, the key's requests return 402 insufficient_quota with "API key quota exhausted". Other keys on the account are not affected.
• Concurrency is the number of requests that can be in flight on this key. Over the cap, requests get 429 apikey_concurrency_limit with Retry-After: 1. Leave it empty for no key-level cap. The account cap still applies on top: the default is 5 requests in flight per account, and it can be raised on request.

Store the key encrypted in your tenant table and never expose it to the tenant's browser. Your backend makes the model call on the tenant's behalf.

Step 2: record the actual charge on every request

For synchronous calls (chat, images, speech), a billed response returns the amount deducted from your balance:

• Header X-HopBase-Cost-USD, for example 0.012.
• In non-streaming JSON, the same value as usage.cost_usd, next to the normal token counts.

A thin wrapper turns this into a ledger entry:

import os, sqlite3
from openai import OpenAI

db = sqlite3.connect("ledger.db")
db.execute("""CREATE TABLE IF NOT EXISTS usage_ledger (
  ts TEXT DEFAULT CURRENT_TIMESTAMP, tenant_id TEXT, model TEXT,
  prompt_tokens INT, completion_tokens INT, cost_usd REAL, request_id TEXT)""")

def client_for(tenant):
    # tenant["hopbase_key"] is that tenant's own key, decrypted server-side
    return OpenAI(base_url="https://api.hop-base.com/v1", api_key=tenant["hopbase_key"])

def chat(tenant, model, messages, **kw):
    raw = client_for(tenant).chat.completions.with_raw_response.create(
        model=model, messages=messages, **kw)
    resp = raw.parse()
    cost = float(raw.headers.get("X-HopBase-Cost-USD") or 0)
    db.execute("INSERT INTO usage_ledger (tenant_id, model, prompt_tokens, completion_tokens, cost_usd, request_id) VALUES (?,?,?,?,?,?)",
               (tenant["id"], model, resp.usage.prompt_tokens,
                resp.usage.completion_tokens, cost, resp.id))
    db.commit()
    return resp

Some rules about this number that matter for billing:

• Failed requests that produced no output are not billed and carry no cost header.
• If the gateway retries a request internally, only the attempt that completed is billed, and that is the amount reported. A key with several groups is charged at the group that served the request.
• If your client disconnects early, the output already generated is still billed. That charge appears in the console's Usage records even though you never saw the header.
• Streaming (SSE) responses carry no cost header, because headers go out before generation starts, and the stream's final usage event has token counts but no amount. Record the token counts (send stream_options: {"include_usage": true}) as a pending entry, then settle the amount from the CSV export (Step 4).

Step 3: async video and image tasks

Video tasks are submitted with POST /v1/video/generate and polled with GET /v1/video/tasks/{task_id}. When a task reaches a final state, the single-task query includes a top-level usage object:

{
  "id": "vt-1",
  "status": "completed",
  "usage": { "cost_usd": 0.5 }
}

Write usage.cost_usd to the ledger when the task finishes. A failed task that was not charged reports 0. On a single-task query, only the key that created the task sees the field. Task lists (GET /v1/video/tasks without an ID) cover the whole account, including tasks created by other keys, so store each task ID with its tenant and never pass a raw list through to a tenant. Video submissions also reserve their estimated cost up front, so a tenant close to its cap can get 402 at submission rather than partway through.

Step 4: reconcile and invoice

At the end of a billing period, export the console's Usage page as CSV, filtered by key and date range (up to 50,000 rows and 400 days per file). Each row has the time, model, usage, official list-price currency, official cost, discount, exchange rate, charged amount and a note, so every line can be recomputed:

official cost × discount ÷ exchange rate = charged amount

Failed requests are marked as not charged, and interrupted requests are marked as charged for the output already produced. Sum the charged amount per key, compare it with your ledger, and settle the streaming entries from Step 2. The console also charts spending and request trends per key. If you work through an MCP client, the read-only get_usage tool returns request, token and cost totals for the calling key, by model, over the last 7 days by default and up to 92 days.

What you charge the tenant is your decision. A common pattern is cost plus a margin, or a credit pack priced above expected cost. Since the ledger holds real charges, either one is a single multiplication.

Step 5: show tenants their remaining credit

To show a live "credit remaining" figure in your dashboard, call GET /v1/usage with the tenant's key:

curl https://api.hop-base.com/v1/usage \
  -H "Authorization: Bearer sk-tenant-key"

For a key with a $10 quota and $2 used, balance is 8, quota.total is 10 and quota.used is 2, all in USD. If the key has no quota of its own, balance is the account balance, so set a quota on every tenant key if you plan to show this number. A key with quota left still stops with 402 if the account balance runs out first. The call itself is not billed.

Error handling that respects tenant boundaries

• 402 insufficient_quota: this tenant's cap is reached, or your account balance is empty. Check GET /v1/usage to tell which, then show "credit exhausted" to that tenant only.
• 429 apikey_concurrency_limit: this tenant is over its own concurrency cap. Queue or back off for this tenant without slowing others.
• 429 user_concurrency_limit: your whole account is at its cap. That calls for a capacity conversation, not a per-tenant fix.
• 404 model_not_found: the tenant asked for a model outside its plan. Map it to an upgrade prompt.

Checklist before you launch

1. Every tenant key has a quota and a concurrency cap.
2. Ledger writes use the returned charge, never a local price table.
3. Streaming and disconnected requests are settled from the CSV export.
4. Async task charges are written when the task reaches a final state, keyed by the task IDs you stored per tenant.
5. Monthly reconciliation compares ledger totals per key with the export.

FAQ

How do I track LLM cost per customer without maintaining a price table?

Read the charge the gateway returns. On HopBase that is the X-HopBase-Cost-USD header or usage.cost_usd for synchronous calls, and usage.cost_usd on finished async tasks. Each is the exact USD amount deducted for that request.

Can I cap how much one customer can spend?

Yes. Set a quota on that customer's API key. When it is used up, that key's requests return 402 insufficient_quota and other keys keep working.

Why is there no cost header on streaming responses?

Headers are sent before generation begins, so the final charge isn't known yet. Record the token counts from the stream's final usage event as a pending entry and settle the amount from the console Usage records or CSV export.

Can different customers get different models?

Yes. Each key includes only the groups you choose, from 1 to 10, so the plan is enforced by the key itself. A model outside the key's groups returns 404.

Where are the details?

See Account and catalog API for cost fields and /v1/usage, Concurrency, timeouts and billing for caps, and our usage reconciliation guide for the CSV method. List prices are on hop-base.com/pricing.