Measure representative requests, separate billed components, model normal and peak volumes, and include review, support, and maintenance. Example rates in this guide are hypothetical.
1. Choose a useful unit of work
Define the event you want to price: a reviewed support reply, a processed invoice, a completed search, or a minute of conversation. One customer request can trigger several model calls and tool operations, so “cost per API request” may not match the product’s real unit of work.
Collect representative traces from a pilot. Record the inputs, generated output, tool calls, retrieved material, and retries for each completed unit. Separate accepted results from failed or abandoned work. A cheap response that always needs rewriting can have a higher business cost than a more useful alternative.
2. Calculate the basic model component
For a model billed by input and output tokens, multiply the measured token volumes by the applicable rates. Keep input and output separate. Some services price cached input, reasoning, tools, or other usage differently, so confirm the current billing categories for your selected account and model.
This standard-library Python example uses invented rates solely to demonstrate arithmetic. It is not a quotation or current price for OpenAI, Claude, Gemini, or any provider. It assumes one call per request and excludes the other costs discussed below.
from decimal import Decimal
requests = Decimal("50000")
input_tokens = Decimal("1200")
output_tokens = Decimal("300")
input_rate_per_million = Decimal("0.80") # invented example rate
output_rate_per_million = Decimal("3.20") # invented example rate
estimated = requests * (
input_tokens * input_rate_per_million
+ output_tokens * output_rate_per_million
) / Decimal("1000000")
print(f"Hypothetical model cost: {estimated:.2f} currency units")3. Add the components around the model
A knowledge assistant may also use document extraction, embeddings, storage, search, reindexing, and application hosting. A voice product adds audio processing and sometimes telephony. An agent may pay for several reasoning turns and tool requests before finishing one task.
Include observability, evaluation jobs, support, and the time people spend reviewing exceptions. Keep one-time development and migration costs separate from recurring operation. Identify which parts scale with request volume and which are baseline infrastructure or staffing costs. This makes growth scenarios easier to understand.
4. Plan normal, peak, and failure scenarios
Create a typical month, a busier month, and a scenario with longer documents or more retries. Use measured distributions when possible; a single average can hide a small group of very expensive requests. Include concurrency and provider limits, since queuing can affect completion rates.
Set operating limits that match the product contract: maximum document size, allowed tool turns, output size, timeouts, and usage alerts. Decide what the user sees when a limit is reached. Treat uncontrolled retries as both a cost and reliability problem, particularly when an action might already have completed.
5. Optimize cost while keeping the acceptance bar
Reduce irrelevant context, improve retrieval, and use suitable output limits before changing a model only for its advertised price. Cache appropriate stable results where your permissions and freshness requirements allow it. Consider smaller models for bounded tasks only after testing their behavior.
Track cost per accepted result together with correction effort, response time, and failure rates. Rerun quality checks after an optimization. The goal is an affordable product that still does the agreed work; a lower bill with more unsupported answers is not a successful improvement.
6. Frequently asked questions
Are the example rates current provider prices?
No. They are deliberately hypothetical arithmetic inputs. Replace them with the billing categories and rates for the provider, account, model, region, and features you actually choose.
Does RAG always make an AI product cheaper?
No. It can reduce irrelevant context or improve usefulness, but it adds ingestion, search, storage, and maintenance. Measure the complete cost and quality of the design.
What should we monitor after launch?
Track usage per completed task, provider spend, infrastructure, failures, retries, and human review effort. Set alerts and assign someone who can investigate unusual changes.
7. Related articles and next steps
Bring representative inputs, expected outputs, and the systems involved. Discuss your requirements with OdooMatrix.