Where the money goes
An AI chatbot has four cost lines:
- Model usage, billed per token of input and output.
- Retrieval, if you use RAG: embeddings, a vector database and its storage.
- Hosting for your bot code, database and queues.
- Platform fees, where they exist. Telegram, Discord and Slack APIs are free to call; the WhatsApp Business Platform charges per message according to Meta's pricing.
For most bots, model usage dominates. That is where to look first.
The token formula
Each request to a chat model sends the whole context again. A typical request contains:
- the system prompt,
- tool definitions (function schemas), if the bot can call tools,
- the conversation history you include,
- retrieved documents, for RAG,
- the user's new message,
and the model returns output tokens. So:
cost per request = input tokens x input price + output tokens x output price
monthly cost = cost per request x requests per day x 30
Input and output are usually priced differently, with output costing more per token. Check your provider's pricing page for current numbers.
A worked example (bring your own prices)
Suppose a support bot sends a 600-token system prompt, 400 tokens of tool schemas, the last 6 turns of history (about 900 tokens), 1,200 tokens of retrieved passages and an 80-token question, and gets a 250-token answer.
- Input: 600 + 400 + 900 + 1,200 + 80 = 3,180 tokens
- Output: 250 tokens
Multiply by your model's prices and your expected daily requests. Our token and cost estimator does this arithmetic with editable prices and shows which part of the request dominates.
Notice where the tokens are: the user's question is under 3% of the input. The fixed parts (system prompt, schemas, retrieved text, history) are what you pay for on every turn.
The levers that cut the bill
- Shorter system prompts. Remove repetition and examples the model does not need.
- Fewer tools per request. Send only the tool definitions relevant to the current step.
- Bounded history. Keep a sliding window, or summarize older turns.
- Fewer, better chunks. Better retrieval means you can send three passages instead of ten.
- Cap output length. Set a maximum and ask for concise answers.
- Prompt caching. Major providers offer discounted pricing for repeated prompt prefixes. Put stable content (system prompt, tools) first so it can be cached. OpenAI applies caching automatically to long prompts; Anthropic lets you mark cacheable blocks.
- Batch processing. Work that does not need an instant answer (classifying tickets, summarizing logs) can go through batch APIs, which OpenAI and Anthropic offer at a discount compared with real-time requests.
- Route by difficulty. A rules-based intent router or a small model can handle greetings, FAQs and commands; send only hard questions to a large model.
Hosting and the rest
The bot itself is usually cheap to host: a webhook bot on a serverless platform may fit in a free tier (see how to keep a bot online for free). Budget separately for a vector database if you use RAG, for logs and monitoring, and for WhatsApp per-message fees if that is your channel.
Protect the budget
- Set spending limits and alerts in your model provider's dashboard.
- Rate limit per user so one person (or one script) cannot run up your bill.
- Cap tool loops in agents: a maximum number of steps per request.
- Log tokens per request so you can see which feature costs what.
Sources
- OpenAI pricing: openai.com/api/pricing and prompt caching: platform.openai.com/docs/guides/prompt-caching
- OpenAI Batch API: platform.openai.com/docs/guides/batch
- Anthropic pricing: anthropic.com/pricing and prompt caching: docs.anthropic.com/en/docs/build-with-claude/prompt-caching
- Anthropic Message Batches: docs.anthropic.com/en/docs/build-with-claude/batch-processing
- WhatsApp pricing: developers.facebook.com/docs/whatsapp/pricing