What matters for a bot
- Latency: chat users notice anything over a couple of seconds. Stream tokens or send a typing indicator.
- Tool calling: lets the model call your functions (look up an order, search docs) with typed arguments.
- OpenAI-compatible API: if a provider accepts the same request format, switching is a base URL and model name change.
- Data terms: read whether API inputs are used for training and how long they are kept.
Comparison
| Provider | OpenAI-compatible | Good for | Pricing page |
|---|---|---|---|
| xAI (Grok) | Yes | General chat, reasoning, long context | docs.x.ai |
| OpenAI | Native format | Broad ecosystem, structured outputs | openai.com |
| Anthropic (Claude) | Compatibility layer; native Messages API recommended | Long instructions, careful tool use | anthropic.com |
| Google Gemini | Compatibility endpoint | Lists a free tier for some models; multimodal | ai.google.dev |
| Mistral | Own API, similar shape | European provider, open-weight models | mistral.ai |
| Groq | Yes | Very fast inference of open models | groq.com |
| OpenRouter | Yes | One key, many providers; easy A/B tests | openrouter.ai |
| Local: Ollama / llama.cpp | Yes | Privacy, no per-token cost; needs a machine with enough RAM or a GPU | ollama.com |
Compatibility facts checked on 2026-10-19 in each provider's documentation.
Keep your bot portable
// Same code, different provider: change BASE_URL, API_KEY and MODEL in .env
const res = await fetch(process.env.BASE_URL + "/chat/completions", {
method: "POST",
headers: { Authorization: `Bearer ${process.env.API_KEY}`, "Content-Type": "application/json" },
body: JSON.stringify({
model: process.env.MODEL,
max_tokens: 400, // cap cost per reply
messages: [
{ role: "system", content: "You are a support bot for Example Shop. Answer in the user's language." },
{ role: "user", content: userText },
],
}),
signal: AbortSignal.timeout(20000),
});
if (!res.ok) throw new Error("model error " + res.status);
const reply = (await res.json()).choices[0].message.content;Estimate prompt size and cost with the token estimator and read the AI chatbots guide for memory, tools and prompt-injection defenses.