Bot reliability: rate limits, queues, idempotency and monitoring

Know the limits

Limits are a contract. Staying under them is cheaper than recovering from bans.

Handle 429 correctly

When you receive 429, wait the time the server tells you (Retry-After, retry_after) before retrying that route. Do not retry immediately in a loop: that extends the penalty and can trigger IP-level bans. Libraries such as discord.js and grammY (with its throttler plugin) handle this for you; if you write raw HTTP calls, centralize them in one client that does.

Throttle outgoing messages

A token bucket allows short bursts while enforcing an average rate:

class TokenBucket {
  constructor(ratePerSec, burst) { this.rate = ratePerSec; this.cap = burst; this.tokens = burst; this.last = Date.now(); }
  async take() {
    for (;;) {
      const now = Date.now();
      this.tokens = Math.min(this.cap, this.tokens + ((now - this.last) / 1000) * this.rate);
      this.last = now;
      if (this.tokens >= 1) { this.tokens -= 1; return; }
      await new Promise(r => setTimeout(r, ((1 - this.tokens) / this.rate) * 1000));
    }
  }
}

const global = new TokenBucket(25, 25);          // under Telegram's broadcast limit
const perChat = new Map();                       // chatId -> TokenBucket(1, 1)

async function send(chatId, payload) {
  if (!perChat.has(chatId)) perChat.set(chatId, new TokenBucket(1, 1));
  await global.take(); await perChat.get(chatId).take();
  return telegram("sendMessage", { chat_id: chatId, ...payload });
}

In a multi-instance deployment, keep the bucket in a shared store (Redis, a Durable Object) or route all sends through one worker.

Queues

Put anything slow or bursty on a queue: AI calls, broadcasts, media processing, calls to third-party APIs. Benefits: fast webhook responses, smooth load, automatic retries and a dead-letter queue for messages that keep failing, so one bad item does not block the rest. Make consumers idempotent because queues deliver at least once.

Idempotency and ordering

• Deduplicate incoming events by ID (update_id, Slack event_id, WhatsApp message id, interaction ID).
• For multi-step conversations, store the state transition with a version number and reject stale updates.
• Do not assume ordering across chats; within a chat, process sequentially if order matters.

Timeouts and circuit breakers

Every outgoing call needs a timeout. If a dependency (an AI provider, a database) is failing, stop calling it for a short period (circuit breaker) and reply with a graceful message ("I am having trouble right now, please try again in a few minutes") instead of piling up requests.

Graceful shutdown and deploys

On SIGTERM, stop accepting new work, finish in-flight handlers within a deadline, flush logs and close connections. For Gateway bots, avoid running two instances with the same shard during a deploy. For webhook bots, deploys are seamless if old and new versions can both process events.

Observability

• Structured logs (JSON) with event ID, platform, handler, outcome, latency and a hashed user ID.
• Metrics: events received, handler errors, outgoing API errors by status, 429 count, queue depth and age, AI tokens used.
• Alerts: error rate above baseline, queue age growing, no events received for an unusual period (webhook broken), repeated 401s (token revoked).
• Status command: an admin-only /status that reports version, uptime and queue depth is handy during incidents.

Testing

• Save real payloads (scrubbed) as fixtures and run handlers against them in unit tests.
• Mock platform APIs, including 429 and 5xx responses, to test retry paths.
• Run a staging bot in a private chat or test server before each release.
• Load test broadcasts with the throttle in place, against a mock, never against real users.