Scaling bots with queues, workers and monitoring: metrics, logs, traces and alerts

The shape of a scalable bot

platform --> ingress (verify, enqueue, 200 OK) --> queue --> workers --> outbound queue --> sender (rate limited) --> platform API

• Ingress verifies the signature, records the event ID, enqueues the event and returns 200 within milliseconds. Platforms have short timeouts (for example, about 3 seconds for Discord interactions and Slack events).
• Workers run your logic and may call databases, LLMs and other APIs.
• The sender is the only component that talks to the platform's send endpoints, so it can enforce rate limits centrally.

Choosing a queue

Whatever you choose: at-least-once delivery means duplicates happen, so handlers must be idempotent. Configure retries with backoff and a dead-letter queue for messages that keep failing.

Ordering

Users expect replies in order. Partition work by chat ID (a key per chat in Kafka, per-chat groups in BullMQ, or a per-chat lock) so one chat's events are processed sequentially while different chats run in parallel.

Broadcasts

Sending one message to many users is a batch job: create one job per recipient, throttle to the platform limit with a safety margin, record per-recipient status, skip users who blocked the bot (Telegram returns 403) and make it resumable. Estimate durations with the rate limit comparison.

Gateway bots

Discord gateway bots must use sharding once they reach the guild threshold (currently 2,500 guilds per shard); large bots run shards across processes or machines. Telegram polling bots cannot be scaled by running more pollers; switch to webhooks behind a load balancer instead.

What to measure

• Latency: time from platform event to reply sent (p50, p95, p99).
• Errors: handler exceptions and failed API calls, by type.
• Queue depth and age of the oldest message.
• 429 responses and retry counts, per route.
• Business signals: active chats, commands used, flow completion, LLM tokens and cost.

Logs and traces

• Log JSON with a correlation ID per update, the chat ID (or a hash), handler name and duration. Redact tokens and message content; the regex library has a pattern for common secret prefixes.
• Use OpenTelemetry traces across ingress, worker and sender to see where time goes.

Health and alerts

• Expose /healthz (process alive) and /readyz (dependencies reachable).
• For Telegram webhooks, getWebhookInfo reports pending_update_count and last_error_message. Poll it and alert when updates pile up.
• Alert on symptoms users feel (latency, error rate, queue age), not on every exception. Each alert should say what to check first.
• Run a synthetic check: a test account sends a command every few minutes and expects a reply.