Axis Inbox

Provider webhooks

This is where messaging providers deliver to inbox: an inbound WhatsApp message, a Meta comment, an SMS delivery receipt, an email bounce. One controller, two routes, nine adapters.

GET  /webhooks/:integrationKey   subscription challenge
POST /webhooks/:integrationKey   signed ingestion
POST /webhooks/email             alias for POST /webhooks/:integrationKey with 'email'

These paths are not under /v1 and never will be. A provider's webhook URL is configured once, inside someone else's dashboard, often by a human who will not be revisiting it. Versioning it would mean asking every customer to reconfigure every provider on a version bump. /v1 is for the API you call; this is the door providers knock on.

Both routes are @Public(). Authentication is the provider's signature, checked per adapter — see Signature verification.

The challenge

Most providers verify ownership of a webhook URL by GETting it with a challenge value and expecting it echoed back:

GET /webhooks/facebook.native?hub.mode=subscribe&hub.verify_token=...&hub.challenge=1158201444

The controller hands the query and headers to the adapter's verifyChallenge, then:

That last detail is load-bearing. Providers compare the raw response body byte for byte, so the challenge response deliberately bypasses the { data, meta } envelope every other route uses. Wrap 1158201444 in JSON and the subscription fails with no useful error.

Ingestion

POST /webhooks/:integrationKey does four things, in this order:

  1. Verify — adapter.verifyWebhook(rawBody, headers) over the raw body bytes.
  2. Persist — a WebhookInbox row with integrationKey, event, rawBody, payloadJson.
  3. Enqueue — the row id onto the webhook-retry queue.
  4. Return { "status": "queued" } with a 200.

Verification happens before any persistence. An unsigned or wrongly-signed delivery is rejected with invalid_api_key and never touches the database, so an attacker cannot fill the inbox table by POSTing junk at a known URL.

The response is queued, not processed. Normalisation, thread upsert, contact resolution and delivery-receipt reconciliation all happen in the worker. Providers get their 200 immediately, which is what keeps them from retrying a delivery that is merely slow.

The enqueue is awaited, and deliberately so: if the row cannot be scheduled, the POST fails and the provider retries, rather than the delivery being accepted and silently dropped. Processing itself runs detached.

The optional-payload trap

WebhookVerification.payload is optional. It exists only to save a re-parse for adapters whose verification already had to parse the body; an adapter that verifies from raw bytes alone correctly returns { ok: true } and nothing else.

let payloadJson: unknown = verification.payload ?? parseJson(rawBody);

Taking verification.payload as the whole answer stored null for those adapters, and normalizeWebhook(null) yields zero events — so the delivery was marked processed with no error and no message. A silent drop that looked like success in every log and every column. The fallback parse is what makes the field genuinely optional.

WebhookInbox is the durable buffer

Every accepted delivery becomes a row before anything interprets it:

model WebhookInbox {
  id             String    @id @default(cuid())
  integrationKey String
  appId          String?
  event          String?
  rawBody        String
  payloadJson    Json?
  status         String    @default("received")
  attemptCount   Int       @default(0)
  nextRetryAt    DateTime?
  lastError      String?
  processedAt    DateTime?
}

rawBody is kept as the original string, not just the parsed object. When a normalisation bug is found six weeks later, the raw bytes are what lets a delivery be reprocessed correctly rather than reconstructed from a lossy parse.

Statuses: received → processed, or failed while retrying, then dead_letter. Retries follow the shared policy — 60s, 5m, 15m, 30m, then 60m, dead-lettered after 8 attempts. A row already processed or dead_letter is skipped on re-entry, so a duplicate queue job is a no-op.

Dead-letter inspection

GET  /v1/admin/webhook-inbox?status=dead_letter
POST /v1/admin/webhook-inbox/:id/requeue

Scope admin. Both are strictly scoped to auth.appId — another app's deliveries are never listed and a requeue of a delivery you do not own returns not_found, not a 403.

status defaults to dead_letter; pass failed or received to see rows still in flight. The default limit is 100.

requeue resets the row to received with attemptCount: 0, clears lastError and processedAt, and enqueues it. It returns the status snapshotted before the enqueue, because awaiting the enqueue can let an in-memory processor advance the row to processed first.

# What failed?
curl -s https://inbox.example.com/v1/admin/webhook-inbox?status=dead_letter \
  -H "x-api-key: sk_..." | jq '.data[] | {id, integrationKey, lastError, attemptCount}'

# Fix the cause, then replay one.
curl -X POST https://inbox.example.com/v1/admin/webhook-inbox/<id>/requeue \
  -H "x-api-key: sk_..." \
  -H "idempotency-key: $(uuidgen)"

Debugging

INBOX_LOG_WEBHOOKS=true logs every inbound delivery with its integration key, signing header and full raw body:

inbound whatsapp sig=sha256=9f3c… body={"results":[…]}

It reads every signing header the service accepts — x-hub-signature-256, x-axis-signature, x-openwa-signature, x-webhook-signature, x-signature — not just Meta's. An earlier version read only two, so a correctly-signed delivery logged as sig=(none), which reads as "unsigned" and sends whoever is debugging chasing a signature problem that did not exist.

It logs raw bodies, which contain customer message content. Use it to diagnose, then turn it off.

INBOX_SKIP_WEBHOOK_VERIFY=true accepts deliveries whose signature failed, logging a warning for each. It is for local development against a provider you cannot get a secret from. Never enable it in production — with it on, anyone who knows your webhook URL can inject messages into any workspace.

← Outbound webhooks Signature verification →