If you send webhooks and don’t retry failed deliveries, you’re silently dropping events every time a customer’s endpoint has a bad five minutes. Reliable webhook delivery is retries, backoff, signatures, and somewhere failed events land when they finally give up.

Retries with exponential backoff

A single failed attempt isn’t a reason to give up — endpoints go down for seconds or minutes, not forever. Retry with increasing delay so you’re not hammering a struggling server:

flowchart TD
    A[Deliver webhook] --> B{Response OK?}
    B -->|Yes| C[Mark delivered]
    B -->|No or timeout| D{Attempts left?}
    D -->|Yes| E["Wait backoff, retry"]
    E --> A
    D -->|No| F[Move to dead letter]
  

const backoffSchedule = [10, 60, 300, 1800, 7200] // seconds: 10s, 1m, 5m, 30m, 2h

async function deliverWebhook(event, attempt = 0) {
  try {
    const res = await fetch(event.endpoint, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'X-Webhook-Signature': sign(event.payload, event.secret),
      },
      body: JSON.stringify(event.payload),
      signal: AbortSignal.timeout(10_000),
    })
    if (res.ok) return { delivered: true }
    throw new Error(`endpoint returned ${res.status}`)
  } catch (err) {
    if (attempt >= backoffSchedule.length) {
      await moveToDeadLetter(event, err)
      return { delivered: false, exhausted: true }
    }
    await scheduleRetry(event, attempt + 1, backoffSchedule[attempt])
    return { delivered: false, retrying: true }
  }
}

Cap retries at five or six attempts over a few hours. Beyond that, the endpoint is either permanently broken or the customer needs to know, not something more retries will fix.

Signatures, so the receiver can trust the payload

Anyone can POST to a public webhook URL. Sign the payload with a secret unique per customer, and tell them to verify it before trusting the body:

const crypto = require('crypto')

function sign(payload, secret) {
  const hmac = crypto.createHmac('sha256', secret)
  hmac.update(JSON.stringify(payload))
  return hmac.digest('hex')
}

Document the verification snippet for your customers — this is the part they’ll get wrong if you don’t hand it to them:

function verify(payload, signature, secret) {
  const expected = sign(payload, secret)
  return crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(expected))
}

Use timingSafeEqual, not === — a plain string comparison leaks timing information an attacker can use to guess the signature byte by byte.

The dead letter queue

Events that exhaust retries need to land somewhere visible, not vanish:

CREATE TABLE webhook_dead_letters (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  event_id UUID NOT NULL,
  endpoint TEXT NOT NULL,
  payload JSONB NOT NULL,
  last_error TEXT,
  failed_at TIMESTAMPTZ DEFAULT now()
);

Surface this table in your dashboard, or at minimum alert on rows appearing in it. A customer whose endpoint has been silently failing for a week and never found out is a support ticket and a trust problem, both avoidable with one query a day.

Idempotency on the receiving end

Retries mean a customer’s endpoint might see the same event twice — once from a delayed original delivery, once from a retry that fired before the first one’s response came back. Include an event_id in every payload and tell customers to dedupe on it. This is their responsibility, but only if you give them the ID to dedupe against.

{
  "event_id": "evt_9f8a3c",
  "type": "payment.succeeded",
  "data": { "amount": 4200 }
}

What to skip

Skip building a full webhook management UI, per-event delivery analytics, or a replay system before you have customers who need any of it. Start with: sign, retry with backoff, dead-letter the exhausted ones, and alert on the dead-letter table filling up. That covers the actual failure modes.