Background Jobs Without a Job Queue Service

27 Feb 2026

You don’t need Sidekiq, BullMQ, or a managed queue service to run background jobs reliably. A table and a worker loop cover more ground than people assume, and they’re something you already know how to debug.

The jobs table

CREATE TABLE jobs (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  type TEXT NOT NULL,
  payload JSONB NOT NULL,
  status TEXT NOT NULL DEFAULT 'pending', -- pending, running, done, failed
  attempts INT NOT NULL DEFAULT 0,
  run_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  locked_at TIMESTAMPTZ,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE INDEX idx_jobs_pending ON jobs (run_at) WHERE status = 'pending';

run_at gives you scheduled jobs for free — “send this email in 24 hours” is just an insert with run_at set to tomorrow. attempts gives you retry counting without extra machinery.

The worker loop

A single worker’s life cycle:

flowchart TD
    A[Poll for job] --> B{Job available?}
    B -->|No| A
    B -->|Yes| C[Claim job]
    C --> D[Run handler]
    D --> E{Success?}
    E -->|Yes| F[Mark done]
    E -->|No| G{Attempts >= 5?}
    G -->|Yes| H["Mark failed, alert"]
    G -->|No| I["Retry with backoff"]

async function workLoop() {
  while (true) {
    const job = await claimNextJob()
    if (!job) {
      await sleep(1000) // nothing to do, wait before polling again
      continue
    }
    await runJob(job)
  }
}

async function claimNextJob() {
  // SKIP LOCKED lets multiple workers run this query concurrently
  // without grabbing the same row
  const { rows } = await db.query(`
    UPDATE jobs SET status = 'running', locked_at = now(), attempts = attempts + 1
    WHERE id = (
      SELECT id FROM jobs
      WHERE status = 'pending' AND run_at <= now()
      ORDER BY run_at
      LIMIT 1
      FOR UPDATE SKIP LOCKED
    )
    RETURNING *
  `)
  return rows[0]
}

FOR UPDATE SKIP LOCKED is the piece that makes this safe to run with more than one worker process — Postgres and MySQL 8+ both support it. Without it, two workers race to claim the same row and one of them does duplicate work.

Running the job and handling failure

const handlers = {
  send_email: async (payload) => sendEmail(payload.to, payload.template),
  generate_report: async (payload) => generateReport(payload.userId),
}

async function runJob(job) {
  try {
    await handlers[job.type](job.payload)
    await db.query(`UPDATE jobs SET status = 'done' WHERE id = $1`, [job.id])
  } catch (err) {
    if (job.attempts >= 5) {
      await db.query(`UPDATE jobs SET status = 'failed' WHERE id = $1`, [job.id])
      await alertOnFailedJob(job, err)
    } else {
      // retry later with backoff, not immediately
      const delaySeconds = 30 * Math.pow(2, job.attempts)
      await db.query(
        `UPDATE jobs SET status = 'pending', run_at = now() + $2 * interval '1 second' WHERE id = $1`,
        [job.id, delaySeconds]
      )
    }
  }
}

Stuck jobs

A worker can crash mid-job, leaving a row stuck at status = 'running' forever. Sweep for these periodically and reset them:

UPDATE jobs SET status = 'pending'
WHERE status = 'running' AND locked_at < now() - interval '10 minutes';

Run this as its own cron job, separate from the worker loop, so a dead worker doesn’t leave orphaned rows indefinitely.

When this stops being enough

This setup handles low-to-moderate throughput fine — hundreds to low thousands of jobs per minute, depending on your database. It falls apart when:

You need priority queues with complex ordering rules.
Polling latency (that 1-second sleep) is too slow for your use case.
Job volume is high enough that polling load itself competes with your app’s real database traffic.

That’s when a real queue (Redis-backed, or a managed service) earns its complexity. Most small apps never hit that line. Don’t add the dependency until your jobs table’s SELECT COUNT(*) numbers tell you to.

What to skip

Skip job priorities, job chaining/DAGs, and a dashboard UI until you actually need them. A SELECT status, count(*) FROM jobs GROUP BY status in a terminal is a perfectly good dashboard for the first year.

IAnswerable