Background Jobs, Cron, and Queues: Choosing the Right Execution Model

"Run it in the background" hides a design decision with three very different answers. Cron fires on a schedule. Queues fire on events. Delayed jobs fire on both — an event at a fixed offset. The three models differ in delivery semantics, observability, and failure behavior, and mixing them up is how you end up with a cron job that deletes orders while the queue that owns those orders is still processing them.

Cron: time as the trigger

Cron is a clock, not a delivery system. At 02:00 the scheduler executes the command. What happens after that is entirely on you: the process exits nonzero, the supervisor notices or doesn't, and the job's failure is visible only where someone thought to look.

# crontab
0 2 * * *  /opt/bin/db-backup --full  >> /var/log/backup.log 2>&1
*/15 * * * *  /opt/bin/metrics-rollup --window 15m

Where cron still earns its place: maintenance with no ordering constraints and a natural wall-clock anchor — nightly backups, log rotation, DDL rollouts at 03:00. The failure model is best effort with external supervision: cron itself has no concept of retries, backoff, or idempotent re-execution. Kubernetes CronJob is the same model with better hygiene — it runs the pod, surfaces Failed status, and relies on the job being idempotent because overdue jobs are simply run late.

Queues: events as the trigger

A work queue (Redis-backed, RabbitMQ, SQS, or an RDBMS table) decouples producing work from executing it. The producer enqueues, the worker dequeues. That separation buys you three things for free: the producer never blocks on execution, workers scale horizontally, and failures are visible as unacked or dead-lettered messages rather than a missing log line.

The dominant semantic is at-least-once delivery. RabbitMQ confirms the message when the worker acks it; if the worker dies mid-processing, the message is requeued and delivered again. SQS hides a message for the visibility timeout, then makes it visible again if the worker never deletes it. Every at-least-once system delivers duplicates under failure — which is why the job's effect must be idempotent (see idempotency keys — the same reasoning applies to jobs).

// worker loop: receive, process, ack
for msg := range ch.Consume("jobs", "worker-1", autoAck=false, ...) {
    err := process(msg.Body)
    if err == nil {
        msg.Ack(false)              // commit: remove from queue
    } else if msg.Nack(false, true) { // requeue for retry
        continue
    } else {
        deadLetter(msg)             // poison message
    }
}

The queue, not the worker, owns retry policy: max delivery counts, requeue delays, and dead-letter queues. A message that fails three times lands in the DLQ — the only place a permanently failing job becomes visible.

Delayed jobs: the hybrid

Scheduled future work — "send the receipt in 24 hours" — is the awkward middle. Options:

• RabbitMQ delayed message exchange / SQS delay queue: the broker holds the message until the delay expires. Simple, but the delay is fixed per queue in SQS (0–15 min), and the message must not outlive broker retention.
• Redis sorted sets: ZADD schedule <unix-ts> job and a worker polls ZRANGEBYSCORE for due items. This is what Celery's eta and Sidekiq's scheduled set do under the hood — simple, but a worker must poll, and ordering guarantees are weaker than a queue's.
• Job schedulers with cron tables: a table of jobs with next_run_at that a sweeper claims with UPDATE ... WHERE next_run_at <= now() RETURNING * (Postgres's atomic claim pattern). Best fit when you need catch-up: missed schedules run late rather than being dropped.

# claim due jobs atomically in Postgres
rows = db.execute("""
    UPDATE scheduled_jobs
    SET state = 'claimed', claimed_at = now()
    WHERE id IN (
        SELECT id FROM scheduled_jobs
        WHERE next_run_at <= now() AND state = 'pending'
        ORDER BY next_run_at LIMIT 100
        FOR UPDATE SKIP LOCKED
    )
    RETURNING id, payload
""")

The decision matrix

Where each model fails

Cron fails silently — a skipped run leaves no artifact unless the job writes one. Queues fail loudly but can flood: a bad deploy that nacks everything spins an infinite requeue loop, and unbounded queues grow until the broker evicts or the OOM killer fires. Delayed jobs fail by lapsing: a Redis-sorted-set job whose worker is down simply stays in the set, and nothing notices until the next poller sweep.

The common discipline across all three: jobs must be idempotent, bounded, and visible. Idempotent so at-least-once delivery is safe. Bounded — every queue gets a max-retry and a DLQ, every cron a timeout and a supervisor. Visible — success and failure both land in a metrics path, because a background job that nobody watches is indistinguishable from one that never ran.