Background Job Processing
A background job is a row in a table or a message on a broker plus a worker that polls it. This video follows one job from enqueue through claim, execution, and acknowledgment, and shows what the queue does when the worker dies mid-job — the retry and backoff machinery that keeps work moving.
Topics covered:
- Enqueue: a row in a table or a message on a broker
- Workers: polling, claiming, and processing units of work
- Acknowledgment: what tells the queue the job is done
- Retries and backoff: what happens on failure
- Concurrency control: how many workers, how much work
- Cron vs queue: scheduled jobs and recurring tasks
Related articles
Background Jobs, Cron, and Queues: Choosing the Right Execution Model
Cron schedules, delayed jobs, and work queues compared: exactly-once vs at-least-once delivery, broker semantics, and when each model fails.
Connection Pools Aren't Free
Pool sizing, exhaustion, idle connections, and the hidden queue — why 'just add a pool' is incomplete advice.
Circuit Breakers and Timeouts: Protecting Services from Cascading Failure
Circuit breaker states — closed, open, half-open — plus timeout budgets and retry limits: the mechanisms that stop cascading failures before they spread.
More in Backend Engineering
Pagination Strategies
Offset, keyset, and cursor pagination compared by database cost, consistency, and behavior when rows are inserted mid-page.
DetailsCaching for the Service Layer
Service-layer caching done right — cache keys, invalidation, TTLs, and the stampede problem that brings services down.
DetailsWebhooks vs Polling
Webhooks and polling compared at the delivery level — delivery guarantees, retries, and when push is worse than pull.
DetailsDatabase Transactions in Practice
ACID in a real database — write-ahead logs, locks, undo and redo, plus isolation levels and what they change about reads.
DetailsRate Limiting Algorithms, Visualized
Token buckets, leaky buckets, and sliding windows compared by memory, precision, and what they do under burst load.
DetailsAPI Idempotency Keys
How idempotency keys work at the storage level — dedupe tables, unique constraints, and making retries safe for clients.
DetailsREST APIs from the Inside
What happens when a REST API processes a request — routing, deserialization, validation, handlers, and response building end to end.
DetailsMessage Queues in Practice
Brokers, producers, and consumers — what a queue does at the protocol level and why it decouples services from availability.
DetailsConnection Pooling Explained
What a connection pool actually does — the handshake cost it hides, checkout and release mechanics, and how pool size changes latency.
DetailsDepth, delivered weekly
One technical dispatch a week — articles and episode notes before they go public.
One technical dispatch per week. No noise.