A CI/CD pipeline is not a fancy button that pushes code to production. It is a machine that turns a commit into a running system, and every stage in that machine has a specific job: prove the code is worth keeping, package it so it cannot be tampered with, and move that exact package through environments until it is serving traffic. Teams that treat the pipeline as plumbing end up with deployments that "work on my machine" — teams that treat it as a product can deploy on a Friday and sleep through it.
The pipeline is a funnel with one input
Every pipeline starts the same way: a commit on a branch triggers a webhook, the CI server clones the repository at that exact revision, and a build begins. The critical discipline is build once, promote many. The artifact produced at the end of the build — a container image, a tarball, a compiled binary — is the only thing that ever moves between environments. You never rebuild in staging or production, because a rebuild is a new artifact with a new hash, and a new hash is a thing you have not tested.
# .github/workflows/deploy.yml — the stages as a pipeline
name: build-and-promote
on:
push:
branches: [main]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: make test
- run: make build
- uses: docker/build-push-action@v6
with:
tags: registry.example.com/api:${{ github.sha }}
push: true
test-staging:
needs: build
runs-on: ubuntu-latest
environment: staging
steps:
- run: kargo promote --from main --to staging
- run: curl --fail --retry 30 https://staging.example.com/healthz
promote-prod:
needs: test-staging
runs-on: ubuntu-latest
environment: production
steps:
- run: kargo promote --from staging --to productionWhat each stage actually proves
The stages are not ritual — each one kills a specific failure class:
- Lint and static analysis catch the mistakes that compile anyway.
- Unit tests prove the smallest units behave, in isolation, fast.
- Integration tests prove units behave together. This is where most real bugs die, and where most pipelines are weakest.
- Artifact build produces the immutable, content-addressed package. Tag it
with the commit SHA, not with
latest.latestis a promise nobody keeps. - Deployment to staging proves the artifact can run on real infrastructure with real dependencies.
- Smoke tests against the deployment prove the deployed system answers requests — this catches config drift and missing environment variables that no amount of unit testing can.
A stage that cannot fail is not a stage; it is decoration. Every gate should have a realistic failure rate and a clear owner.
Artifact immutability is the contract
The artifact registry is the pipeline's memory. If the image tagged abcdef1 can
be overwritten by a later push, then staging tested one binary and production ran
another, and nobody can prove otherwise. Immutability means the tag maps to
exactly one digest, forever. When you need to know what is running in production
at 3 AM, the answer is kubectl get deploy -o jsonpath='{.status.observedGeneration}'
plus the image digest — not a guess about which commit "made it."
Why the pipeline is the product
The pipeline is the highest-leverage code in the repository. It encodes every decision about how the team ships: what gets tested, what gets reviewed, what gets deployed when. When the pipeline is slow or flaky, engineers work around it — skipping tests, committing directly to release branches, deploying from laptops. When the pipeline is fast and trustworthy, engineers stop fighting it and start relying on it.
The numbers that matter are lead time (commit to production) and change failure rate (deployments that require a rollback, fix, or mitigation). If lead time is measured in days because the pipeline has manual approval gates that nobody watches, the pipeline is a queue, not a machine. If the change failure rate is high, the pipeline is too weak — the tests are not catching what production will.
The gate that should not exist
Manual approval steps are the most common way teams destroy a pipeline's value. A human clicking "approve" at 2 PM on a Tuesday is not a quality gate — it is a bus factor. If you need humans in the loop, put them where they add judgment: reviewing the diff before merge, and deciding rollback during an incident. Between those two points, the machine should run unassisted.
The pipeline is the product because it is the only thing standing between every commit and every user. Make it deterministic, make it immutable, and make it fast enough that engineers never think about it — then watch the failure rate fall.