A server you patch in place is a server whose exact state nobody can fully describe six months later. Immutable infrastructure fixes that by refusing to let servers change after they’re created — you rebuild instead of patch, every time.

The core rule

Once a server (or container, or VM image) is deployed, nothing SSHes into it and changes it. Need a new package, a config change, a security patch? Build a new image with the change baked in, and replace the running instance with it. The old one gets destroyed, not updated.

packer build -var "app_version=2.4.0" app-image.pkr.hcl
# produces ami-0abc123... with the new version baked in

terraform apply -var "ami_id=ami-0abc123..."
# replaces the running instance with a new one from that image

Nothing about the running server changed — a different server replaced it.

Why this reduces surprises

The bug that’s hardest to reproduce is the one caused by a server’s accumulated history: a manual apt install from a debugging session eight months ago, a config edit nobody wrote down, a cron job someone added and forgot. Mutable servers accumulate this kind of undocumented state continuously. Immutable ones can’t — there’s no path for undocumented state to persist, because every server is built fresh from the same definition.

This also makes “it works on that server but not this one” nearly impossible, since every instance of a given image is bit-for-bit identical. Debugging gets easier because you’ve eliminated an entire category of “what’s different about this box” investigation.

What it costs

• Build time. Every change means a new image build, not a quick in-place edit — a few minutes of build time you didn’t pay before, on every deploy.
• Statelessness discipline. Anything the app needs to persist — uploaded files, session data, the database — has to live outside the instance, in object storage or a managed database, because the instance itself is disposable and will be destroyed on the next deploy.
• Image storage and versioning overhead. You’re now managing a library of images, not one server that evolves — a small tax, but a real one.

Where it fits at small scale

You don’t need a full image-pipeline setup to get the core benefit. The minimum version that still counts as immutable:

• Container images built in CI, tagged by commit SHA, never edited after push.
• Deploys that pull the new tag and replace the running container, rather than docker exec-ing in to patch a running one.
• A Dockerfile (or Packer template) that is the actual, complete definition of the server — not a rough approximation of steps someone remembers taking.

FROM node:20-slim
COPY . /app
RUN npm ci --production
CMD ["node", "/app/server.js"]

If your deploy process is “build this image, run this image, never patch the running one,” you already have immutable infrastructure, container-shaped, without needing full VM image pipelines.

When patching in place is still fine

A single dev or staging box you rebuild rarely and don’t depend on for uptime doesn’t need this discipline — the point of immutability is protecting against drift you’ll actually pay for later, and a throwaway box has no “later” worth protecting.

The gotcha

Immutable infrastructure only holds if nobody breaks the rule under pressure. The first time there’s an incident and someone SSHes in to hotfix a running instance “just this once, we’ll bake it into the image tomorrow,” you’ve reintroduced drift — and that instance is now different from what your pipeline believes it deployed. Either fix forward through a real rebuild, even during an incident, or accept that the rule doesn’t actually hold at your org and plan around that honestly.