The question comes up with every new service: container or virtual machine? The comparisons answer with resource-usage numbers, and invariably conclude that the container is lighter. That is true, and it is rarely what should decide.
A bad choice does not show up on day one. It shows up six months later, when you need to mount a network share, load a kernel module, or properly isolate a service that has gone bad.
The difference that explains all the rest
An LXC container shares the host kernel. It isolates user space only: processes, network, filesystem. There is no kernel boot and no hardware virtualization layer, so a container starts in a second and consumes only what its processes actually consume.
A VM ships its own kernel on virtualized hardware. It boots more slowly, reserves memory, and costs a few percent of performance. In exchange, it is genuinely independent from the host.
Everything else follows from that. Every advantage of the container and every limitation of the container is a direct consequence of that shared kernel.
The cases where the VM is the only answer
Four situations settle it immediately, no discussion:
- A different operating system. Windows, BSD, a dedicated firewall appliance: the shared kernel makes the container impossible by construction.
- A kernel module or low-level hardware access. The container has no kernel of its own: there is nothing for it to load anything into.
- A real security boundary. Internet-facing service, sensitive data, code you do not trust: a privilege-escalation flaw in the kernel crosses a container, not a VM.
- Docker. Nesting a container engine inside a container sometimes works, breaks regularly, and pushes you into enabling options that cancel out the isolation. A VM settles the question for good.
The cases where the container wins hands down
Conversely, LXC is the right default for anything that is a plain Linux service under your own control: a web server, a database, a monitoring service, an internal tool. You get instant start-up, a memory footprint proportional to real usage, and a density that lets you run dozens of services where a handful of VMs already saturated the machine.
The container also has an often-overlooked advantage when it comes to sharing hardware. A GPU handed to a VM via passthrough is taken away from the host and dedicated to that VM alone. The same GPU can instead be used simultaneously by several containers and by the host. For transcoding or shared inference, that is decisive.
Privileged or unprivileged: do not cave
An unprivileged container applies UID mapping: root inside the container maps to a user with no rights on the host. An escape lands on a harmless account rather than on real root. That is the default, and it should stay that way.
The scenario that ruins it is always the same. A network mount refuses to work inside the container, a quick search suggests switching to privileged, it works, you move on. You have just traded isolation for five minutes of convenience.
What the choice changes in day-to-day operations
Two practical differences weigh heavily over time.
Live migration first: a VM moves from one node to another with no perceptible interruption, a container has to be stopped. If you plan on hardware maintenance without downtime, that is a concrete argument.
Backups next. Both back up fine, but a container produces noticeably smaller and faster archives, because it carries neither a kernel nor a full disk image. On a homelab that backs up every day, the gap in size and duration becomes very visible.
The rule in one sentence
Container by default, VM as soon as the service touches the kernel or has to be genuinely isolated.
Put another way: if the service is an ordinary Linux program that you control, take a container. If it demands its own kernel, another operating system, dedicated hardware, or if it is exposed to third parties, take a VM. And if you are honestly torn about an exposed service, take the VM: the overhead is marginal, the safety margin is not.
Comments