↓ Skip to main content
  1. Posts/

Running KVM inside a VPS: where it helps and where it hurts

·4 mins
Infrastructure Kvm Virtualization Vps Linux
Table of Contents

The question
#

Every few months a client asks us some version of this:

“Can I run KVM inside my VPS and spin up a few VMs?”

Yes, you can, but the conditions attached to that yes matter more than the yes itself.

Does nested KVM work at all?
#

Nested virtualization means your VPS, which is already a KVM guest on your provider’s hypervisor, runs KVM itself and hosts its own guests. That gives you three levels: the physical host, your VPS, and the VMs inside your VPS.

It only works if all of the following hold:

  1. The outer hypervisor has nested virtualization enabled in the CPU flags it exposes to your VPS.
  2. The VPS CPU advertises vmx (Intel) or svm (AMD).
  3. The kernel on your VPS loads kvm_intel / kvm_amd with nested=1.

You can check from inside the VPS:

grep -E 'vmx|svm' /proc/cpuinfo | head -1
# if you see vmx or svm, nested is exposed

cat /sys/module/kvm_intel/parameters/nested 2>/dev/null || \
  cat /sys/module/kvm_amd/parameters/nested 2>/dev/null
# should print Y or 1

Whether nested virtualization is exposed depends on the provider and the plan. Many hosts leave it off on shared nodes, since one noisy neighbour spinning up VMs is enough to ruin everyone’s day. Run the check above, and if the flags are missing, ask your provider’s support before you build anything that depends on them.

The performance cost
#

Nested KVM has real overhead. These numbers come from a test VPS (4 vCPU, 8 GB, NVMe, Debian 12, QEMU 8.1):

Benchmark                        Native    L1 VPS    L2 (nested)   overhead vs L1
sysbench cpu prime=20000         4.21 s    4.32 s    4.98 s         +15 %
fio randread 4k qd=32 (IOPS)    88 k      71 k      42 k            -41 %
iperf3 single-stream (Gbps)      9.4       9.1       5.2            -43 %

The CPU penalty is modest. Disk I/O and network both lose around 40%, which is what you pay when every I/O operation gets translated twice.

If a workload is mostly CPU and RAM, like compilers, build agents or small test suites, it runs fine nested. Databases, busy web servers and anything that pushes a lot of network traffic will feel the difference.

Good uses
#

  • Throwaway test environments. If you want to try a Kubernetes install, a new distro or a kernel upgrade, a disposable VM inside your VPS is quicker and cheaper than renting a second VPS.
  • Teaching and labs. One VPS running 3-4 small Alpine VMs for a networking course is good value.
  • Legacy stacks you have to keep alive. An old PHP 5.6 app that you don’t want anywhere near your main system can sit isolated in its own small inner VM.
  • Container escape testing. For security work you want a nested target you can wipe without a second thought.

Where it goes wrong
#

  • Production databases in nested VMs. The -41% on IOPS in the table above is not a typo, and a database that matters shouldn’t be living with it.
  • Anything where you care about resources. Your VPS already shares a physical machine with other tenants, and nesting puts a second layer of noisy neighbours on top of that.
  • Reselling slices of your VPS as VPSes. It’s technically possible and commercially a disaster: the overhead eats your margin, and the first abuse report lands on you.
  • Windows guests. It can be done, but between the QEMU display, drivers and licensing it’s rarely worth the headache.

Reducing the overhead
#

If you do run nested, three changes help a lot.

1. Use virtio everywhere
#

<disk type='file' device='disk'>
  <driver name='qemu' type='qcow2' cache='none' io='native'/>
  <source file='/var/lib/libvirt/images/guest.qcow2'/>
  <target dev='vda' bus='virtio'/>
</disk>

<interface type='network'>
  <source network='default'/>
  <model type='virtio'/>
</interface>

The default emulated ide and rtl8139 devices will wreck nested performance. Treat virtio-blk and virtio-net as required, not as a tuning option.

2. Use raw or preallocated qcow2, not thin
#

A thin-provisioned qcow2 image on top of a thin-provisioned VPS disk means two layers of sparse allocation, and your fio numbers will show it.

qemu-img create -f qcow2 -o preallocation=metadata guest.qcow2 20G
# or just: qemu-img create -f raw guest.img 20G

3. Pin and reserve, don’t over-commit
#

If your VPS has 4 vCPU and 8 GB RAM, don’t give 4 vCPU and 7 GB to a single inner VM. Give it 2 vCPU and 4 GB and keep real headroom. Nested setups get fragile when the host (your VPS, in this case) is under memory pressure: KSM starts fighting for pages and performance falls off non-linearly.

Try containers first
#

Nine times out of ten, a client asking for nested KVM really wants isolation rather than a full virtual machine. Docker, systemd-nspawn or LXC inside the VPS add almost no overhead and give you 80% of the isolation benefits.

If you need separate kernels, for custom modules, different distro versions or kernel-level testing, nested KVM is the right tool. For everything else, start with containers.

Related