Manage debian VMs on LVM with cloud-init

My daily work include managing Debian virtual machines on bare-metal servers. I only use stable CLI tools available in Debian:

The installation of a new VM should be fully automated, i.e. not using the Debian installer.

I used to have my own scripts to create a new VM disk on LVM volume, it was based on grml-debootstrap and some xml templates. It worked well for a bunch of years. But it wasn’t packaged, wasn’t configurable and I had to make small modifications to make it work with various environments…

It was time to use new tools!

Create a base image

Debian now provides cloud images ready to be provisioned by cloud-init.

There are multiple flavors of cloud images, for bare-metal server choose the genericcloud variant. This is almost, or even the same, images that are shipped for openstack based environments. Do not use nocloud variant, they are for testing purpose only.

Let’s download the qcow2 image, convert it to LVM and use it as a base image. The image is 2G on disk, so we must use a 2G volume.

$ wget https://cdimage.debian.org/cdimage/cloud/buster/daily/20200502-251/debian-10-genericcloud-amd64-daily-20200502-251.qcow2
$ lvcreate -n debian10 -T vg/thin -V2G
$ qemu-img convert debian-10-genericcloud-amd64-daily-20200502-251.qcow2 -O raw /dev/mapper/vg-debian10

Since our volume is thinly provisioned, we can reclaim unused space using virt-sparsify:

$ virt-sparsify --in-place /dev/mapper/vg-debian10

Create a new VM image from base image

We can create a new VM from this base image by using a snapshot. Remember that LVM thin snapshot are copy-on-write, so it’s free and we can drop the original volume safely.

$ lvcreate -kn -n myvm -s vg/debian10

Now we want a disk of 10G for our new image, we can resize it and cloud-init will extend the filesystem on startup! This is a HUGE improvement over my custom scripts…

$ lvresize -L10G vg/myvm

Prepare configuration data for cloud-init

Basically the procedure I want to create a VM is:

  • Assign the VM a hostname and a IP address from DHCP
  • Install my ssh key in the VM

To configure cloud-init we have the nocloud “data source”. There are two kind of data to provision the image with cloud-init, user-data and meta-data. See cloud config example, cloud-init has a lots of helpers to provision an image.

The configuration can be set:

  • in a iso file mounted in the VM
  • in smbios option when starting qemu

Since I don’t want to setup a http server just to serve meta-data, I used an iso image containing user-data and an empty file meta-data (the file must be present). hostname is set via the -smbios option.

$ cat < EOF > user-data
#cloud-config
ssh_authorized_keys:
  - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIACbwRLBlOnxXtENlAYFAmmUKWf+ihtCFLIQRyxKu97/ phil@philpep.org
EOF
$ touch meta-data
$ genisoimage  -output /var/lib/libvirt/images/seed.iso -volid cidata -joliet -rock user-data meta-data

Create and start the VM

The VM can now be created with virt-install:

$ virt-install --name myvm \
    --import \
    --os-variant debian10 \
    --memory 1024 \
    --memorybacking hugepages=on \
    --vcpu 2 \
    --network bridge=brvm0 \
    --graphics vnc \
    --disk /dev/mapper/vg-myvm \
    --disk /var/lib/libvirt/images/seed.iso,readonly=on \
    --qemu-commandline='-smbios type=1,serial=ds=nocloud;h=myvm' \
    --noreboot \
    --autostart \
    --noautoconsole

The VM will not boot immediately, so we can get its MAC address and register it in DHCP and DNS servers

$ virsh dumpxml myvm  | grep 'mac address'
    <mac address='52:54:00:1b:82:2c'/>

After that the VM is ready to boot:

$ virsh start --console myvm

And can be accessed over ssh. Note the disk has been resized to 10G, see the full cast:

Destroying and restart

You’ll likely iterate until you find the best cloud-init configuration for you, so here’s how to destroy the VM:

$ virsh shutdown myvm
$ virsh undefine myvm
$ lvremove -y vg/myvm