My daily work include managing Debian virtual machines on bare-metal servers. I only use stable CLI tools available in Debian:
- obviously qemu/kvm
- lvm volumes, thinly provisioned.
- libvirt provide CLI utilities to run multiple VMs.
- a virtual network bridge and a dhcp server, either dnsmasq or isc-dhcp-server, with static leases
The installation of a new VM should be fully automated, i.e. not using the Debian installer.
I used to have my own scripts to create a new VM disk on LVM volume, it was based on grml-debootstrap and some xml templates. It worked well for a bunch of years. But it wasn’t packaged, wasn’t configurable and I had to make small modifications to make it work with various environments…
It was time to use new tools!
Create a base image
There are multiple flavors of cloud images, for bare-metal server choose the
genericcloud variant. This is almost, or even the same, images that are
shipped for openstack based environments. Do not use
nocloud variant, they are for testing purpose only.
Let’s download the
qcow2 image, convert it to LVM and use it as a
base image. The image is 2G on disk, so we must use a 2G volume.
$ wget https://cdimage.debian.org/cdimage/cloud/buster/daily/20200502-251/debian-10-genericcloud-amd64-daily-20200502-251.qcow2
$ lvcreate -n debian10 -T vg/thin -V2G
$ qemu-img convert debian-10-genericcloud-amd64-daily-20200502-251.qcow2 -O raw /dev/mapper/vg-debian10
Since our volume is thinly provisioned, we can reclaim unused space using virt-sparsify:
$ virt-sparsify --in-place /dev/mapper/vg-debian10
Create a new VM image from base image
We can create a new VM from this base image by using a snapshot. Remember that LVM thin snapshot are copy-on-write, so it’s free and we can drop the original volume safely.
$ lvcreate -kn -n myvm -s vg/debian10
Now we want a disk of 10G for our new image, we can resize it and cloud-init will extend the filesystem on startup! This is a HUGE improvement over my custom scripts…
$ lvresize -L10G vg/myvm
Prepare configuration data for cloud-init
Basically the procedure I want to create a VM is:
- Assign the VM a hostname and a IP address from DHCP
- Install my ssh key in the VM
To configure cloud-init we have the nocloud “data source”.
There are two kind of data to provision the image with cloud-init,
See cloud config
cloud-init has a lots of helpers to provision an image.
The configuration can be set:
- in a iso file mounted in the VM
smbiosoption when starting qemu
Since I don’t want to setup a http server just to serve
meta-data, I used an
iso image containing
user-data and an empty file
meta-data (the file
must be present). hostname is set via the
$ cat < EOF > user-data
- ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIACbwRLBlOnxXtENlAYFAmmUKWf+ihtCFLIQRyxKu97/ firstname.lastname@example.org
$ touch meta-data
$ genisoimage -output /var/lib/libvirt/images/seed.iso -volid cidata -joliet -rock user-data meta-data
Create and start the VM
The VM can now be created with
$ virt-install --name myvm \
--os-variant debian10 \
--memory 1024 \
--memorybacking hugepages=on \
--vcpu 2 \
--network bridge=brvm0 \
--graphics vnc \
--disk /dev/mapper/vg-myvm \
--disk /var/lib/libvirt/images/seed.iso,readonly=on \
--qemu-commandline='-smbios type=1,serial=ds=nocloud;h=myvm' \
The VM will not boot immediately, so we can get its MAC address and register it in DHCP and DNS servers
$ virsh dumpxml myvm | grep 'mac address'
After that the VM is ready to boot:
$ virsh start --console myvm
And can be accessed over ssh. Note the disk has been resized to 10G, see the full cast:
Destroying and restart
You’ll likely iterate until you find the best cloud-init configuration for you, so here’s how to destroy the VM:
$ virsh shutdown myvm
$ virsh undefine myvm
$ lvremove -y vg/myvm