KVM, libvirt, and Ubuntu 14.04
I recently upgraded one of the servers in the office to Ubuntu 14.04, the most recent LTS release. While doing this, I decided that I would figure out a clean way of managing virtual machines. This is what I found:
Canonical now publishes Ubuntu cloud images.
These match the official images that are used on EC2. It turns out that they
can also be used in a local environment, complete with the customizations that
user-data provide.
libvirt appears to be the cleanest abstraction of
KVM/QEMU, Xen, LXC, and others, so I took a stab at using that to manage my
VMs.
Installation was easy:
sudo apt-get install libvirt-bin
Creating VMs (well, importing existing images and running them) was another
matter. Over the last few years we've seen a proliferation of tools that
attempt to provision VMs without needing to create XML manifests (ultimately
culminating in OpenStack). virt-install, which comes with libvirt ended up
being the tool for me.
To start, I downloaded a qcow2 Ubuntu Cloud image into libvirt's images/
directory:
curl -LO http://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img
sudo cp trusty-server-cloudimg-amd64-disk1.img /var/lib/libvirt/images/
virsh pool-refresh default # tell libvirt to re-scan for new files
I also added an ethernet bridge device so that the VMs could co-exist on the network with everything else in the office:
cat <<EOF | sudo tee -a /etc/network/interfaces
auto br0
iface br0 inet dhcp
bridge_ports eth0
EOF
When booting one of the Ubuntu Cloud images on its own, it briefly hangs while
waiting for network connections to EC2's internal network that exposes
metadata. Rather than replicating their environment, the Cloud images have
a relatively secret super
power:
the ability to pull cloud-init
configurations off of a secondary mounted image.
To create an image containing a `cloud-init configuration, create 2 files:
meta-data and user-data. These are their minimal forms (meta-data appears
to be ignored / fails to set the hostname):
cat <<EOF > meta-data
instance-id: iid-local01;
local-hostname: ubuntu
EOF
cat <<EOF > user-data
#cloud-config
EOF
Next, write these into an ISO and copy it into libvirt's images/ directory:
genisoimage -output configuration.iso -volid cidata -joliet -rock user-data meta-data
sudo cp configuration.iso /var/lib/libvirt/images/
virsh pool-refresh default
With this in place, it's now possible to boot a VM and avoid the network wait.
This will create a new VM ("domain" in libvirt parlance) with 1G RAM, 1 vCPU,
and bridged networking:
virsh vol-clone --pool default trusty-server-cloudimg-amd64-disk1.img test.img
virt-install -r 1024 \
-n test \
--vcpus=1 \
--autostart \
--memballoon virtio \
--network bridge=br0 \
--boot hd \
--disk vol=default/test.img,format=qcow2,bus=virtio \
--disk vol=default/configuration.iso,bus=virtio
virsh list
To see the generated XML for this domain, run:
virsh dumpxml test
To stop it:
virsh destroy test
To remove it:
virsh undefine test
virsh vol-delete --pool default test.img
Now, since we have the power of cloud-init, we can modify the initial boot
configuration to do some initial provisioning. To do that, update user-data:
cat <<EOF > user-data
#cloud-config
# upgrade packages on startup
package_upgrade: true
# install git
packages:
- git
# create a user
runcmd:
- [ useradd, -c, Seth Fitzsimmons, -u, 1001, -G, sudo, -U, -M, -p, $5$FVJ1C48Rlhy/$GOidCu4a0qTmngqhFMGT7z/N.8nYTuXaaGzEDPhfIL., -s, /bin/bash, seth ]
EOF
cloud-init supports user creation since 0.7.0 (trusty comes with 0.7.5), but
it does not appear to work locally and I'd like to be able to re-use these
configurations with Ubuntu 12.04 images (which ship with cloud-init 0.6.3), so
I'm doing the same thing by hand withruncmd`.
So far (which hasn't been that long), this has been working well. One of the next steps is to achieve a similar streamlined workflow for LXC / Docker, similar to what Mike wrote up about LXC and Virtualbox.
