I recently upgraded one of the servers in the office to Ubuntu 14.04, the most recent LTS release. While doing this, I decided that I would figure out a clean way of managing virtual machines. This is what I found:
Canonical now publishes Ubuntu cloud images.
These match the official images that are used on EC2. It turns out that they
can also be used in a local environment, complete with the customizations that
libvirt appears to be the cleanest abstraction of
KVM/QEMU, Xen, LXC, and others, so I took a stab at using that to manage my
Installation was easy:
sudo apt-get install libvirt-bin
Creating VMs (well, importing existing images and running them) was another
matter. Over the last few years we've seen a proliferation of tools that
attempt to provision VMs without needing to create XML manifests (ultimately
culminating in OpenStack).
virt-install, which comes with
libvirt ended up
being the tool for me.
To start, I downloaded a qcow2 Ubuntu Cloud image into
curl -LO http://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img sudo cp trusty-server-cloudimg-amd64-disk1.img /var/lib/libvirt/images/ virsh pool-refresh default # tell libvirt to re-scan for new files
I also added an ethernet bridge device so that the VMs could co-exist on the network with everything else in the office:
cat <<EOF | sudo tee -a /etc/network/interfaces auto br0 iface br0 inet dhcp bridge_ports eth0 EOF
When booting one of the Ubuntu Cloud images on its own, it briefly hangs while
waiting for network connections to EC2's internal network that exposes
metadata. Rather than replicating their environment, the Cloud images have
a relatively secret super
the ability to pull
configurations off of a secondary mounted image.
To create an image containing a `
cloud-init configuration, create 2 files:
user-data. These are their minimal forms (
to be ignored / fails to set the hostname):
cat <<EOF > meta-data instance-id: iid-local01; local-hostname: ubuntu EOF cat <<EOF > user-data #cloud-config EOF
Next, write these into an ISO and copy it into
genisoimage -output configuration.iso -volid cidata -joliet -rock user-data meta-data sudo cp configuration.iso /var/lib/libvirt/images/ virsh pool-refresh default
With this in place, it's now possible to boot a VM and avoid the network wait.
This will create a new VM ("domain" in
libvirt parlance) with 1G RAM, 1 vCPU,
and bridged networking:
virsh vol-clone --pool default trusty-server-cloudimg-amd64-disk1.img test.img virt-install -r 1024 \ -n test \ --vcpus=1 \ --autostart \ --memballoon virtio \ --network bridge=br0 \ --boot hd \ --disk vol=default/test.img,format=qcow2,bus=virtio \ --disk vol=default/configuration.iso,bus=virtio virsh list
To see the generated XML for this domain, run:
virsh dumpxml test
To stop it:
virsh destroy test
To remove it:
virsh undefine test virsh vol-delete --pool default test.img
Now, since we have the power of
cloud-init, we can modify the initial boot
configuration to do some initial provisioning. To do that, update
cat <<EOF > user-data #cloud-config # upgrade packages on startup package_upgrade: true # install git packages: - git # create a user runcmd: - [ useradd, -c, Seth Fitzsimmons, -u, 1001, -G, sudo, -U, -M, -p, $5$FVJ1C48Rlhy/$GOidCu4a0qTmngqhFMGT7z/N.8nYTuXaaGzEDPhfIL., -s, /bin/bash, seth ] EOF
cloud-init supports user creation since 0.7.0 (trusty comes with 0.7.5), but
it does not appear to work locally and I'd like to be able to re-use these
configurations with Ubuntu 12.04 images (which ship with
cloud-init 0.6.3), so
I'm doing the same thing by hand withruncmd`.
So far (which hasn't been that long), this has been working well. One of the next steps is to achieve a similar streamlined workflow for LXC / Docker, similar to what Mike wrote up about LXC and Virtualbox.