Local Ubuntu VM provisioning with cloud-init

VMs are widely used in QA to provision clean and controlled test environment. A traditional way to create new VMs automatically in Ubuntu is to download an ISO, prepare a preseed and install a system from this image. This method is nice but involves lot of technical parts and is more or less reliable when it comes to mass-provisioning VMs depending on the technologies you choose.

Cloud technologies bring us new ways to provision local images for testing. cloud-init is one of those. It helps you get a pristine environment in a couple of minutes without the trouble of preseeding, pxe booting, dnsmasq setup or libvirt machine definition.

The first thing you need is a base image. We are testing the development release so lets get a base server image of Ubuntu Quantal. Cloud images for the development release are built daily and available from http://cloud-images.ubuntu.com/quantal/current/

mkdir /tmp/cloudinit && cd /tmp/cloudinit
wget http://cloud-images.ubuntu.com/quantal/current/quantal-server-cloudimg-amd64-disk1.img

You can start this image directly with kvm but as it is, it is not really well suited for QA and we need to tune it a bit.

First, the size. By default the size of the image is 2GB

$ qemu-img info quantal-server-cloudimg-amd64-disk1.img 
image: quantal-server-cloudimg-amd64-disk1.img
file format: qcow2
virtual size: 2.0G (2147483648 bytes)
disk size: 221M
cluster_size: 65536

This is not enough to build some packages, or install dependencies for desktop packages. Let resize it to 4GB. cloud-init will take care of expanding the existing filesystem to the maximum space available.

$ qemu-img resize quantal-server-cloudimg-amd64-disk1.img +2G
Image resized.

Now we must create 2 files meta-data and user-data, the latter will contain the cloud-config settings.

{ echo instance-id: iid-local01; echo local-hostname: autopkgtest; } > meta-data

PUBKEY=$(cat ~/.ssh/id_rsa.pub )

cat > user-data << EOF
#cloud-config
locale: en_US.UTF-8

password: ubuntu
chpasswd: { expire: False }
ssh_pwauth: True

ssh_authorized_keys:
 - $PUBKEY

apt_update: true
apt_upgrade: true
byobu_by_default: system

packages:
 - eatmydata
 - autopkgtest
 - dpkg-dev
 - pbuilder
EOF

meta-data and user-data will be loaded from a VFAT or an ISO9660 filesystem. We will create an ISO:

$ genisoimage  -output seed.iso -volid cidata -joliet -rock user-data meta-data
I: -input-charset not specified, using utf-8 (detected in locale settings)
Total translation table size: 0
Total rockridge attributes bytes: 331
Total directory bytes: 0
Path table size(bytes): 10
Max brk space used 0
183 extents written (0 MB)

All the pieces are here to provision our pristine test environment. It can be started it with kvm:

$ kvm -m 1024 -smp 2 -net nic -net user -redir tcp:54322::22 -drive file=quantal-server-cloudimg-amd64-disk1.img,if=virtio -drive file=seed.iso,if=virtio

This will start a Qemu window. After the boot sequence you should see and prompt with the hostname ‘autopkgtest’. You can login with the account with setup in user-data file: ubuntu/ubuntu
Alternatively you can ssh to the machine with:

$ ssh -i ~/.ssh/id_rsa -p 54322 -l ubuntu localhost

Once logged in run the following command to verify that the filesystem has been resized to 4G

$ df -h /dev/vda1

You can follow the progress of cloud-init in the log file /var/log/cloud-init.log (the example here takes some time to install due to the number of packages) You’ll know when the installation is finished when the file /var/lib/cloud/instance/boot-finished is created. It contains the time it took to setup the machine.

$ cat /var/lib/cloud/instance/boot-finished
283.51

Once the file is there you can verify that dpkg-dev has been installed for example:

$ apt-cache policy dpkg-dev
dpkg-dev:
  Installed: 1.16.1.2ubuntu8
  Candidate: 1.16.1.2ubuntu8

Finally, poweroff the VM

$ sudo poweroff

We have setup a pristine VM in less than 5 minutes. Now we can use it as our base VM to run autopkgtest for example.
Create a new qcow image to boot, backed by your original image

$ qemu-img create -f qcow2 -b quantal-server-cloudimg-amd64-disk1.img quantal-amd64-apport.imgFormatting 'quantal-amd64-apport.img', fmt=qcow2 size=4294967296 backing_file='quantal-server-cloudimg-amd64-disk1.img' encryption=off cluster_size=65536

Start it:

$ kvm -m 1024 -smp 2 -net nic -net user -redir tcp:54322::22 -drive file=quantal-amd64-apport.img,if=virtio

Wait until it boots (DataSourceEc2.py tries to reach http://169.254.169.254/2009-04-04/meta-data/instance-id for 2 minutes then gives up) then login. You are all set and ready to run autopkgtest.
Inside the VM run:

$ sudo -i
# apt-get source -d apport 
[...]
# adt-run apport_2.2.3*dsc --- adt-virt-null

For details about cloud-init and cloud-config you can read the documentation shipped with the source of cloud-init:
http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/files/head:/doc/
http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt

Advertisements