Creating a Gentoo Cloud Image

Introduction

Along with getting Gentoo running in Rackspace, I've also been working on creating a proper KVM-based Gentoo cloud image. This article details my steps so far.

Introduction
Creating the Base Virtual Machine
Installing and Configuring Gentoo
Kernel Configuration
- Automatic Resizing and initramfs
- Building the Kernel
Serial Console
Final Steps
Shrinking the Virtual Disk
Importing the Image
Conclusion

Creating the Base Virtual Machine

To begin, download a minimal Gentoo ISO. This ISO will allow you to boot into a live Gentoo environment and work on a local installation with the added ability to mount and umount the root partition.

Next, create a basic KVM-based virtual machine. You can use any tool you'd like for this. I personally use the virt-manager utility as I find the GUI interface easier to work with when it comes to KVM and Qemu options.

The virtual hard disk should be approximately 10gb in size. Although I'll explain how to shrink the disk to a much smaller size later, 10gb will allow sufficient room to work with. Use virtio for both the disk and NIC driver.

With all of these items in place, boot the virtual machine using the ISO as the boot device. You'll soon be at a root prompt. Feel free to start the ssh service so you can access the virtual machine from a more comfortable terminal.

Now partition and format /dev/vda. I recommend just a single partition that will be mounted as root. You'll be able to access this partition through /dev/vda1. The choice of filesystem is up to you. I've been using ext4 for now. I'd like to look into building a ZFS-based cloud image, though.

Installing and Configuring Gentoo

With /dev/vda1 ready to go, install Gentoo. I use a modified version of this installation script which you can find here. Make sure you thoroughly read through it, remove any unneeded steps, and modify any parts as needed – this is not a one-size-fits-all script.

Once the script has finished, chroot into the environment. /mnt/gentoo should already be mounted, but if it's not, do the following:

$ mount /dev/vda1 /mnt/gentoo
$ for i in proc dev sys
> do
> mount --rbind /$i /mnt/gentoo/$i
> done
$ chroot /mnt/gentoo /bin/bash

Once logged in, set a root password (the password will be removed as one of the last steps):

$ passwd

Next, enable net.eth0 to start at boot. Unless you made any changes to the installation script, eth0 will be configured for DHCP which I recommend for a cloud image. Also set ssh to start at boot, too.

$ cd /etc/init.d
$ ln -s net.lo net.eth0
$ rc-update add net.eth0 default
$ rc-update add sshd default

In order for the cloud image to receive power commands such as "power off" and "reboot", acpid must be installed:

$ emerge acpid
$ rc-update add acpid default

Set the timezone. I chose to use standard GMT:

$ cp /usr/share/zoneinfo/GMT /etc/localtime
$ echo GMT > /etc/timezone

Now install any utilities you'd like to have on all of your Gentoo instances.

$ emerge -a tmux vim git genkernel

Kernel Configuration

Next, configure the kernel. You need to make sure that all virtio-related options are enabled. You can easily do this by editing the /usr/src/linux/.config file rather than navigating through the make menuconfig interface.

Automatic Resizing and initramfs

In cloud environments, you have the ability to choose different "flavors" (such as "m1.tiny" or "m1.xlarge"). These flavors usually have different root disk sizes. Xen-based cloud environments automatically handle resizing the root disk contained in the image to match the root disk size chosen in the flavor. KVM-based environments, however, don't, so your image must handle this task itself. If it doesn't, then your cloud instance's root disk will be the same size no matter what flavor you choose – the original size from when the image was imported into the cloud environment.

There are two parts to resizing:

Modifying /dev/vda's partition table to account for the new size.
Resizing the filesystem to account for the new partition's size.

These tasks are normally done in initramfs before /dev/vda1 is mounted. An initramfs overlay can be created to alter the initramfs image and add a few extra steps.

I have published my initramfs overlay here. This overlay provides all utilities needed to perform the disk resizing. See the bin directory for the included commands. Also see line 607 of the linuxrc file to see what steps are being performed in order to do the resize.

Building the Kernel

With the custom initramfs ready to go, build the kernel:

$ genkernel --linuxrc=/root/initramfs/linuxrc --initramfs-overlay=/root/initramfs/overlay --install all

This takes care of the kernel as well as the first part of resizing a cloud image.

Serial Console

For OpenStack-based clouds, you can see the console log either through the Horizon web interface or via the nova console-log command. You need to enable a serial console to do this, though. First, edit /boot/grub/menu.lst and add:

serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=5 serial console

Then modify the kernel entry by appending console=tty0 console=ttyS0,115200n8 to the kernel line:

title Gentoo Linux 2.8.13
root (hd0,0)
  kernel /boot/kernel-genkernel-x86_64-3.8.13-gentoo root=/dev/vda1 console=tty0 console=ttyS0,115200n8
  initrd /boot/initramfs-genkernel-x86_64-3.8.13-gentoo

Second, uncomment ttyS0 in /etc/inittab.

Final Steps

Most cloud environments pull information down from the cloud metadata server. Examples of information are SSH keys and the hostname. Copy this script to /etc/local.d/ to enable this.

Next, break out of the chroot environment and clean the system up:

$ exit
$ rm -rf /mnt/gentoo/var/log/*
$ echo > /mnt/gentoo/root/.bash_history
$ rm /mnt/gentoo/etc/ssh/sshd_host*
$ rm -rf /mnt/gentoo/usr/portage/distfiles/*
$ vi /mnt/gentoo/etc/shadow (remote root's password)

Shrinking the Virtual Disk

At this point, your Gentoo installation should be approximately 3gb in size (df -h). If you installed several extra packages, the size will be larger. I highly recommend making sure your installation is less than 5gb, though. This will allow you to boot into the smallest flavor in most cloud environments.

As created at the beginning of this article, the virtual disk is 10gb in size. You need to change this to be less than 5gb.

First, unmount /mnt/gentoo and all other mounted file systems.

Next, shut down the live ISO environment.

Finally, shrink the image. virt-sparsify is a utility that can help with this:

$ virt-sparsify --convert qcow2 /var/lib/libvirt/images/gentoo.img gentoo-shrunk.img

Importing the Image

The resulting gentoo.img (or whatever you named the virtual disk) is now ready to be imported into a cloud environment. The total size of the gentoo.img file should be approximately 3gb.

Import gentoo.img into the cloud environment(s) you'll want to use Gentoo in. For example, use Glance to import the image into OpenStack:

$ glance image-create --name "Gentoo 20130707" --is-public false --disk-format qcow2 --container-format bare < gentoo.img

Conclusion

This article detailed the steps I've used to create a Gentoo cloud image from scratch. It walked through the process of creating a KVM-based virtual machine, installing and configuring Gentoo, configuring support for cloud-specific features such as a growing root disk and cloud-init, and finally, importing the resulting image into a cloud environment.

I'm interested to learn if there are better ways to accomplish any of these steps – please feel free to comment.

terrarum