Building an LXC Server - Ubuntu 14.04 Edition

Introduction

This article is a basic step-by-step HOWTO to create a server capable of hosting LXC-based containers.

Introduction
Prerequisites and Dependencies
Install LXC
Configure LXC
- Back to ZFS
Using LXC
- Creating a Container
- Testing ZFS Deduplication
Port Forwarding
- lxc-nat
Conclusion

Prerequisites and Dependencies

This server will be using Ubuntu 14.04. As 14.04 has just been released, some steps might change in the future.

apt Update

First, make sure all of the base packages are up to date:

$ sudo apt-get update
$ sudo apt-get dist-upgrade

Open vSwitch

The previous version of this article advocated Open vSwitch. I have since stopped using OVS as I've been able to configure Linux Bridge with the exact same features by using newer kernels.

ZFS

The newer LXC builds support ZFS as a backing store. This means that deduplication, compression, and snapshotting can all be taken advantage of. To install ZFS on Ubuntu 14.04, do:

$ sudo apt-add-repository ppa:zfs-native/daily
$ sudo apt-get update
$ sudo apt-get install ubuntu-zfs

Install LXC

Ubuntu 14.04 provides LXC 1.0.3, which is the latest version. I'm not sure if Ubuntu 14.04 will continue providing up-to-date versions of LXC, given it being an LTS release. If it falls behind, it might be beneficial to switch to the ubuntu-lxc/daily ppa.

To install LXC, just do:

$ sudo apt-get install lxc

Configure LXC

Back to ZFS

By default, LXC will look for a zpool titled lxc:

$ sudo zpool create -f tank /dev/vdc

Make sure deduplication and compression are turned on:

$ sudo zfs set dedup=on tank
$ sudo zfs set compression=on tank

LXC can use ZFS's native snapshot features. To make sure you can see snapshots when running zfs list, do the following:

$ sudo zpool set listsnapshots=on tank

To configure LXC to use ZFS as the backing store and set the default LXC path, add the following to /etc/lxc/lxc.conf:

lxc.lxcpath = /tank/lxc/containers
lxc.bdev.zfs.root = tank/lxc/containers

Ensure /tank/lxc/containers, or whichever path you choose, exists:

$ sudo zfs create tank/lxc
$ sudo zfs create tank/lxc/containers

Using LXC

Creating a Container

Create the first container by doing:

$ sudo lxc-create -t ubuntu -n test1 -B zfs -- -S /root/.ssh/id_rsa.pub

When the command has finished, you'll see that LXC has created a new ZFS partition:

$ sudo zfs list
$ df -h

Testing ZFS Deduplication

You can see the ZFS dedup stat by doing:

$ sudo zpool list
NAME   SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
tank  99.5G   186M  99.3G     0%  1.01x  ONLINE  -

With that number in mind, create a second container:

$ sudo lxc-create -t ubuntu -n test2 -B zfs -- -S /root/.ssh/id_rsa.pub

When the command has finished, review the ZFS stat:

$ sudo zpool list
NAME   SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
tank  99.5G   187M  99.3G     0%  2.02x  ONLINE  -

The dedup ratio doubled. This effectively means that no new disk space was consumed when the new container was created!

Port Forwarding

By default, LXC uses the veth networking mode for containers. This is the most robust networking mode. Other modes exist and I highly recommend this article for a detailed look at them.

veth mode can be thought of as a form of NAT and the LXC server is now acting as a NAT'd gateway for all of the containers running on the server. If you want the containers to be accessible from the public internet, you will need to do some port forwarding.

lxc-nat

Update: Zan Loy has made a much better version of the lxc-nat script mentioned below. The improved version can be found here.

Update 2: Daniël created a Python version of lxc-nat which can be found here.

I have put together a small script called lxc-nat that will configure port forwarding based on entries made in /etc/lxc/lxc-nat.conf.

For example, if you have Apache running in a container called www, create the following entry:

10.0.0.1:80 -> www:80

Or if you want to access www via SSH:

10.0.0.1:2201 -> www:22

Conclusion

This article showed the steps used to configure a server to host LXC-based containers on a ZFS storage backend.

terrarum