Building an LXC Server

Introduction

This article is a basic step-by-step HOWTO to create a server capable of hosting LXC-based containers.

This article has been superseded by Building an LXC Server - Ubuntu 14.04 Edition

Introduction
Prerequisites and Dependencies
Install LXC
Configure LXC
- More ZFS
- LXC Network Settings
Using LXC
- Creating a Base Container
- Cloning the Base Container
Port Forwarding
- lxc-nat
Conclusion

Prerequisites and Dependencies

This server will be using Ubuntu 12.04. Since 12.04 is getting pretty old, some work will be done to bring it up to date and configure it to use more modern components.

apt Update

First, make sure all of the base packages are up to date:

$ sudo apt-get update
$ sudo apt-get dist-upgrade

Kernel Update

Ubuntu has backported the 3.8 kernel from Ubuntu 13.04 to 12.04. To install it, do:

$ sudo apt-get install linux-generic-lts-raring
$ sudo reboot

Open vSwitch

LXC doesn't natively use Open vSwitch, but I like having it installed as a more modern replacement to the standard Linux kernel bridging. Ubuntu 12.04 ships with Open vSwitch 1.4 which is really old. To install 1.9, do:

$ sudo apt-add-repository ppa:sgran/openvswitch-precise
$ sudo apt-get update
$ sudo apt-get install openvswitch-datapath-lts-raring-dkms openvswitch-brcompat openvswitch-switch
$ sudo echo BRCOMPAT=yes >> /etc/default/openvswitch-switch
$ sudo reboot

ZFS

The newer LXC builds support ZFS as a backing store. This means that deduplication, compression, and snapshotting can all be taken advantage of. To install ZFS on Ubuntu 12.04, do:

$ sudo apt-add-repository ppa:zfs-native/daily
$ sudo apt-get update
$ sudo apt-get install ubuntu-zfs

Install LXC

I recommend adding an apt repository that will have the daily LXC builds:

$ sudo apt-add-repository ppa:ubuntu-lxc/daily
$ sudo apt-get update

Once the repository is added, installing LXC is as simple as:

$ sudo apt-get install lxc

Configure LXC

More ZFS

By default, LXC will look for a zpool titled lxc:

$ sudo zpool create -f lxc /dev/vdc
$ sudo rmdir /var/lib/lxc
$ sudo ln -s /lxc /var/lib/

Make sure deduplication and compression are turned on:

$ sudo zfs set dedup=on lxc
$ sudo zfs set compression=on lxc

LXC can use ZFS's native snapshot features. To make sure you can see snapshots when running zfs list, do the following:

$ sudo zpool set listsnapshots=on lxc

LXC Network Settings

Next, review the settings in /etc/default/lxc-net, such as the default NAT'd subnet. If you make any changes, restart the lxc-net service:

$ sudo /etc/init.d/lxc-net restart

Using LXC

Creating a Base Container

To take advantage of ZFS's deduplication, create a base container:

$ sudo lxc-create -t ubuntu -n ubuntu-base -B zfs -- -S /root/.ssh/id_rsa.pub

When the command has finished, you'll see that LXC has created a new ZFS partition:

$ sudo zfs list
$ df -h

Cloning the Base Container

Now when you want to launch an Ubuntu container, just clone the base container:

$ sudo lxc-clone -B zfs -s ubuntu-base mysql-server

When the command completes, you'll see that LXC has created a ZFS snapshot in addition to another ZFS partition. Also, you can see that almost no extra disk space was used due to deduplication:

$ sudo zfs list
$ sudo zpool list

Port Forwarding

By default, LXC uses the veth networking mode for containers. This is the most robust networking mode. Other modes exist and I highly recommend this article for a detailed look at them.

veth mode can be thought of as a form of NAT and the LXC server is now acting as a NAT'd gateway for all of the containers running on the server. If you want the containers to be accessible from the public internet, you will need to do some port forwarding.

lxc-nat

I have put together a small script called lxc-nat that will configure port forwarding based on entries made in /etc/lxc/lxc-nat.conf.

For example, if you have Apache running in a container called www, create the following entry:

10.0.0.1:80 -> www:80

Or if you want to access www via SSH:

10.0.0.1:2201 -> www:22

Conclusion

This article showed the steps used to configure a server to host LXC-based containers. It used the latest Ubuntu 12.04 kernel and up to date versions of Open vSwitch, ZFS, and LXC itself.

terrarum