Setting Up Linux Cgroups

Last week’s column introduced Linux Control Groups (or cgroups), a feature initially developed to limit resource usage in the Linux kernel. But it can do much more, including tweak memory, bandwidth and CPU usage of system processes as well as deny access to system resources. This week, let’s delve a bit deeper and actually set up and manage a cgroup.

One of the reasons I’m writing about this topic is that, while useful, it seems vastly under-used and definitely could be simpler. Right now it’s as user friendly as SELinux. But if more folks use the feature, it will get more attention — and likely improve.

On Fedora 14, you’ll want the libcgroup package (yum install libcgroup). On Ubuntu 10.10, you’ll want the cgroup-bin and libcgroup1 packages. For the purposes of this column, I’m using Fedora 14 — you will find some differences between Fedora and Ubuntu. The kernel and userland utilities should work the same way, but the default configurations are different. Fedora automatically mounts all the controllers under /cgroup/controller name, while Ubuntu has mounts only for cpu, cpuacct, and devices under /mnt/cgroups.

Simple Configuration of Cgroups

Let’s start by confirming that the service starts as advertised. On F14, run service cgconfig start and check under /cgroup/ that the requisite directories are created. Dive under the individual directories, and you’ll see the parameters that can be tuned. For example, under /cgroup/memory you’ll see the following files:

cgroup.event_control
cgroup.procs
memory.failcnt
memory.force_empty
memory.limit_in_bytes
memory.max_usage_in_bytes
...

There’s more than that, but you get the idea. If you want to create a limit for maximum memory usage, it’s stored in a file called memory.max_usage_in_bytes. (Remember, everything is a file.)

Let’s say you want to limit the memory provided to a daemon and all the processes it spawns. How about Apache? First you’ll create a group statement in /etc/cgconfig.conf like so:

group http {
	memory {
		memory.limit_in_bytes = 1024M;
	}
}

Next, add this to the /etc/sysconfig/httpd.conf:

CGROUP_DAEMON="memory:/http"

You should be good to go. Just start the cgconfig service and then the httpd service.

I could write much more about cgroups, and I may return to the topic again if there’s sufficient interest — let me know in the comments. In the meantime, the Fedora wiki is a good source of info on the topic, despite being written ahead of the Fedora 11 release (as of this writing, anyway), so some bits may be slightly out of date.