Good Bye BlogFranz and 20th Century

Hello franzgedank.

Finally decided to give WordPress and try and leave this klunky place behind.

The chroot Hack to making Debian kFreeBSD a Usable Firewall

A Rant Against Debian kFreeBSD
Don't get me wrong, I love the fact that there is a Debian distribution based on FreeBSD but if you are going to mention PF as a reason for using it, you should at least have a version of tcpdump that can read packet drops and build of pflogd so they can be logged. How useful is a firewall that you can't even tell when there are policy violations?

Not very. So I'd run across this blog and even sort of tried to apply the patches. Yes, that is the right answer (and I've been meaning to build my own Debian kFreeBSD package repository, but I'm lazy) but I today on the kFreeBSD FAQ that showed how you could run a native FreeBSD binaries in a chroot. Incidentally you can do the reverse as well, meaning run Debian *BSD chroots on top of the real FreeBSD.

Here is a quick and easy way to get tcpdump and pflogd working on a Debian kFreeBSD without having to compile anything.

Preparation
I won't repeat all the instructions from the FAQ because they just work but here are the high level steps:

1. Pull down the .iso for FreeBSD 8.2 (obviously pick the right architecture)
2. Mount the iso using mdconfig (this is basically like loopback filesystem)
3. Extract the sources from the install CD
4. Strip out the libraries and files that you need

I decided to create an /opt/native for my chroot jail

/opt/native/etc:

total 92

drwxr-xr-x 2 root root 512 Aug 14 13:26 .

drwxr-xr-x 10 root root 512 Aug 14 13:21 ..

-rw-r--r-- 1 root root 1667 Aug 14 13:25 aliases

-rw-r--r-- 1 root root 429 Aug 14 13:21 group

-rw------- 1 root root 1433 Aug 14 13:25 master.passwd

-rw-r--r-- 1 root root 1329 Aug 14 13:22 passwd

-rw-r--r-- 1 root root 40960 Aug 14 13:26 pwd.db

-rw------- 1 root root 40960 Aug 14 13:26 spwd.db

/opt/native/lib:

total 2848

drwxr-xr-x 2 root root 512 Aug 14 13:12 .

drwxr-xr-x 10 root root 512 Aug 14 13:21 ..

-r--r--r-- 1 root root 1432616 Aug 14 12:08 libcrypto.so.6

-r--r--r-- 1 root root 32104 Aug 14 12:08 libcrypt.so.5

-r--r--r-- 1 root root 1155172 Aug 14 12:09 libc.so.7

-r--r--r-- 1 root root 182668 Aug 14 12:08 libpcap.so.7

-r--r--r-- 1 root root 56832 Aug 14 13:12 libutil.so.8

/opt/native/libexec:

total 244

drwxr-xr-x 2 root root 512 Aug 14 12:15 .

drwxr-xr-x 10 root root 512 Aug 14 13:21 ..

-r-xr-xr-x 1 root root 220628 Aug 14 12:15 ld-elf.so.1

/opt/native/sbin:

total 24

drwxr-xr-x 2 root root 512 Aug 14 12:11 .

drwxr-xr-x 10 root root 512 Aug 14 13:21 ..

-r-xr-xr-x 1 root root 18924 Aug 14 12:11 pflogd

/opt/native/usr:

total 8

drwxr-xr-x 4 root root 512 Aug 14 12:06 .

drwxr-xr-x 10 root root 512 Aug 14 13:21 ..

drwxr-xr-x 2 root root 512 Aug 14 12:06 bin

drwxr-xr-x 2 root root 512 Aug 14 12:10 sbin

/opt/native/usr/bin:

total 4

drwxr-xr-x 2 root root 512 Aug 14 12:06 .

drwxr-xr-x 4 root root 512 Aug 14 12:06 ..

/opt/native/usr/sbin:

total 644

drwxr-xr-x 2 root root 512 Aug 14 12:10 .

drwxr-xr-x 4 root root 512 Aug 14 12:06 ..

-r-xr-xr-x 1 root root 627792 Aug 14 12:10 tcpdump

/opt/native/var:

total 10

drwxr-xr-x 5 root root 512 Aug 14 13:24 .

drwxr-xr-x 10 root root 512 Aug 14 13:21 ..

drwxr-xr-x 2 root root 512 Aug 14 13:23 empty

drwxr-xr-x 2 root root 512 Aug 14 13:26 log

drwxr-xr-x 2 root root 512 Aug 14 13:30 run

/opt/native/var/empty:

total 4

drwxr-xr-x 2 root root 512 Aug 14 13:23 .

drwxr-xr-x 5 root root 512 Aug 14 13:24 ..

/opt/native/var/log:

total 10

drwxr-xr-x 2 root root 512 Aug 14 13:26 .

drwxr-xr-x 5 root root 512 Aug 14 13:24 ..

-rw------- 1 root root 4290 Aug 14 14:12 pflog

/opt/native/var/run:

total 6

drwxr-xr-x 2 root root 512 Aug 14 13:30 .

drwxr-xr-x 5 root root 512 Aug 14 13:24 ..

-rw-r--r-- 1 root root 5 Aug 14 13:30 pflogd.pid

Create your chroot scripts on the host filesystem

They should look like this:

root@debian:/opt# ls -al /usr/local/bin/native*

-rwxr--r-- 1 root staff 74 Aug 14 13:30 /usr/local/bin/nativepflogd

-rwxr--r-- 1 root staff 47 Aug 14 12:12 /usr/local/bin/nativetcpdump

and inside

root@debian:/opt# cat /usr/local/bin/native*

exec chroot /opt/native/ /sbin/pflogd -s 250 -i pflog0 -f /var/log/pflog

exec chroot /opt/native /usr/sbin/tcpdump "$@"

Make sure the chroot jail can get to devices

# mount -t devfs devfs /opt/native/dev

This should show up as

# mount
devfs on /opt/native/dev (devfs, local, multilabel)

Confirm that it actually works

You can start pflogd by running /usr/local/bin/nativpflogd

root@debian:/opt# ps aux | grep pflog

root 9115 0.0 0.1 3240 1264 ? S+ 14:18 0:00 grep pflog

64 8965 0.0 0.1 9680 1508 ? S 13:30 0:00 pflogd: [running] -s 250 -i pflog0 -f /var/log/pflog

root 8964 0.0 0.1 9680 1448 ? Ss 13:30 0:00 pflogd: [priv]

root@debian:/opt#

# nativetcpdump -nr /var/log/pflog

reading from file /var/log/pflog, link-type PFLOG (OpenBSD pflog file)

17:30:47.166732 IP 192.168.56.1.64248 > 192.168.56.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST

17:30:47.437045 IP 192.168.56.1.64248 > 192.168.56.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST

17:30:47.707640 IP 192.168.56.1.64248 > 192.168.56.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST

The 5 Minute Guide to Running OpenVZ on CentOS 6.x

As much as I like Debian (and the fact that it has OpenVZ kernels in the repos, for now...) CentOS is really the best distribution for running this container-based virtualization environment. This assumes a minimal CentOS 6.0 install.

1) Add the OpenVZ repos

See the Quick Installation guide and ensure that:

[root@opti330 yum.repos.d]# cat /etc/yum.repos.d/openvz.repo
[openvz-utils]
name=OpenVZ utilities
#baseurl=http://download.openvz.org/current/
mirrorlist=http://download.openvz.org/mirrors-current
enabled=1
gpgcheck=1
gpgkey=http://download.openvz.org/RPM-GPG-Key-OpenVZ

[openvz-kernel-rhel6]
name=OpenVZ RHEL6-based kernel
#baseurl=http://download.openvz.org/kernel/branches/rhel6-2.6.32/current/
mirrorlist=http://download.openvz.org/kernel/mirrors-rhel6-2.6.32
enabled=1
gpgcheck=1
gpgkey=http://download.openvz.org/RPM-GPG-Key-OpenVZ

2) Install the packages (after updating of course)

# yum install openvz-kernel-rhel6 vzctl vzquota bridge-utils

3) Update /etc/sysctl.conf

net.ipv4.ip_forward = 1
net.ipv4.conf.all.proxy_arp = 1

This ARP proxying is really only needed if you are doing veth networking (the default we'll use below)

4) Reboot

You kernel should now be:

[root@opti330 yum.repos.d]# uname -a
Linux opti330 2.6.32-042stab024.1 #1 SMP Tue Jul 26 15:23:12 MSD 2011 x86_64 x86_64 x86_64 GNU/Linux

5) Update /etc/sysconfig/iptables to allow traffic to/from venet0 (only if you are using venet)

-A FORWARD -i venet0 -o eth0 -j ACCEPT

-A FORWARD -i eth0 -o venet0 -j ACCEPT

Or something close. You will also want to update your iptables policy or disable it or disable it.

6) Now you can create your VE's per the instructions on my OpenVZ Wiki Page.