Sunday, August 14, 2011

The chroot Hack to making Debian kFreeBSD a Usable Firewall

A Rant Against Debian kFreeBSD
Don't get me wrong, I love the fact that there is a Debian distribution based on FreeBSD but if you are going to mention PF as a reason for using it, you should at least have a version of tcpdump that can read packet drops and build of pflogd so they can be logged. How useful is a firewall that you can't even tell when there are policy violations?

Not very. So I'd run across this blog and even sort of tried to apply the patches. Yes, that is the right answer (and I've been meaning to build my own Debian kFreeBSD package repository, but I'm lazy) but I today on the kFreeBSD FAQ that showed how you could run a native FreeBSD binaries in a chroot. Incidentally you can do the reverse as well, meaning run Debian *BSD chroots on top of the real FreeBSD.

Here is a quick and easy way to get tcpdump and pflogd working on a Debian kFreeBSD without having to compile anything.

Preparation
I won't repeat all the instructions from the FAQ because they just work but here are the high level steps:
  1. Pull down the .iso for FreeBSD 8.2 (obviously pick the right architecture)
  2. Mount the iso using mdconfig (this is basically like loopback filesystem)
  3. Extract the sources from the install CD
  4. Strip out the libraries and files that you need

I decided to create an /opt/native for my chroot jail


/opt/native/etc:
total 92
drwxr-xr-x 2 root root 512 Aug 14 13:26 .
drwxr-xr-x 10 root root 512 Aug 14 13:21 ..
-rw-r--r-- 1 root root 1667 Aug 14 13:25 aliases
-rw-r--r-- 1 root root 429 Aug 14 13:21 group
-rw------- 1 root root 1433 Aug 14 13:25 master.passwd
-rw-r--r-- 1 root root 1329 Aug 14 13:22 passwd
-rw-r--r-- 1 root root 40960 Aug 14 13:26 pwd.db
-rw------- 1 root root 40960 Aug 14 13:26 spwd.db

/opt/native/lib:
total 2848
drwxr-xr-x 2 root root 512 Aug 14 13:12 .
drwxr-xr-x 10 root root 512 Aug 14 13:21 ..
-r--r--r-- 1 root root 1432616 Aug 14 12:08 libcrypto.so.6
-r--r--r-- 1 root root 32104 Aug 14 12:08 libcrypt.so.5
-r--r--r-- 1 root root 1155172 Aug 14 12:09 libc.so.7
-r--r--r-- 1 root root 182668 Aug 14 12:08 libpcap.so.7
-r--r--r-- 1 root root 56832 Aug 14 13:12 libutil.so.8

/opt/native/libexec:
total 244
drwxr-xr-x 2 root root 512 Aug 14 12:15 .
drwxr-xr-x 10 root root 512 Aug 14 13:21 ..
-r-xr-xr-x 1 root root 220628 Aug 14 12:15 ld-elf.so.1

/opt/native/sbin:
total 24
drwxr-xr-x 2 root root 512 Aug 14 12:11 .
drwxr-xr-x 10 root root 512 Aug 14 13:21 ..
-r-xr-xr-x 1 root root 18924 Aug 14 12:11 pflogd

/opt/native/usr:
total 8
drwxr-xr-x 4 root root 512 Aug 14 12:06 .
drwxr-xr-x 10 root root 512 Aug 14 13:21 ..
drwxr-xr-x 2 root root 512 Aug 14 12:06 bin
drwxr-xr-x 2 root root 512 Aug 14 12:10 sbin

/opt/native/usr/bin:
total 4
drwxr-xr-x 2 root root 512 Aug 14 12:06 .
drwxr-xr-x 4 root root 512 Aug 14 12:06 ..

/opt/native/usr/sbin:
total 644
drwxr-xr-x 2 root root 512 Aug 14 12:10 .
drwxr-xr-x 4 root root 512 Aug 14 12:06 ..
-r-xr-xr-x 1 root root 627792 Aug 14 12:10 tcpdump

/opt/native/var:
total 10
drwxr-xr-x 5 root root 512 Aug 14 13:24 .
drwxr-xr-x 10 root root 512 Aug 14 13:21 ..
drwxr-xr-x 2 root root 512 Aug 14 13:23 empty
drwxr-xr-x 2 root root 512 Aug 14 13:26 log
drwxr-xr-x 2 root root 512 Aug 14 13:30 run

/opt/native/var/empty:
total 4
drwxr-xr-x 2 root root 512 Aug 14 13:23 .
drwxr-xr-x 5 root root 512 Aug 14 13:24 ..

/opt/native/var/log:
total 10
drwxr-xr-x 2 root root 512 Aug 14 13:26 .
drwxr-xr-x 5 root root 512 Aug 14 13:24 ..
-rw------- 1 root root 4290 Aug 14 14:12 pflog

/opt/native/var/run:
total 6
drwxr-xr-x 2 root root 512 Aug 14 13:30 .
drwxr-xr-x 5 root root 512 Aug 14 13:24 ..
-rw-r--r-- 1 root root 5 Aug 14 13:30 pflogd.pid

Create your chroot scripts on the host filesystem

They should look like this:

root@debian:/opt# ls -al /usr/local/bin/native*
-rwxr--r-- 1 root staff 74 Aug 14 13:30 /usr/local/bin/nativepflogd
-rwxr--r-- 1 root staff 47 Aug 14 12:12 /usr/local/bin/nativetcpdump

and inside

root@debian:/opt# cat /usr/local/bin/native*
exec chroot /opt/native/ /sbin/pflogd -s 250 -i pflog0 -f /var/log/pflog
exec chroot /opt/native /usr/sbin/tcpdump "$@"


Make sure the chroot jail can get to devices

# mount -t devfs devfs /opt/native/dev

This should show up as

# mount
devfs on /opt/native/dev (devfs, local, multilabel)

Confirm that it actually works

You can start pflogd by running /usr/local/bin/nativpflogd

root@debian:/opt# ps aux | grep pflog
root 9115 0.0 0.1 3240 1264 ? S+ 14:18 0:00 grep pflog
64 8965 0.0 0.1 9680 1508 ? S 13:30 0:00 pflogd: [running] -s 250 -i pflog0 -f /var/log/pflog
root 8964 0.0 0.1 9680 1448 ? Ss 13:30 0:00 pflogd: [priv]
root@debian:/opt#

# nativetcpdump -nr /var/log/pflog
reading from file /var/log/pflog, link-type PFLOG (OpenBSD pflog file)
17:30:47.166732 IP 192.168.56.1.64248 > 192.168.56.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
17:30:47.437045 IP 192.168.56.1.64248 > 192.168.56.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
17:30:47.707640 IP 192.168.56.1.64248 > 192.168.56.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST