The chroot Hack to making Debian kFreeBSD a Usable Firewall

A Rant Against Debian kFreeBSD
Don't get me wrong, I love the fact that there is a Debian distribution based on FreeBSD but if you are going to mention PF as a reason for using it, you should at least have a version of tcpdump that can read packet drops and build of pflogd so they can be logged. How useful is a firewall that you can't even tell when there are policy violations?

Not very. So I'd run across this blog and even sort of tried to apply the patches. Yes, that is the right answer (and I've been meaning to build my own Debian kFreeBSD package repository, but I'm lazy) but I today on the kFreeBSD FAQ that showed how you could run a native FreeBSD binaries in a chroot. Incidentally you can do the reverse as well, meaning run Debian *BSD chroots on top of the real FreeBSD.

Here is a quick and easy way to get tcpdump and pflogd working on a Debian kFreeBSD without having to compile anything.

Preparation
I won't repeat all the instructions from the FAQ because they just work but here are the high level steps:

1. Pull down the .iso for FreeBSD 8.2 (obviously pick the right architecture)
2. Mount the iso using mdconfig (this is basically like loopback filesystem)
3. Extract the sources from the install CD
4. Strip out the libraries and files that you need

I decided to create an /opt/native for my chroot jail

/opt/native/etc:

total 92

drwxr-xr-x 2 root root 512 Aug 14 13:26 .

drwxr-xr-x 10 root root 512 Aug 14 13:21 ..

-rw-r--r-- 1 root root 1667 Aug 14 13:25 aliases

-rw-r--r-- 1 root root 429 Aug 14 13:21 group

-rw------- 1 root root 1433 Aug 14 13:25 master.passwd

-rw-r--r-- 1 root root 1329 Aug 14 13:22 passwd

-rw-r--r-- 1 root root 40960 Aug 14 13:26 pwd.db

-rw------- 1 root root 40960 Aug 14 13:26 spwd.db

/opt/native/lib:

total 2848

drwxr-xr-x 2 root root 512 Aug 14 13:12 .

drwxr-xr-x 10 root root 512 Aug 14 13:21 ..

-r--r--r-- 1 root root 1432616 Aug 14 12:08 libcrypto.so.6

-r--r--r-- 1 root root 32104 Aug 14 12:08 libcrypt.so.5

-r--r--r-- 1 root root 1155172 Aug 14 12:09 libc.so.7

-r--r--r-- 1 root root 182668 Aug 14 12:08 libpcap.so.7

-r--r--r-- 1 root root 56832 Aug 14 13:12 libutil.so.8

/opt/native/libexec:

total 244

drwxr-xr-x 2 root root 512 Aug 14 12:15 .

drwxr-xr-x 10 root root 512 Aug 14 13:21 ..

-r-xr-xr-x 1 root root 220628 Aug 14 12:15 ld-elf.so.1

/opt/native/sbin:

total 24

drwxr-xr-x 2 root root 512 Aug 14 12:11 .

drwxr-xr-x 10 root root 512 Aug 14 13:21 ..

-r-xr-xr-x 1 root root 18924 Aug 14 12:11 pflogd

/opt/native/usr:

total 8

drwxr-xr-x 4 root root 512 Aug 14 12:06 .

drwxr-xr-x 10 root root 512 Aug 14 13:21 ..

drwxr-xr-x 2 root root 512 Aug 14 12:06 bin

drwxr-xr-x 2 root root 512 Aug 14 12:10 sbin

/opt/native/usr/bin:

total 4

drwxr-xr-x 2 root root 512 Aug 14 12:06 .

drwxr-xr-x 4 root root 512 Aug 14 12:06 ..

/opt/native/usr/sbin:

total 644

drwxr-xr-x 2 root root 512 Aug 14 12:10 .

drwxr-xr-x 4 root root 512 Aug 14 12:06 ..

-r-xr-xr-x 1 root root 627792 Aug 14 12:10 tcpdump

/opt/native/var:

total 10

drwxr-xr-x 5 root root 512 Aug 14 13:24 .

drwxr-xr-x 10 root root 512 Aug 14 13:21 ..

drwxr-xr-x 2 root root 512 Aug 14 13:23 empty

drwxr-xr-x 2 root root 512 Aug 14 13:26 log

drwxr-xr-x 2 root root 512 Aug 14 13:30 run

/opt/native/var/empty:

total 4

drwxr-xr-x 2 root root 512 Aug 14 13:23 .

drwxr-xr-x 5 root root 512 Aug 14 13:24 ..

/opt/native/var/log:

total 10

drwxr-xr-x 2 root root 512 Aug 14 13:26 .

drwxr-xr-x 5 root root 512 Aug 14 13:24 ..

-rw------- 1 root root 4290 Aug 14 14:12 pflog

/opt/native/var/run:

total 6

drwxr-xr-x 2 root root 512 Aug 14 13:30 .

drwxr-xr-x 5 root root 512 Aug 14 13:24 ..

-rw-r--r-- 1 root root 5 Aug 14 13:30 pflogd.pid

Create your chroot scripts on the host filesystem

They should look like this:

root@debian:/opt# ls -al /usr/local/bin/native*

-rwxr--r-- 1 root staff 74 Aug 14 13:30 /usr/local/bin/nativepflogd

-rwxr--r-- 1 root staff 47 Aug 14 12:12 /usr/local/bin/nativetcpdump

and inside

root@debian:/opt# cat /usr/local/bin/native*

exec chroot /opt/native/ /sbin/pflogd -s 250 -i pflog0 -f /var/log/pflog

exec chroot /opt/native /usr/sbin/tcpdump "$@"

Make sure the chroot jail can get to devices

# mount -t devfs devfs /opt/native/dev

This should show up as

# mount
devfs on /opt/native/dev (devfs, local, multilabel)

Confirm that it actually works

You can start pflogd by running /usr/local/bin/nativpflogd

root@debian:/opt# ps aux | grep pflog

root 9115 0.0 0.1 3240 1264 ? S+ 14:18 0:00 grep pflog

64 8965 0.0 0.1 9680 1508 ? S 13:30 0:00 pflogd: [running] -s 250 -i pflog0 -f /var/log/pflog

root 8964 0.0 0.1 9680 1448 ? Ss 13:30 0:00 pflogd: [priv]

root@debian:/opt#

# nativetcpdump -nr /var/log/pflog

reading from file /var/log/pflog, link-type PFLOG (OpenBSD pflog file)

17:30:47.166732 IP 192.168.56.1.64248 > 192.168.56.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST

17:30:47.437045 IP 192.168.56.1.64248 > 192.168.56.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST

17:30:47.707640 IP 192.168.56.1.64248 > 192.168.56.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST

FreeBSD 8.0 with rum0 and wpa_supplicant on Lenovo S10-2

It looks like the driver for rum has changed slightly in FreeBSD 8.0 from FreeBSD 7.2 because I was not able to use the same command-line syntax as I did previously. Basically the only thing different I did was the ifconfig wlan create...

I had this card running on old Dell Optiplex acting as a bridge for my kids network (and they were watching a lot of streaming media) and I was surprisingly impressed with it. Decent performance.

mfranz-bsd8#
ugen4.3:  at usbus4
rum0:  on usbus4
rum0: MAC/BBP RT2573 (rev 0x2573a), RF RT2528

mfranz-bsd8# cat /etc/wpa_supplicant.conf
network={
 ssid="xxx"
 psk="xxxx"
}


mfranz-bsd8# ifconfig wlan create wlandev rum0
wlan0
mfranz-bsd8# ifconfig wlan0
wlan0: flags=8802 metric 0 mtu 1500
 ether 00:1c:10:e6:1a:02
 media: IEEE 802.11 Wireless Ethernet autoselect (autoselect)
 status: no carrier
 ssid "" channel 1 (2412 Mhz 11b)
 country US authmode OPEN privacy OFF txpower 0 bmiss 7 scanvalid 60
 bgscan bgscanintvl 300 bgscanidle 250 roam:rssi 7 roam:rate 1
 bintval 0
mfranz-bsd8#

mfranz-bsd8# wpa_supplicant -c /etc/wpa_supplicant.conf -i wlan0
CTRL-EVENT-SCAN-RESULTS
Trying to associate with xxxxxxxxxx (SSID='xxxxxxxx' freq=2437 MHz)
Associated with xxxxxxxxxxx
WPA: Key negotiation completed with xxxxxxxxxxx [PTK=CCMP GTK=TKIP]
CTRL-EVENT-CONNECTED - Connection to xxxxxxxxxx completed (auth) [id=0 id_str=]

And while I'm at it, I hadn't seen any who actually installed 8.0 on a Lenovo Netbook but so far so good. I've got X working (I'll blog on that later) and re seems to work well enough. Obviously the Broadcom 4312's aren't going to work, but if you have USB wifi card or a tether you will be ok.

Next step see if I can get my Novatel u727 card working. I suspect it should work just fine, because it worked well on OpenBSD, but you never know...

Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
       The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:48:17 UTC 2009
   root@almeida.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Atom(TM) CPU N270   @ 1.60GHz (1602.40-MHz 686-class CPU)
 Origin = "GenuineIntel"  Id = 0x106c2  Stepping = 2
 Features=0xbfe9fbff
 Features2=0x40c39d>
 AMD Features2=0x1
 TSC: P-state invariant
real memory  = 1073741824 (1024 MB)
avail memory = 1026433024 (978 MB)
ACPI APIC Table: 
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 1 core(s) x 2 HTT threads
cpu0 (BSP): APIC ID:  0
cpu1 (AP/HT): APIC ID:  1
ioapic0: Changing APIC ID to 4
ioapic0  irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0:  on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit> port 0x408-0x40b on acpi0
acpi_ec0:  port 0x62,0x66 on acpi0
acpi_hpet0:  iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
acpi_button0:  on acpi0
acpi_lid0:  on acpi0
acpi_button1:  on acpi0
pcib0:  port 0xcf8-0xcff on acpi0
pci0:  on pcib0
vgapci0:  port 0x60f0-0x60f7 mem 0x58280000-0x582fffff,0x40000000-0x4fffffff,0x58300000-0x5833ffff irq 16
at device 2.0 on pci0
agp0:  on vgapci0
agp0: detected 7932k stolen memory
agp0: aperture size is 256M
vgapci1:  mem 0x58200000-0x5827ffff at device 2.1 on pci0
pci0:  at device 27.0 (no driver attached)
pcib1:  at device 28.0 on pci0
pci1:  on pcib1
pcib2:  at device 28.1 on pci0
pci2:  on pcib2
pci2:  at device 0.0 (no driver attached)
pcib3:  at device 28.2 on pci0
pci3:  on pcib3
re0:  port 0x2000-0x20ff mem 0x52010000-0x52010fff,0x52000000-0x5200ffff irq 18 at
device 0.0 on pci3
re0: Using 1 MSI messages
re0: Chip rev. 0x24800000
re0: MAC rev. 0x00400000
miibus0:  on re0
rlphy0:  PHY 1 on miibus0
rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
re0: Ethernet address: 00:26:22:0b:07:28
re0: [FILTER]
pcib4:  at device 28.3 on pci0
pci4:  on pcib4
uhci0:  port 0x60a0-0x60bf irq 16 at device 29.0 on pci0
uhci0: [ITHREAD]
uhci0: LegSup = 0x0f00
usbus0:  on uhci0
uhci1:  port 0x6080-0x609f irq 17 at device 29.1 on pci0
uhci1: [ITHREAD]
uhci1: LegSup = 0x0f00
usbus1:  on uhci1
uhci2:  port 0x6060-0x607f irq 18 at device 29.2 on pci0
uhci2: [ITHREAD]
uhci2: LegSup = 0x0f00
usbus2:  on uhci2
uhci3:  port 0x6040-0x605f irq 19 at device 29.3 on pci0
uhci3: [ITHREAD]
uhci3: LegSup = 0x0f00
usbus3:  on uhci3
ehci0:  mem 0x58344400-0x583447ff irq 16 at device 29.7 on pci0
ehci0: [ITHREAD]
usbus4: EHCI version 1.0
usbus4:  on ehci0
pcib5:  at device 30.0 on pci0
pci5:  on pcib5
isab0:  at device 31.0 on pci0
isa0:  on isab0
atapci0:  port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x60c0-0x60cf irq 16 at device 31.1 on pci0
ata0:  on atapci0
ata0: [ITHREAD]
atapci1:  port 0x60d8-0x60df,0x60fc-0x60ff,0x60d0-0x60d7,0x60f8-0x60fb,0x6020-0x602f mem 0x583440
00-0x583443ff irq 17 at device 31.2 on pci0
atapci1: [ITHREAD]
atapci1: AHCI called from vendor specific driver
atapci1: AHCI v1.10 controller with 4 1.5Gbps ports, PM not supported
ata2:  on atapci1
ata2: [ITHREAD]
ata3:  on atapci1
ata3sm0:  irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: [ITHREAD]
psm0: model Generic PS/2 mouse, device ID 0
cpu0:  on acpi0
est0:  on cpu0
p4tcc0:  on cpu0
cpu1:  on acpi0
est1:  on cpu1
p4tcc1:  on cpu1
pmtimer0 on isa0
orm0:  at iomem 0xcf000-0xcffff pnpid ORM0000 on isa0
sc0:  at flags 0x100 on isa0
sc0: VGA <16 flags="0x300">
vga0:  at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: parallel port not found.
Timecounters tick every 1.000 msec
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 12Mbps Full Speed USB v1.0
usbus4: 480Mbps High Speed USB v2.0
ad4: 152627MB  at ata2-master SATA150
ugen0.1:  at usbus0
uhub0:  on usbus0
ugen1.1:  at usbus1
uhub1:  on usbus1
ugen2.1:  at usbus2
uhub2:  on usbus2
ugen3.1:  at usbus3
uhub3:  on usbus3
ugen4.1:  at usbus4
uhub4:  on usbus4
: [ITHREAD]
GEOM: ad4: partition 1 does not start on a track boundary.
GEOM: ad4: partition 1 does not end on a track boundary.
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
Root mount waiting for: usbus4
Root mount waiting for: usbus4
Root mount waiting for: usbus4
uhub4: 8 ports with 8 removable, self powered
Root mount waiting for: usbus4
Root mount waiting for: usbus4
ugen4.2:  at usbus4
Trying to mount root from ufs:/dev/ad4s2a
ugen0.2:  at usbus0
ums0:  on usbus0
ums0: 2 buttons and [XY] coordinates ID=0
drm0:  on vgapci0
vgapci0: child drm0 requested pci_enable_busmaster
info: [drm] AGP at 0x40000000 256MB
info: [drm] Initialized i915 1.6.0 20080730

Is conntrackd really pfsync+CARP for Linux?

Say it aint' so Joe, but conntrack-tools says it "provides and equivalent of OpenBSD's pfsync."

What can do the conntrack-tools for me?

Lots of cool things. conntrackd covers the specific aspects of stateful Linux firewalls to enable high availability solutions and it can be used as statistics collector of the firewall use as well. The command line interface conntrack provides an interface to add, delete and update flow entries, list current active flows in plain text/XML, current IPv4 NAT'ed flows, reset counters atomically, flush the connection tracking table and monitor connection tracking events among many other.

This is something I've been wondering about for a while and it looks like this project has been around since 2006.

Here is a presentation on the capabilities of this. There isn't much test data here, but based on the stats in the talk, the performance of conntrackd (my testing/production observations was done on similar hardware DL-145G3) look a significantly worse than FreeBSD/OpenBSD with PF+pfsync+CARP. Note that the CARP/VRRP functionality is performed by keepalived.

*BSD can be used in the Enterprise for high availability gigabit packet filtering, but it would be interesting to see if anyone is using iptables+conntrackd+keepalived for this?

Update
This presentation about a successful migration from Linux to OpenBSD confirmed my suspicions about conntrackd not being ready for prime time. And This USENIX article provided an interesting comparison between OpenBSD and Iptables.

Linux is, in general, more efficient than OpenBSD. In both router and bridge configurations, it spends less time forwarding packets. Furthermore, iptables filters packets more quickly than PF, with only one exception (in our testing): if the transport-layer protocol of the transit packet, say, UDP, differs from the specifiedtransport-protocol type of a sequence of rules—“protocol type” set to “TCP”in this example—PF ignores those rules and confronts the packet only with the rest of the set, acting more efficiently than Linux, which confronts the packet with all the rules in the set.

This feature of PF is very interesting. UDP-based attacks are very insidious, and most firewalls have rules to prevent many types of UDP datagram from accessing the network. Nevertheless, most traffic from and to a protected network is made up of TCP streams (protocols such as HTTP, SMTP, and FTP all use TCP). In such a case, PF may be more effective: it does not spend processing time comparing TCP packets with the set of rules destined to block UDP datagrams, avoiding delay in processing legitimate packets. Finally, unlike iptables, PF performs automatic optimization of the rule set, processing it in multiple linked lists [7, 8]. A way to optimize the search on the rule set for iptables is to resort to the “jump” parameter [18] for jumping to a subset of rules (i.e., a chain) reserved for TCP or UDP packets, depending on protocol type.

Of course even discounting performance, the iptables rulesets are much less elegant than ipf/pf.

Drop Dead Simple OpenVPN on OpenBSD 4.4

Like OpenBSD, OpenVPN is something I always end using every couple of years but not often enough to stay fluent with the setup & configuration. Although I don't get into it here, you can do some really cool stuff with bridged mode. We actually used it in the SCADA Honeynet to send traffic to a target PLC. OpenVPN is also an ideal free VPN solution for a small company since it is available on Windows, OSX, and Linux.

After reviewing docs and blog entries since the last time is used it, I found that not only are the too way many howtos out there (when can too much documentation be a bad thing?) many of them are overkill for what I needed and focus on using certificate authentication, when shared key was all I needed. But the Static Key Mini-HOWTO was too simple.

This configuration could be used to provide remote access to a private network over the Internet (or, as the case is here) providing access to the Internet over an insecure wireless network. The OpenVPN server becomes the default route. There is obviously a lot more than you

1. Install on OpenBSD 4.4 (OpenVPN server) via ports (or package if if you lazy)

# cd /usr/ports/net/openvpn && make install

2. Install on Debuntu (client)

# apt-get install openvpn

3. Generate your static key on the server

openvpn --genkey --secret static.key

4. Create Server OpenVPN config (/etc/openvpn/server.conf

dev tun0
port 1234
ifconfig 10.0.0.1 10.0.0.2
secret static.key
ping 15
verb 4

4. Create your PF rules in /etc/pf.conf. The key rules I added were to allow the incoming UDP OpenVPN traffic (port 1234) and to allow all the traffic in on the tun0.

ext_if="xl0"
int_if="rl0"
vpn_if="tun0"
set skip on lo
scrub in
nat on $ext_if from !($ext_if) -> ($ext_if)
block in log
pass out keep state
pass quick on $int_if no state
pass in on $vpn_if keep state
pass in on $ext_if proto udp to ($ext_if) port 1234
pass out proto icmp keep state
pass in  proto icmp keep state
pass in on $ext_if proto tcp to ($ext_if) port ssh

5. OpenVPN client config (in /etc/openvpn/openvpn.conf)

Paste over the key to /etc/openvpn/static.key

dev tun0
remote 192.168.1.44
port 1234
nobind
ifconfig 10.0.0.2 10.0.0.1
secret /etc/openvpn/static.key
redirect-gateway def1
ping 15
verb 4

The new option I learned about was "redirect-gateway def1" which adds a default route to the OpenVPN terminating tunnel address (10.0.0.1) so that all non-local traffic gets sent over the tunnel, which is what I want. Obviously this leaves any local traffic unprotected.

mfranz@mfranz-t61:~$ netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
10.0.0.1        0.0.0.0         255.255.255.255 UH        0 0          0 tun0
192.168.1.100   0.0.0.0         255.255.255.255 UH        0 0          0 venet0
192.168.1.44    192.168.10.254  255.255.255.255 UGH       0 0          0 wlan0
192.168.10.0    0.0.0.0         255.255.255.0   U         0 0          0 wlan0
192.168.122.0   0.0.0.0         255.255.255.0   U         0 0          0 vnet0
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 wlan0
0.0.0.0         10.0.0.1        128.0.0.0       UG        0 0          0 tun0
128.0.0.0       10.0.0.1        128.0.0.0       UG        0 0          0 tun0
0.0.0.0         192.168.10.254  0.0.0.0         UG        0 0          0 wlan0

6. Startup your server (and add this to rc.local)

openvpn --daemon --config /etc/openvpn/server.conf

When you are testing, obviously don't select the --daemon option

7. Connect with the client

# openvpn --config /etc/openvpn/openvpn.conf

Sat Nov 29 14:47:34 2008 OpenVPN 2.1_rc7 i486-pc-linux-gnu [SSL] [LZO2] [EPOLL] built on Jun 11 2008
Sat Nov 29 14:47:34 2008 /usr/sbin/openvpn-vulnkey -q /etc/openvpn/static.key
Sat Nov 29 14:47:34 2008 Static Encrypt: Cipher 'BF-CBC' initialized with 128 bit key
Sat Nov 29 14:47:34 2008 Static Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Sat Nov 29 14:47:34 2008 Static Decrypt: Cipher 'BF-CBC' initialized with 128 bit key
Sat Nov 29 14:47:34 2008 Static Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Sat Nov 29 14:47:34 2008 TUN/TAP device tun0 opened
Sat Nov 29 14:47:34 2008 ifconfig tun0 10.0.0.2 pointopoint 10.0.0.1 mtu 1500
Sat Nov 29 14:47:34 2008 Data Channel MTU parms [ L:1544 D:1450 EF:44 EB:4 ET:0 EL:0 ]
Sat Nov 29 14:47:34 2008 Local Options hash (VER=V4): '5c3fe1ab'
Sat Nov 29 14:47:34 2008 Expected Remote Options hash (VER=V4): '522471df'
Sat Nov 29 14:47:34 2008 UDPv4 link local: [undef]
Sat Nov 29 14:47:34 2008 UDPv4 link remote: 192.168.1.44:1234
Sat Nov 29 14:47:44 2008 Peer Connection Initiated with 192.168.1.44:1234
Sat Nov 29 14:47:46 2008 Initialization Sequence Completed

Any Netflow probes for OpenBSD 4.4?

So I built a new OpenBSD 4.4 box on real hardware (Optiplex GX-100/128MB) so I could ensure the ratio of end hosts to forwarding devices remains less than one on my home network. The great thing about OpenBSD is they haven't touch the installer in the ten years I've used it and the network install always works like a charm assuming you don't fat finger the mirrors.

Pull down some packages, tweak the pf.conf (but forget to enable IP forwarding in sysctl.conf) fire the last system I built back in 2001 or so (K7 with 1.2GB) and then decided to add netflow. The obvious choice is pfflowd which fails to compile and ports says is broken. Spin my wheels around net/if_pfsync.c, browse the diffs. Hmmm... maybe this is harder, screw up the patch for 4.3 I find from Next (well actually I keep bouncing back and forth) try fprobe. 

No luck. Probably a pthreads issue, this might actually work, though?

Finally softflowd which compiles and appears to work, but for some weird reason I'm not seeing the traffic on the wire although it is definitely recording flows. Netstat shows it is has bound the sockets but not traffic is being generated. No firewall drops. Try disabling PF, nothing. Routing table fine. 

Weird.

pflow: a reason to play around with OpenBSD again

I know it is possible to use pfflowd, but pflow looks cool.

From the man page

# ifconfig pflow0 flowsrc 10.0.0.1 flowdst 10.0.0.2:1234

Curphey on Building vs. Breaking

True that

I have grown increasingly disillusioned with the information security industry and especially disillusioned with the application security industry (whatever that really is). Why? I will get onto the information security part where fluffy compliance and best practice culture seems to be gaining acceptance in future posts (probably after a few glasses of wine) but if we take the application security industry specifically then I personally find it is disappointing that after a decade of it being considered a discipline in it’s own right, it is still predominantly made up of breakers and not builders.

I, too, have had anxiety about much vuln work and that is why I'm not in the product/application security bidness anymore. And there is much in Linus's monkey comments that I found hard to disagree with. And bonus points for going after the OpenBSD crowd.

Finding your first vuln, crashing a PIX (or 7200 in IOS) is fun the first time, but bug hunting is ultimately a cheap thrill. And educating product teams to find their own bugs, change their processes, document and design products better is ultimately more rewarding.

L2 Bridge ACLs on Cisco 800 Series ISRs

So it's obvious from this blog I was never a CCIE.

Hell, I barely passed the CCNA 2.0 exam many years ago (not because my IOS skills were that lacking, it was a bad exam, I tell you)

So had a hell of a time finding the extremely simple way to filter MAC addresses on a bridge interface, such as what I'm using on my 851 at home on my kids subnet. Well it was just my kids subnet until the damn Verizons Westel started acting up so bad with WPA with my Linux boxes lately.


I'm too lazy to do WEP (although it does work) and I've never had any luck with WPA under IOS. And yeah the first thing I did was wipe the web interface from flash.

So I figured how hard could it be. But I couldn't find it anywhere until I ran across a CCIE study guide on bridge filtering. Duh.

851w#sh access-lists 700                                                        
Bridge address access list 700                                                  
    permit 0012.f0xx.xxxx   0000.0000.0000 (23 matches)                         
    permit 001d.7exx.xxxx   0000.0000.0000 (38 matches)                         
    permit 0013.e8xx.xxxx   0000.0000.0000 (1930 matches)                       
    permit 0013.5fxx.xxxx   0000.0000.0000   

I learned the hard way there is an implicit deny at the end. And with the 700 series ACLs you don't need to have the 0000.0000.0000

So then you just add "input-address-list 700" to your bridge group and viola!

interface Dot11Radio0.1                                                         
 encapsulation dot1Q 1 native                                                   
 bridge-group 1                                                                 
 bridge-group 1 subscriber-loop-control                                         
 bridge-group 1 input-address-list 700                                          
 bridge-group 1 spanning-disabled                                               
 bridge-group 1 block-unknown-source                                            
 no bridge-group 1 source-learning                                              
 no bridge-group 1 unicast-flooding    

Completely secure, because like the OpenBSD folks I don't rely on security at Layer 2 or 3. It's all in the OS and applications.

And I didn't try it (enough for one night, my reading glasses are already on) but I'm guessing I could do masks as well so I could filter out Apple or Dell MAC addresses and only allow Intel Wireless client adapters.

Now that would be really secure!

New OpenBSD LiveCD

Looks like there is a new OpenBSD LiveCD distro called BSDanywhere. Having built both Linux and OpenBSD LiveCD's the latter is a definitely a pain in the ass, although it is possible.

Although I'm not sure *why* you would want to do it, because with the exception of PF there is no reason to actually use a BSD. But it would be cool if someone wrote a nice GNOME based installer for OpenBSD (like Ubuntu) just to piss off the zealots. Oh and throw in a 20th century package management system while you are at it.

(I did actually get dpkg working on OpenBSD 3.9 once, but never got to apt)

OpenBSD Finally Gets WPA/WPA2

This could be good news for all those fools^H^H^H^Hfolks that try to run *BSD on their desktop/laptops (although I'm sure someone will prove me wrong, cause I didn't take the time to man those devices)

Damien Bergamini (damien@) just committed WPA-support (wikipedia) for OpenBSD. In the commit message, Damien states that "bwi(4), malo(4), ral(4), iwn(4), wpi(4), ural(4), rum(4), upgt(4), and zyd(4) should work." And, Damien says "support for more chipsets should arrive soon."

To me this is of more significance for folks that build wireless routers than anything else.

But what do I know/care. Although truth be told I wouldn't mind running PF as my home firewall if the USB EVDO card would work.

The Sweet Heresy of GNU/kFreeBSD

mfranz@gutsy61:~$ ssh root@192.168.1.105
root@192.168.1.105's password:
Last login: Sat Mar 1 13:30:50 2008 from 192.168.1.107
GNU/kFreeBSD debian 6.3-1-486 #0 Fri Feb 15 22:53:48 CET 2008 i686

The programs included with the Debian GNU/kFreeBSD system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/kFreeBSD comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
debian:~# uname -a
GNU/kFreeBSD debian 6.3-1-486 #0 Fri Feb 15 22:53:48 CET 2008 i686 i386 Intel(R) Pentium(R) Dual CPU E2140 @ 1.60GHz GNU/kFreeBSD

425kpps it is for the DL385G2 (on FreeBSD 6.2)

So for a while this afternoon, I was worried that I wasn't going to be able to generate enough traffic using Click Router on a Compaq DL-145G3 to flatten (meaning, cause packets to drop/stop being forwarded, cause the CPU Interrupt Utilization was around 30%) its big brother/sister.

But if you were always wondered what it took to bring on an interrupt storm on a Compaq DL-385G2 (Opteron with 4 Cores + Quad Intel PCIe NIC) it is around 425 kpps UDP (using 60 byte packets)

Tomorrow I find out what it takes on the Intel counterpart (DL380G5)

S.M.A.R.T. on Gutsy T-61

After my my dismal report on my T-61's SATA performance I've been wondering about whether I have everything tweaked right and I tonight I ran across an entry on Thinkwiki about SMART.

Basically if you install the smartmontools package you get a utility called smartctl that allows you to do stuff like

root@gutsy61:~# smartctl -a /dev/sda
smartctl version 5.37 [i686-pc-linux-gnu] Copyright (C) 2002-6 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF INFORMATION SECTION ===
Device Model:     ST9120822AS
Serial Number:    5LZ1Q720
Firmware Version: 3.CLF
User Capacity:    120,034,123,776 bytes
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   7
ATA Standard is:  Exact ATA specification draft version not indicated
Local Time is:    Sun Feb 10 19:49:22 2008 CST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                 ( 426) seconds.
Offline data collection
capabilities:                    (0x53) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        No Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (  57) minutes.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   100   067   034    Pre-fail  Always       -       196651010
  3 Spin_Up_Time            0x0003   099   099   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       822
  5 Reallocated_Sector_Ct   0x0033   100   100   036    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   059   055   030    Pre-fail  Always       -       111693386018
  9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       149348897784391
 10 Spin_Retry_Count        0x0013   100   100   034    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       902
187 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       0
189 Unknown_Attribute       0x003a   048   048   000    Old_age   Always       -       52
190 Temperature_Celsius     0x0022   062   052   045    Old_age   Always       -       656801830
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       26
192 Power-Off_Retract_Count 0x0032   001   001   000    Old_age   Always       -       4294967291
193 Load_Cycle_Count        0x0022   026   026   000    Old_age   Always       -       148448
194 Temperature_Celsius     0x001a   038   048   000    Old_age   Always       -       38 (Lifetime Min/Max 0/16)
195 Hardware_ECC_Recovered  0x0012   070   042   000    Old_age   Always       -       196651010
196 Reallocated_Event_Count 0x0010   099   099   000    Old_age   Offline      -       157255932577023
197 Current_Pending_Sector  0x003e   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0000   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0000   100   253   000    Old_age   Offline      -       0
202 TA_Increase_Count       0x0000   100   253   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

Here is an article that explains all this junk and smartmontools and this runs on FreeBSD too so this is something to consider on that front, too.

Using Hashes Like it is 1999

This week I picked up [what I thought would be] a quick logfile analysis task. Things started out great. I took the time to look at the logfile format and generalized 4-5 different messages (with appropriate regexes to get the data I needed) generated by the security device. Next I extended a basic "logrunner" class I wrote last month for analyzing the debug output from the Intel FreeBSD drivers (basically you do some sysctl's and it dumps some kernel messages to see counters missed, received packets--much better than netstat).

In my logrunner class, you basically can "attach" various simple regex matches and a symbol and you get a nice hash back with the values you want and it hides all the low-level details of matching or handling time stamps, etc. (HINT: If you are mucking with syslog files in Ruby and you are not using the Time API, you are a fool, but I digress).

After a few hours in I thought things were going fine, before some distractions kept me from working on it again until until the next afternoon (I overconfidently estimated this would take about 4 hours from start to finish), so I was in a rush. The initial desire to develop something be a more general purpose tool and that was designed properly was replaced with the brute force, quick hack, get-r-done approach.

I ended up iterating through the output hashes output by the logrunner tool to create more hashes some with the IP address as a key, others with a username as the key. And all of this pointed to at least another hash (or two) so I ended up with something like:

blah[blah][blah] = { 1 => { a => b, c => d }, 3 => { a=> q, d => z } }

This would have been a trivial task except there was no single session identifier (or even username or IP address) on each line that I could tie the various pieces of data together. Then I kept getting confused (and alternating between |k| and |k,v| with my Ruby blocks) it took my longer than I had hoped but I was done in about 7. I had the output I wanted. Went from a few hundred megs of logs to a nice Excel-friendly CSV file. And I thought I was done.

Until Friday afternoon, when I found it some additional data was needed. Extracting the data wasn't a problem (that was done in 5 minutes), but correlating it and getting the report format was. Should I add another hash? Redefine the hashes I'd written? Five o'clock on Friday (with restless hungry kids) is not a time for clarity of thought, but this morning I realized the Ruby I was written was as unreadable as the Perl I used to write back in the day.

Spending the afternoon driving out in the snow which turned to sleet which turned to rain finally beat some sense into me. I mapped out the data on paper (this time) and did right. Came up with 6 simple classes (2 base and 4 sub) to abstract away the hashes and ended up with less than 1/10th of the lines of code in the main loop and a 1/3 of the iterations. Nothing fancy, no Ruby foo, nothing that couldn't be done in Python. And the code is actually readable. The moral of the story? If you are using hashes 4-6 levels deep you have a problem. Stop, step away from the keyboard and come up with a cleaner design. Do it right the first time, you won't regret it. Because quick hacks have a funny way of running on systems for a long, long time.

OpenBSD Kernel Janitors Sound Good to Me

I must be bored because I'm reading I'm reading openbsd-misc but this kernel janitor thread was pretty classic and almost as good as the one last month when a Google recruiter propositioned Theo.

Here are some quotes although I skipped the good ones:

> > Development is not the same process as writing a whiny mail.
>
> that is a shame.  i can probably better understand the relectance to
> re-visit this if it has failed before.  perhaps, others are right,
> perhaps linux can tolerate it because it's not as good as openbsd.

and

> i think we'll simply agree to disagree.  i personally find it quite
> disheartening to hear the attitude that prevails here but that's the
> community's decision.  it certainaly seems to refelect the attitute
> of it's leaders (developers).
>

Consider it the voice of experience (bitter).

Its easy to tell which ones are the programmers.

They write code, then they submit it, it does not suck too much and they
take the suggestions of the current project leads. Then they resubmit
better code.

The rest of us should simply buy CD's, ask and answer the occasional
question, and other wise keep quiet.

And those weren't even the best (the one where Theo tells the n00b he has anger issues, in particular) but this Blogger word verification is pissing me off, but if you want ask some questions to the OpenBSD crowd to stir the pot, here are some ideas:

• Ask if anyone is interested in porting dpkg/apt to OpenBSD (I actually did the former over the Xmas holidays last year)
• Ask where the "forums" for OpenBSD are and point to the Ubuntu Forums as examples
• Ask where the "desktop edition" of OpenBSD, you know the one with the GUI isntaller instead of this
  

New Click Router Release!

While there are loads of crude packet generators like hping, sendip, nemesis (and back in the day I used to used a set of tools called spak in a TCP/IP Security course I wrote back in 1998) that you could use to reproduce various L2/L3 attack or send the arbitrary frame. But if you need to have tight control of the packet rate and packet size (like smartbits/avalanche) to do performance testing of forwarding devices, the free/Open Source tools are pretty primitive. The built in Linux packet generator allows a mean spew of frames (I measured around 300kpps on my T-61!) but it is either a firehose or a trickle since the delay mechanism was just not effective to set the consistent packet rate.

The Click Modular Router however, does not suffer form these limitations and I used it extensively this Spring when I was comparing interrupt utilization across OpenBSD and FreeBSD PF implementations. But unfortunately it only ran on older kernels (and believe me I tried) so today's release is good news.

I have been meaning to release a UbuntuTrinux-Click release that has an easy to deploy version of Click. Maybe this will help me get on to that task.

CVE-2007-4091 and the Lack of Actionable Info in Vulnerability Disclosures

I was going to blog on something more interesting tonight -- like Cormac McCarthy's Novel, The Road which I read in almost one sitting yesterday evening -- but I got distracted by the new rsync vulnerability disclosed last Wednesday which once again show how little useful information (from the point of view of an end user/administrator) shows up in the disclosures by either the vendors or the finders.

For example:

It still pays to have a look at open source projects.
rsync 2.6.9 contains two off by one stack overflows, one from which the target buffer is next to the
saved frame pointer.
The problematic function is f_name().
Obviously it expects a target buffer size
of MAXPATHLEN bytes. Otherwise
the size parameter calculation to
strlcpy() is wrong.
Lets have a look at f_name() calls within the two following pictures.
An offset is added to the fname buffer
which is of size MAXPATHLEN.
The offset is the stringlen of dir.root
plus one (due to the slash).
Within successfull_send(), the buffer
should be neighbor of the saved

And USN-500-1 is only slightly more useful:

Sebastian Krahmer discovered that rsync contained an off-by-one miscalculation when handling certain file paths. By creating a specially crafted tree of files and tricking an rsync server into processing them, a remote attacker could write a single NULL to stack memory, possibly leading to arbitrary code execution.

So this is only a server issue? In my state of exhaustion (had to work most of the weekend) I am more worried about attacks against the "client?" Like a more trusted centralized server pulling files from many more exposed (less trusted) server. So an attacker creaties a malicious path (greater than 1024) on a remote server (plus whatever else is needed...) to compromise the "rsync client" pulling from the servers? If I'm running rsync+ssh am I just as vulnerable? Is this only an rsyncd issue?

Of course most bug finders could give a shit about real access world issues that ultimately allow risk decisions to be made, and help folks that run systems must be upgraded immediately, which can wait? Or how does this vuln compare to others?
I'm not sure I buy the CVSS 6.8 in the NVD. The NVD entry says this is a pre-auth?

I think you get the point here. More questions than answers. Or do you just blindly update the .deb or RPM? So Ubuntu and Debian have updates out but doesn't look like this is in FreeBSD ports yet and nothing in CVS yet. And the rsync in OSX, can you say 2.6.3

Forget about it.

Any luck ILO100 (DL145G3) on non-Windows JVM?

So HP DL145 G3's are nice, cheap (non-RAID), AMD-64 1U servers with Lights Out Management (they use the ILO100s built by ServerEngines) and the Java Applet KVM (which runs VNC or something like it) worked well on OSX, Linux, and obviously Windows for the eval hardware we had from HP. But whatever the version of only works with the JRE1.4.2-13 of Windows. Using any other JVM results in a NullPointer Exception and the message "General Exception: Optional Package Installation is Aborted"



Anyone else have luck? Oh well, guess I'll be dealing with HP support this week, wish me luck.

Early Returns: Franz: 1 VMWare+CARP+FreeBSD: 0

So running FreeBSD 6.2 CARP on VMWare Server is the subject of some confusion and struggling for me today. This closed CARP bug turned out to be a red herring and this VMTN thread made it seem that it was only possible to use CARP on ESX (if you enable promiscous mode on the virtual switch) and/or possibly with VMWare workstation with teaming. But I don't have admin privs on the ESX cluster. And I want to be able to do development and testing on my laptop and our VMWware Server.

On the drive home I remembered similar battle battle I faced a year ago working with the SCADA Honeynet where broadcast traffic (ARP Requests) were being forwarded to a virtual interface but the unicast was failing. Today pings to the CARP interface were failing even though I had an ARP entry on the host I was pinging from and of course the ICMP Echos were going out--but not making it to the virtual interface.

So it looks like once again loosening permissions on /dev/vmnet[0-9] is the solution to the problem.

i4965: Two Steps Forward, One Step Back

iwl4965:    with FH_INT = 0x00010000
iwl4965: REPLY_CT_KILL_CONFIG_CMD succeeded
iwl4965: REPLY_ADD_STA failed
iwl4965: no version for "release_firmware" found:  kernel tainted.
iwl4965: Intel(R) Wireless WiFi Link 4965AGN driv er for Linux, 0.0.35d
iwl4965: Copyright(c) 2003-2007 Intel Corporation
iwl4965: Detected Intel Wireless WiFi Link 4965AG N
iwl4965: Radio disabled by HW RF Kill switch

So with 2.6.21.5 + iwlwifi-0.0.35 and mac80211-9.0.0 and Linux Wireless Tools v 29 (you'll need the firmware from the Intel site as well) I was able to get to able to get the hardware detected, see a few access points, associate a couple of times, and oh yeah get kernel stack traces every time I tried to get a lease. I was surprised to see the kernel honored the "kill switch" on the front of the laptop.

Oh well, I'll try again next weekend. It's back to a wide open Aironet card when running Linux (God knows why WEP isn't working with the Aironet 352 and the Truemobile running OpenWRT). If you are in the neighborhood, you can have free wifi if you spoof the 3 MACs I have in the filter list (until I get WPA working I'll adopt the bonehead OpenBSD view (I need to find a better link, but that will do) that Wireless security protocols encourage a false sense of security) but I'm sure its much easier to hop on my neighbor's Linksys. Assuming they didn't close it down. But I've got fprobe running the OpenWRT box (as simple as ipkg install fprobe, very cool!) so I'm watching you :)

#  --- ---- ---- Report Information --- --- ---
#
# Fields:    Total
# Symbols:   Disabled
# Sorting:   Descending Field 1
# Name:      UDP/TCP destination port
#
# Args:      flow-stat -f 5 -S1
#
#
# port      flows                 octets                packets
#
80          223                   1012583               4549            
123         182                   21280                 280             
3000        132                   56380                 271             
443         20                    77666                 331             
2050        16                    42523                 239             
53          16                    15287                 239             
60321       7                     20647                 69              
22          7                     24766                 307