just rtfm ng_netflow(4)

On FreeBSD 6.2, to use the built in kernel netflow export capability, as long as the ng_ether is loaded, the example at the bottom of ng_netflow(4) should work just fine. However, I did put up a a wiki page on the topic, after googling all over. It will be interesting to see what sort of performance is possible with it

Compiling Flow-tools on OSX/gcc4

Recent (at least 4.0.x) versions of gcc don't like flow-tools but there is a patch

franz-g4:~/Desktop/flow-tools-0.68 mdfranz$ ./configure --prefix=/opt/local
checking for a BSD-compatible install... /usr/bin/install -c

SNIP

franz-g4:~/Desktop/flow-tools-0.68 mdfranz$ make
Making all in lib
make all-am
source='ftio.c' object='ftio.o' libtool=no \
depfile='.deps/ftio.Po' tmpdepfile='.deps/ftio.TPo' \
depmode=gcc3 /bin/sh ../depcomp \
gcc -I. -I./lib -I. -I. -I. -g -Wall -g -Wall -c `test -f 'ftio.c' || echo './'`ftio.c
ftio.c: In function 'readn':
ftio.c:2270: error: address of register variable 'ptr' requested
ftio.c:2270: warning: target of assignment not really an lvalue; this will be a hard error in the future
ftio.c:2270: confused by earlier errors, bailing out
make[2]: *** [ftio.o] Error 1
make[1]: *** [all] Error 2
make: *** [all-recursive] Error 1
franz-g4:~/Desktop/flow-tools-0.68 mdfranz$ patch -p1 <>

Painless Non-Enterprise Netflow

Tonight I released another ubuntutrinux-core snaphot that include fprobe and a few tools from flow-tools. I'll spare the introduction to Netflow except to comment why this might useful for Linux/Trinux (its obvious for routers!) as opposed to other network monitoring tools.

In terms of data your are getting about what you would get with a port logger such as ippl or other port listeners that log connections from hosts

root@gx620:/tmp# flow-cat biglast | flow-print | head
srcIP dstIP prot srcPort dstPort octets packets
24.136.0.111 239.255.255.250 2 0 0 32 1
24.136.0.189 239.255.255.250 2 0 0 32 1
82.211.81.145 24.136.x.y 17 123 123 76 1
24.136.2.30 239.255.255.250 2 0 0 32 1
24.136.2.67 224.0.0.251 2 0 0 32 1
24.136.2.67 239.255.255.253 2 0 0 32 1
10.48.120.1 224.0.0.1 2 0 0 28 1
24.136.0.163 239.255.67.250 2 0 0 32 1
24.136.19.48 224.0.0.253 2 0 0 32 1

Notice this is mostly multicast cruft on RCN with the exception of NTP traffic to the Ubuntu time source. But big deal. Some of you may remember (back in the day!) a NSWC tool called SHADOW (where Northcutt and Irwin made their claim to fame) that was basically a collection of Perl scripts that managed tcpdump file capture and viewing through a web interface.

Well flow-tools allows you to a lot of the same stuff with much less overhead and all from the command-line

$ flow-cat biglast | flow-stat -f5 -S 1 | head -25

# Args: flow-stat -f5 -S 1
#
#
# port flows octets packets
#
80 2464 12632818 162315
53 1099 214515 3053
1026 414 250208 710
32768 405 131967 766
1027 314 155033 314
123 186 14136 186
443 177 333745 3020
7 175 19075 175
5222 141 36842 370
3408 120 15892 220

Basically you "flow-cat" the saved file to a number of different tools, flow-stat being the most useful for me. Not terribly surprising HTTP is at the top nor the 1026 to my firewall. Damn cable.

$ flow-cat biglast | flow-stat

#
# Fields: Total
# Symbols: Disabled
# Sorting: None
# Name: Overall Summary
#
# Args: flow-stat
#
Total Flows : 13168
Total Octets : 380416991
Total Packets : 517136
Total Time (1/1000 secs) (flows): 222894382
Duration of data (realtime) : 34320
Duration of data (1/1000 secs) : 364934
Average flow time (1/1000 secs) : 16926.9733
Average packet size (octets) : 735.6227
Average flow size (octets) : 28889.5043
Average packets per flow : 39.2722
Average flows / second (flow) : 36.1758
Average flows / second (real) : 0.3837
Average Kbits / second (flow) : 8360.8132
Average Kbits / second (real) : 88.6753

After concatenating all this data into a single file, it only took about about 800k for about 10 hours of traffic.

And how did I kick all this off?

First I ran fprobe on Trinux and made sure it was working by testing it out with EHNT
which is the quickest way (it took me a while to wander through the flow-tools manpages, and the ubuntu startup script (in /etc/init.d) for capturing flows didn't work. I'm using pcap to get this but there is a version of fprobe that can generate flow from iptables.

#fprobe -u nobody collector-ip:collector port

BTW, netflow uses UDP. You can sniff to make sure the flow updates are being sent. And then on the server (you'll want to be more restrictive on the local and remote ports, the 0's)

#flow-capture -w /raid/flows/ 0/0/4444 -S20

which creates the directory hierarchy like:

root@gx620:/raid/flows/2007# ls -alR | less

.:
total 0
drwxr-xr-x 3 root root 72 2007-04-29 12:04 .
drwxr-xr-x 3 root root 72 2007-04-29 12:04 ..
drwxr-xr-x 3 root root 80 2007-04-29 12:04 2007-04

./2007-04:
total 2
drwxr-xr-x 3 root root 80 2007-04-29 12:04 .
drwxr-xr-x 3 root root 72 2007-04-29 12:04 ..
drwxr-xr-x 2 root root 2208 2007-04-29 22:15 2007-04-29

./2007-04/2007-04-29:
total 326
drwxr-xr-x 2 root root 2208 2007-04-29 22:15 .
drwxr-xr-x 3 root root 80 2007-04-29 12:04 ..
-rw-r--r-- 1 root root 1145 2007-04-29 12:15 ft-v05.2007-04-29.121249-0500
-rw-r--r-- 1 root root 4500 2007-04-29 12:30 ft-v05.2007-04-29.121904-0500
-rw-r--r-- 1 root root 2153 2007-04-29 12:45 ft-v05.2007-04-29.123001-0500


Of course there are tons more options (the -S20 just says write a status report to syslog every 20 minutes like:

Apr 29 22:20:00 localhost flow-capture[27718]: STAT: now=1177903200 startup=1177867131 src_ip=192.168.100.1 dst_ip=192.168.169.162 d_ver=5 pkts=2918 flows=13828 lost=1 reset=0 filter_drops=0

Saturday Mornings with YAML

ubuntutrinux is progressing nicely to the point that I was able to replace my OpenBSD Internet-facing facing pf box. Sure iptables sucks for real firewalls, but I burned Trinux on a CD with a static set of rules on a Old Celeron-based Optiplex GX100 with no hard drive. Good enough. BTW, I added frobe to be able to export netflow data. With device file creation mostly complete, I'm moving to the initialization scripts and to a new YAML configuration scheme copy the spirit (if not the letter) of pfsense.

Right now I'm playing with something like:

network:
  hostname: franzfw
  interfaces:
    eth0:
      address: dhcp
    eth2:
      address: 192.168.100.1
      netmask: 255.255.255.0
  routes:
    -'192.168.169.0/24': 192.168.100.2
  forwarding: true

And when I run a little "check" script you see:

franz-g4:~/dev/playin/ruby mdfranz$ ruby ycheck.rb config.yaml
yf is a:Hash
{"network"=>
{"routes"=>{"-'192.168.169.0/24'"=>"192.168.100.2"},
"forwarding"=>true,
"interfaces"=>
{"eth0"=>{"address"=>"dhcp"},
 "eth2"=>{"netmask"=>"255.255.255.0", "address"=>"192.168.100.1"}},
"hostname"=>"franzfw"}}
With the following keys:
-network
Hostname:franzfw
interfaces is a:Hash
Configuring:eth0
Configuring:eth2

With the script that created this...

require 'yaml'
require 'pp'
yf = YAML.load_file(ARGV[0])
print "yf is a:"
print yf.class
print "\n"
pp yf
print "With the following keys:\n"
yf.keys.each { |k| puts "-" + k }
puts  "Hostname:" + yf['network']['hostname']
interfaces = yf['network']['interfaces']
print "interfaces is a:"
print interfaces.class
print "\n"
print interfaces.keys.each { |i| puts "Configuring:" + i }

Don't have time to get into the details (see the YAML Cookbook for more info) but there were a couple of areas I got hung up on:

• Using tabs instead of whitespaces. Ruby YAML doesn't like tabs, but if your vimrc is setup for Python you should be good to go.
  
• Getting hashes instead of arrays (and vice versa). Play around with ident's and -'s until you get what you need
  

XML parsing (or creating XML configuration files) isn't something I do on a daily basis (thank God!) but I've done enough to know that this is a lot easier to read and parse.

Since ruby is so small (less than 2MB compressed) it will be included as part of the core and probably everything outside /init may use it.

Where my Linux device files went and how sysfs helped me find them

I shouldn't have been, but the other I was surprised to see (unlike back in the day) the /dev directory isn't completely populated with every device know to man. They are gone, replaced by something called udev which creates the files depending which hardware is available. So unlike in the old Trinux, I can't simply copy into the target filesystem. There are no hd[abcd] devices on my development box. Although there is a script (/usr/share/initramfs-tools/hooks/udev) that is used by the initial ramdisk do to this, but it looked like overkill, particularly once I start digging into /etc/udev/rules.d. Sure I could have got it working, but how long would it take? What other tools would it depend on? That may or may not be in Busybox. All I wanted to do was create basic ide and scsi devices and the /dev/cciss/ devices used by the Compaq RAID adapters in the DL380s we have at work.

What I needed was the major and minor numbers for all devices are available in the system. Once I figured that out I could use mknod to create them myself. Quick and dirty, like everything else Trinux. Of course the downside is /sbin/udevd is a daemon that is running and I assume helps. I will have to manually run the scripts that created the devices as new kernel modules are installed and new classes of devices. Of course, with ubuntutrinux I'm building as large as kernels as possible with almost everything in it as possible.

So how do I find out what devices are there and learn the major/minor numbers?

That is where sysfs comes in, allowing you to get detailed information about hardware devices. A funny side note. I wouldn't recommend browsing this hierarchy with the busybox ls because I was getting segfaults.

So let's look at the SCSI (actually SATA) devices present on my Dell server and where they show up with sysfs.

mdfranz@gx620:/sys$ ls -al /dev/sd*
brw-rw---- 1 root disk 8,  0 2007-04-25 16:20 /dev/sda
brw-rw---- 1 root disk 8,  1 2007-04-25 16:20 /dev/sda1
brw-rw---- 1 root disk 8,  2 2007-04-25 16:20 /dev/sda2
brw-rw---- 1 root disk 8,  3 2007-04-25 16:20 /dev/sda3
brw-rw---- 1 root disk 8, 16 2007-04-25 16:20 /dev/sdb
brw-rw---- 1 root disk 8, 17 2007-04-25 16:20 /dev/sdb1
brw-rw---- 1 root disk 8, 18 2007-04-25 16:20 /dev/sdb2
mdfranz@gx620:/sys$ find /sys -name "sd*"
/sys/module/sd_mod
/sys/block/sdb
/sys/block/sdb/sdb2
/sys/block/sdb/sdb1
/sys/block/sda
/sys/block/sda/sda3
/sys/block/sda/sda2
/sys/block/sda/sda1
/sys/bus/scsi/drivers/sd
mdfranz@gx620:/sys$ cd /sys/block/sdb/
mdfranz@gx620:/sys/block/sdb$ cat dev
8:16

10 Days of Trinux, 9 Years Later

It's actually been longer since I lamented the lack of well maintained ramdisk distros but the project activity log say 10 days so I'll go with that.

It was certainly tempting to throw in the towel and say Trinux peaked in 2000 and be done with it (which it actually did, and what I thought when I let the trinux.org domain expire a few months back) I think there are some good reasons to re-introduce (and continue to maintain) a minimal ramdisk Linux distribution in 2007 -- 9 years after first started serving up it off of www.txdirect.net/~mdfranz and a SATLUG page. And I'm not including the fact that there are still some folks left on trinux-talk from the old days and that maintaining a distro is fun. So here is a heavily annotated (but limited to a single sentence) rationale for what I'm up to with ubuntutrinux.

While Firmware Linux is a well-designed project and actually works (unlike the bloated buildroot) there is a need for a small (meaning under a 10 MB core) non-uclibc distribution or distribution toolchain (i.e distribution creator) that allows releases to be packaged quickly (like under 10 minutes, not counting kernel compilation) using the latest Linux 2.6 kernels that allow easily deployment applications compiled on a standard desktop/server Linux (like Ubuntu) for a variety of boot media: CD-ROM, USB, PXE, or a Linux /boot partition.

Although the documentation is incomplete and I haven't been doing predictable numbered releases yet, it does this now and almost everything is up on snapshots directory.

During the first go around I made the mistake of supporting some boot media more than others. For me a single floppy plus network package loading was the only way to go. ISO were impractical and inflexible.

It should be a no suprise if you read the italicized text above, but with ubuntutrinux, the key design principles are:

• Simplicity, speed, and ease of use
• Maximum deployment
• Interoperability

The rest is implementation.

Death and Four Hours to Spare?

After the brief scare when I thought my home network was owned by a South American botnet (turns out I accidentally associated to my neighbor's wireless network, that was the reason for the Spanish in my ngrep dumps) my Tax Return is supposedly getting transmitted to intuit. But the fan is blowing like crazy. Hmm.. Maybe it is locked up.

 207 TurboTax 2  99.8% 18:53.28   6   126   635  44.0M  42.6M+ 68.7M   430M

Force quit and start again. This time I remember to set my scroll buffer

T 192.168.170.100:49284 -> 208.240.240.200:80 [AP]
 GET /etx.ctl HTTP/1.1..User-Agent: CFNetwork/129.20..Accept: */*..Accept-La
 nguage: en..Accept-Encoding: gzip, deflate..Connection: keep-alive..Host: e
 txprod.intuit.com....                                                     
##
T 208.240.240.200:80 -> 192.168.170.100:49284 [AP]
 HTTP/1.1 200 OK..Server: Netscape-Enterprise/6.0..Date: Wed, 18 Apr 2007 01
 :49:44 GMT..Content-type: text/plain..Connection: close....               
##
T 208.240.240.200:80 -> 192.168.170.100:49284 [AP]
 Reply: validate-volume;.                                                  
##
T 208.240.240.200:80 -> 192.168.170.100:49284 [AP]
 Date: April 17, 2007   06:49 PM PDT..                                     
##
T 208.240.240.200:80 -> 192.168.170.100:49284 [AP]
 602 Could not complete request for given parameters..                     
##
T 208.240.240.200:80 -> 192.168.170.100:49284 [AP]
 ----..                                                                    
########
T 192.168.170.100:49285 -> 208.240.240.200:80 [AP]
 GET /etx.ctl?val HTTP/1.1..User-Agent: CFNetwork/129.20..Accept: */*..Accep
 t-Language: en..Accept-Encoding: gzip, deflate..Connection: keep-alive..Hos
 t: etxprod.intuit.com....                                                 
###
T 208.240.240.200:80 -> 192.168.170.100:49285 [AP]
 Reply: validate-volume;.                                                  
##
T 208.240.240.200:80 -> 192.168.170.100:49285 [AP]
 HTTP/1.1 200 OK..Server: Netscape-Enterprise/6.0..Date: Wed, 18 Apr 2007 01
 :49:44 GMT..Content-type: text/plain..Connection: close....               
##
T 208.240.240.200:80 -> 192.168.170.100:49285 [AP]
 Date: April 17, 2007   06:49 PM PDT..                                     
##
T 208.240.240.200:80 -> 192.168.170.100:49285 [AP]
 006 OK to proceed with validation..                                       
##
T 208.240.240.200:80 -> 192.168.170.100:49285 [AP]
 ----..                                                                    
#########
T 192.168.170.100:49286 -> 208.240.240.200:443 [AP]
 ...........F%y...+.e.I....>#....s.P*..U.".. C.#...h4.W.L.V.....;.*.4..|+g.T
 q.<./.....5.................2.3.4.8.9.:.......................... 

Wonder how many folks are in the same boat.

Booting ubuntutrinux (or other initrafms fare) within Qemu

Getting re-acquainted with embedded linux (and building ramdisk distros) over the past week or so has been sort of painful (I'm still struggling with the transition from initrd to initramfs) but over the weekend I manged to "release" a kernel and filesystem built on busybox 1.4.2 and Linux 2.6.20 as well some scripts (stripped down versions of the mkinitrd stuff in ubuntu) I used to create them.

But I'm still not happpy. Rob Landley's site and various documentation he has written has proved to be quite useful but still haven't solved the main problem of getting a initramfs to behave like the initrd+linuxrc's did with Trinux but I have ditched VMWare for QEMU. I do builds on my Dell Optiplex Pentium D then copy them over to my PowerBook where they run (even on a PPC!)


franz-g4:~/ubuntutrinux-test-0 mdfranz$ qemu -nographic -kernel bzImage -initrd initramfs -append "console=ttyS0" /dev/zero


And then you get within a Terminal...

starting qemu...
(qemu) [ 0.000000] Linux version 2.6.20.2 (root@gx620) (gcc version 4.0.3 (Ubuntu 4.0.3-1ubuntu5)) #1 SMP PREEMPT Sat Apr 7 07:56:09 CDT 2007
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] sanitize start
[ 0.000000] sanitize end
[ 0.000000] copy_e820_map() start: 0000000000000000 size: 000000000009fc00 end: 000000000009fc00 type: 1
[ 0.000000] copy_e820_map() type is E820_RAM
[ 0.000000] copy_e820_map() start: 000000000009fc00 size: 0000000000000400 end: 00000000000a0000 type: 2
[ 0.000000] copy_e820_map() start: 00000000000e8000 size: 0000000000018000 end: 0000000000100000 type: 2
[ 0.000000] copy_e820_map() start: 0000000000100000 size: 0000000007f00000 end: 0000000008000000 type: 1
[ 0.000000] copy_e820_map() type is E820_RAM
[ 0.000000] copy_e820_map() start: 00000000fffc0000 size: 0000000000040000 end: 0000000100000000 type: 2
[ 0.000000] BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
[ 0.000000] BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
[ 0.000000] BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
[ 0.000000] BIOS-e820: 0000000000100000 - 0000000008000000 (usable)
[ 0.000000] BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
[ 0.000000] 0MB HIGHMEM available.
[ 0.000000] 128MB LOWMEM available.
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0 -> 4096
[ 0.000000] Normal 4096 -> 32768
[ 0.000000] HighMem 32768 -> 32768
[ 0.000000] early_node_map[1] active PFN ranges
[ 0.000000] 0: 0 -> 32768
[ 0.000000] DMI not present or invalid.
[ 0.000000] ACPI: Disabling ACPI support
[ 0.000000] Allocating PCI resources starting at 10000000 (gap: 08000000:f7fc0000)
[ 0.000000] Detected 18.432 MHz processor.
[ 2.901454] Built 1 zonelists. Total pages: 32512
[ 2.901788] Kernel command line: console=ttyS0
[ 2.911864] Found and enabled local APIC!
[ 2.912649] Enabling fast FPU save and restore... done.
[ 2.913650] Enabling unmasked SIMD FPU exception support... done.
[ 2.916144] Initializing CPU#0
[ 2.919308] PID hash table entries: 512 (order: 9, 2048 bytes)
[ 2.927577] Console: colour VGA+ 80x25
[ 3.149985] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
[ 3.156115] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
[ 3.302374] Memory: 121064k/131072k available (2010k kernel code, 9508k reserved, 653k data, 312k init, 0k highmem)
[ 3.341009] virtual kernel memory layout:
[ 3.341098] fixmap : 0xfff4f000 - 0xfffff000 ( 704 kB)
[ 3.341176] pkmap : 0xff800000 - 0xffc00000 (4096 kB)
[ 3.341253] vmalloc : 0xc8800000 - 0xff7fe000 ( 879 MB)
[ 3.341330] lowmem : 0xc0000000 - 0xc8000000 ( 128 MB)
[ 3.341405] .init : 0xc039f000 - 0xc03ed000 ( 312 kB)
[ 3.341482] .data : 0xc02f69d3 - 0xc0399f70 ( 653 kB)
[ 3.341557] .text : 0xc0100000 - 0xc02f69d3 (2010 kB)
[ 3.431756] Checking if this processor honours the WP bit even in supervisor mode... Ok.
[ 3.659081] Calibrating delay using timer specific routine.. 93.63 BogoMIPS (lpj=187261)
[ 3.691223] Security Framework v1.0.0 initialized
[ 3.716745] SELinux: Disabled at boot.
[ 3.720889] Mount-cache hash table entries: 512
[ 3.756947] CPU: L1 I cache: 8K
[ 3.771871] CPU: L2 cache: 128K
[ 3.784194] Compat vDSO mapped to ffffe000.
[ 3.802657] Checking 'hlt' instruction... OK.
[ 3.920842] SMP alternatives: switching to UP code
[ 3.931291] Freeing SMP alternatives: 10k freed
[ 3.943582] CPU0: Intel Pentium II (Klamath) stepping 03
[ 3.964529] SMP motherboard not detected.
[ 4.230102] Brought up 1 CPUs
[ 4.267331] NET: Registered protocol family 16
[ 4.314839] EISA bus registered
[ 4.322200] PCI: PCI BIOS revision 2.10 entry at 0xf9fa0, last bus=0
[ 4.325915] PCI: Using configuration type 1
[ 4.328017] Setting up standard PCI resources
[ 4.427408] ACPI: Interpreter disabled.
[ 4.441367] Linux Plug and Play Support v0.97 (c) Adam Belay
[ 4.483253] pnp: PnP ACPI: disabled
[ 4.498100] PnPBIOS: Scanning system for PnP BIOS support...
[ 4.541351] PnPBIOS: PnP BIOS support was not detected.
[ 4.587718] PCI: Probing PCI hardware
[ 4.609690] * Found PM-Timer Bug on the chipset. Due to workarounds for a bug,
[ 4.610142] * this clock source is slow. Consider trying other clock sources
[ 4.645056] PCI quirk: region b000-b03f claimed by PIIX4 ACPI
[ 4.685876] PCI: Using IRQ router PIIX/ICH [8086/7000] at 0000:00:01.0
[ 4.710713] PCI: BIOS reporting unknown device 01:00
[ 4.733156] PCI: BIOS reporting unknown device 01:00
[ 4.749968] PCI: BIOS reporting unknown device 01:00
[ 4.784264] PCI: BIOS reporting unknown device 01:00
[ 4.795444] PCI: BIOS reporting unknown device 01:00
[ 4.815190] PCI: BIOS reporting unknown device 01:00
[ 4.843408] NET: Registered protocol family 8
[ 4.845769] NET: Registered protocol family 20
[ 4.895700] PCI: Ignore bogus resource 6 [0:0] of 0000:00:02.0
[ 4.918304] NET: Registered protocol family 2
[ 5.044717] IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 5.075773] TCP established hash table entries: 4096 (order: 3, 49152 bytes)
[ 5.123529] TCP bind hash table entries: 2048 (order: 2, 24576 bytes)
[ 5.146642] TCP: Hash tables configured (established 4096 bind 2048)
[ 5.180315] TCP reno registered
[ 5.249366] checking if image is initramfs... it is
[ 24.071603] Freeing initrd memory: 4956k freed
[ 24.134872] audit: initializing netlink socket (disabled)
[ 24.160085] audit(1176777411.196:1): initialized
[ 24.215895] VFS: Disk quotas dquot_6.5.1
[ 24.234002] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[ 24.280635] io scheduler noop registered
[ 24.295858] io scheduler anticipatory registered (default)
[ 24.299539] io scheduler deadline registered
[ 24.302895] io scheduler cfq registered
[ 24.306052] Limiting direct PCI/PCI transfers.
[ 24.308877] PCI: PIIX3: Enabling Passive Release on 0000:00:01.0
[ 24.312882] Activating ISA DMA hang workarounds.
[ 24.347502] isapnp: Scanning for PnP cards...
[ 25.303982] isapnp: No Plug & Play device found
[ 27.407116] Real Time Clock Driver v1.12ac
[ 27.430981] Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
[ 27.472354] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16450
[ 27.575725] RAMDISK driver initialized: 16 RAM disks of 65536K size 1024 blocksize
[ 27.634910] Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
[ 27.668313] ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
[ 27.694047] PNP: No PS/2 controller found. Probing ports directly.
[ 27.715008] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 27.741112] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 27.769258] mice: PS/2 mouse device common for all mice
[ 27.802423] input: AT Translated Set 2 keyboard as /class/input/input0
[ 27.865277] EISA: Probing bus 0 at eisa.0
[ 27.902510] EISA: Detected 0 cards.
[ 27.922112] TCP cubic registered
[ 27.954340] NET: Registered protocol family 1
[ 27.979541] Testing NMI watchdog ... CPU#0: NMI appears to be stuck (0->0)!
[ 28.141213] Using IPI No-Shortcut mode
[ 28.188656] Time: tsc clocksource has been installed.
[ 28.275904] Freeing unused kernel memory: 312k freed
Starting init!
mknod: /dev/console: File exists
mknod: /dev/null: File exists
bash: no job control in this shell
bash-3.1#

I just wish there were a way to "stay in" this filesystem (I think it is rootfs) and get job control, run the simple busybox init and spawn shells without mounting another root, but I'm guessing I'll have to create another ramdisk filesystem and switch_root into it.

label default
    kernel bzimage
    append initrd=initrd.gz root=/dev/ram0 rw

Then linuxrc got executed and then after exiting it, Busybox init kicked in.

Two Linux Kernel Lessons of The Day

So I learned two important things about Linux kernel compilation today:

1. On Edgy (for some kernels like 2.6.16.13, where I'm still trying to get Click running on real hardware) you have to disable gcc stack protection by adding -fno-stack-protector to CFLAGS. I actually didn't figure this out the first time but when I was getting the error trying to build GRUB2 (because GRUB sucks on MacBooks, something about the keyboard working less that 25% of the time) , I figured something was up
   
2. Never, never, never, never try to compile a kernels (well maybe 2.0 might be ok) on the modular drive bay of a Dell Latitude, where I run Linux so as to not disrupt my "Corporate Image" -- Let's just say it took about 3-4 hours on a P4-1.6 with a Gig of Ram. Yikes.
   

But the day wasn't all bad, as I discovered Ruby Dnet and more importantly and interesting blog by Brian Caswell about using DL. Of course I started this trail because Ruby is so lame and doesn't have tons of stuff like Python Netifaces for easily (and cross-platform-ily) getting address information.

Hardening Drupal/Common PHP Apps?!

So I'm getting ready to deploy a website for our church (most likely with Drupal) and while I'm excited building a site, I'm not thrilled about putting a large, commonly deployed PHP app out there. Kudos to WordPress although really only one paragraph is useful. The rest is the normal blah blah database blah blah firewall blah blah file permissions blah blah.

Sure there is the hardened php project but that doesn't help me much. Does it? If there isn't an Ubuntu (Dapper) package I'm certainly not going to use it. Maybe the fact that here is a php-suhosin patch for Debian 4.0 might be one reason to run Debian but that wasn't an option (yet) with my VPS provider.

Yeah there is the howto but I repeat, I'm not going to any crap that looks like this:

erver1:/usr/src/php5-5.2.0# dpkg-buildpackage
dpkg-buildpackage: source package is php5
dpkg-buildpackage: source version is 5.2.0-8+etch1
dpkg-buildpackage: source changed by sean finney
dpkg-buildpackage: host architecture i386
dpkg-buildpackage: source version without epoch 5.2.0-8+etch1
dpkg-checkbuilddeps: Unmet build dependencies: apache-dev (>= 1.3.23) apache2-prefork-dev (>= 2.0.53-3) bison chrpath debhelper (>= 3) firebird2-dev flex (>= 2.5.4) freetds-dev libapr1-dev (>= 1.2.7-8) libbz2-dev (>= 1.0.0) libc-client-dev libcurl3-openssl-dev | libcurl3-dev libdb4.4-dev libexpat1-dev (>= 1.95.2-2.1) libfreetype6-dev libgcrypt11-dev libgd2-xpm-dev (>= 2.0.28-3) libjpeg62-dev libkrb5-dev libldap2-dev libmcrypt-dev libmhash-dev (>= 0.8.8) libmysqlclient15-dev | libmysqlclient12-dev libncurses5-dev libpam0g-dev libpcre3-dev (>= 6.6) libpng12-dev libpq-dev | postgresql-dev libpspell-dev librecode-dev libsnmp9-dev | libsnmp-dev libsqlite0-dev libt1-dev libtidy-dev libwrap0-dev libxmltok1-dev libxml2-dev (>= 2.4.14) libxslt1-dev (>= 1.0.18) re2c unixodbc-dev
dpkg-buildpackage: Build dependencies/conflicts unsatisfied; aborting.
dpkg-buildpackage: (Use -d flag to override.)


I barely have enough time to maintain and develop the site, let alone compile stuff from source. And with the ongoing month of PHP bugs folks would have to do a lot of compiling. Not realistic.

And I'm the lucky one since this is VPS box (where I at least have root) and not a shared hosting environment like most of the poor saps running PHP apps, I reckon.

BTW I'm not even worried about XSS, SQL Injection yet. Just basic stuff, like figuring out the best Apache config that will actually work with drupal and which files (like blahblahxmlrpc.php) I can safely get rid of from my drupal directory without breaking the install..

So real answers yet (although I've started a PHP Security Wiki page) and will also document what I've done on a Hardening Drupal page as well and I'd love any suggestions folks have on the latter.

PXE FreeBSD 6.2 Install

So I have a brain-dead HP DL145 that doesn't like booting from USB CD-ROM's (no built in SATA either) with FreeBSD (OpenBSD 4.1 was fine, BTW)

Fortunately this blog showed how to use memdisk and pxelinux to get the job done.

In short, you convert the /boot directory from the network install image and pass that to memdisk:

# cat /tftpboot/pxelinux.cfg/default 
label linux
        kernel bzimage
        append initrd=initrd

label freebsd
        kernel memdisk
        append initrd=boot.hd harddisk

The only thing that tripped me up is that memdisk is a binary that must be copied over to your /tftpboot partition

# pwd
/tftpboot
# ls -al
total 84720
drwxr-xr-x   3 root  wheel       512 Apr 13 04:17 .
drwxr-xr-x  16 root  wheel       512 Apr 12 10:34 ..
-rwxr-xr-x   1 root  wheel  33554432 Apr 13 03:31 boot.hd
-rwxr-xr-x   1 root  wheel   3542048 Apr 12 04:18 bzimage
-rwxr-xr-x   1 root  wheel   5919658 Apr 12 04:18 initrd
-rwxr-xr-x   1 root  wheel     19132 Apr 13 03:48 memdisk
-r-xr-xr-x   1 root  wheel    215040 Apr 12 10:26 pxeboot
-rwxr-xr-x   1 root  wheel     13204 Apr 12 04:16 pxelinux.0
drwxr-xr-x   2 root  wheel       512 Apr 12 04:30 pxelinux.cfg

Unleash the Ramdisk Within [Ubuntu/Debian]

If you check out your grub configuration file (menu.1st) you'll see something similar to this:

title Ubuntu, kernel 2.6.15-28-686
root (hd0,1)
kernel /boot/vmlinuz-2.6.15-28-686 root=/dev/sda2 ro quiet splash
initrd /boot/initrd.img-2.6.15-28-686
savedefault
boot

Now initrd's (initial ramdisks) were the bread and butter technology I used to build Trinux. Although it says initrd it is actually an initrams, a kernel feature introduced in late 2.5 . meaning it is available in the 2.6 kernel you are probably running.

What is this file and how is it created?

root@gx620:/boot# file initrd.img-2.6.15-28-686
initrd.img-2.6.15-28-686: gzip compressed data, from Unix, max compression

So it can be gunzipped and un-cpio'd (-vid are the options, do an "info cpio") and you can get whats inside. This file is created by /usr/sbin/mkinitramfs and you can create you own. Let's say you wanted to create a new one with the latest kernel (2.6.20.2, right now) to check out some new features you would do "mkinitramfs -o my-initramfs 2.6.20.2" based on:

Usage: /usr/sbin/mkinitramfs [OPTION]... <-o outfile> [version]

Options:
-d confdir Specify an alternative configuration directory.
-k Keep temporary directory used to make the image.
-o outfile Write to outfile.
-r root Override ROOT setting in mkinitrd.conf.

One of the really cool things about this script is the copy_exec function which adds files to the new ramdisk image, automatically populating /lib and /usr/lib (or whatever) which the required dynamic libraries.

copy_exec /sbin/modprobe /sbin
copy_exec /sbin/depmod /sbin
copy_exec /sbin/rmmod /sbin
mkdir -p ${DESTDIR}/etc/modprobe.d
cp -a /etc/modprobe.d/* ${DESTDIR}/etc/modprobe.d

/sbin/rmmod /sbinmkdir -p ${DESTDIR}/etc/modprobe.dcp -a /etc/modprobe.d/* ${DESTDIR}/etc/modprobe.d

As you can see, the module utilities aren't terribly interesting, but even a simple tool like wget depends on a quite a few libraries:

root@gx620:/usr/share/initramfs-tools# ldd /sbin/insmod
      linux-gate.so.1 =>  (0xffffe000)
      libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7dac000)
      /lib/ld-linux.so.2 (0xb7eed000)
root@gx620:/usr/share/initramfs-tools# ldd /usr/bin/wget
      linux-gate.so.1 =>  (0xffffe000)
      libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7fce000)
      librt.so.1 => /lib/tls/i686/cmov/librt.so.1 (0xb7fc6000)
      libssl.so.0.9.8 => /usr/lib/i686/cmov/libssl.so.0.9.8 (0xb7f88000)
      libcrypto.so.0.9.8 => /usr/lib/i686/cmov/libcrypto.so.0.9.8 (0xb7e59000)
      libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7d2a000)
      /lib/ld-linux.so.2 (0xb7fe3000)
      libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb7d18000)
      libz.so.1 => /usr/lib/libz.so.1 (0xb7d04000)

This took me ages to do with Trinux.

But how does the new system boot? Instead of a good old linuxrc (used by initrd) initramfs uses an init shell script which you can see in /usr/share/initramfs-tools/init. If you look near the end, you can see where the "real" init is called

root@gx620:/usr/share/initramfs-tools# tail -f init
while [ ! -x ${rootmnt}${init} ]; do
       panic "Target filesystem doesn't have ${init}"
done
maybe_break init

# Unset the problematic debug variable and chain to real filesystem
unset debug
exec run-init ${rootmnt} ${init} "$@" <${rootmnt}/dev/console >${rootmnt}/dev/console

If you comment out these last few lines and add a /bin/sh you could (and then rebuild with mkinitramfs.

I'll be documenting more of this on my wiki and as part of ubuntutrinux, so stay tuned!

Trinux Sourceforge Statistics Shock

Although the trajectory is downward, I was shocked to monthly Trinux download statistics within the 1.5-3K/month range and even more frightening hit counts still in the 50-60k/month range. Who are these people? And what happened in March/April 2002?

Google-Code & ubuntutrinux

I hosted my first project (a little something called ubuntutrinux) on Google Code and was amazed how pleasant the experience was compared to Sourceforge. Not that Sourceforge is terrible, but Google's setup is just so easy. Wiki, Subversion, Issues, all in about 5 minutes. No approval needed. It tempting to say that it is because Google's infrastructure is in Python (vs. PHP crap like Sourceforge) but probably not the case. Or maybe it is.

Is there any reason to use Debian anymore?

I've been an on and off Debian user since at least Bo. So I finally broke down and did an Etch (network) install (under VMWare Fusion of course) this morning. Nice, quick, clean, non-GUI install. The way they should be. Actually I still like this install better than Ubuntu's, but I'm not sure I'll be using it much.

I still may see how painful it is to get VMware tools running under Debian (compared to Ubuntu). But since Ubuntu Server (which I was initially skeptical of) I can't really see any reason to use Debian anymore.

VMWare Fusion Beta 3 Out!

You can see the release notes for yourself, but there are some big improvements such as the ability to edit more hardware (like add drives, which I needed last night, and the presence of a floppy drive), host only networking, as well as other cosmetic changes, like the ability to remove the big ugly toolbar. So all in all, things are coming along nicely with the product. I guess the big question will be the price, which for me would be the only reason for switching to Parallels.

vmnet8: flags=8863 mtu 1500
      inet 172.16.184.1 netmask 0xffffff00 broadcast 172.16.184.255
      ether 00:50:56:c0:00:08
vmnet1: flags=8863 mtu 1500
      inet 192.168.85.1 netmask 0xffffff00 broadcast 192.168.85.255
      ether 00:50:56:c0:00:01

Are there any actively maintained ramdisk distros in 2007 that use 2.6.x?

Back in the day. Everybody did them. Hell, even I had one called Trinux: A Linux Security Toolkit which I maintained from from 1998 to 2004. There was LRP, tomsrtbt, FREESCO, HAL91, and many others. A

ll before Knoppix came and spoiled the party with their fancy talk of remastering and compressed filesystems and running from CD-ROM.

But where are they now? Devil Linux had a recent release but it uses 2.4.x. Most other became dormant in 2003-2004.

Basically I need a convenient platform for running some network tools like The Click Modular Router Project.

Am I going to have to bring Trinux back from the dead?

Dual Headed MacBook (Ubuntu)

I'm ashamed to say that although I've been using X since about 1995 (Slackware something or other) I have never tried to get it to work with two monitors. Since I find myself using Linux more lately on my MacBook it was time to get it working. Caveat: the trackpad is kind of flaky so I only recommend using Linux on the MacBooks if you are using and external keypad and mouse. There may be workaround but I haven't tried them and I only boot into Ubuntu like when I'm analyzing a 200-300MB dump in Ethereal or something.

Fortunately the Ubuntu Macbook wiki community documentation provides a solid page on how to Install Ubuntu (edgy) on Core 2 Duo MacBooks, it required a bit of tweaking and gdm [re]starting to get it working.

I created a wiki page for it containing the complete config (see the attachments link on the bottom if you want to download it) but the key step was to append the following to the existing xorg.conf file

Section "Device"
      Identifier      "MonitorLayout Device"
      Driver          "i810"
      BusID           "PCI:0:2:0"
      Option          "MonitorLayout" "NONE,CRT+LFP"
EndSection

Section "Screen"
      Identifier      "MonitorLayout Screen"
      Device          "MonitorLayout Device"
      #Monitor         "Color LCD"
      Monitor         "Generic Monitor"

      DefaultDepth    24
      SubSection "Display"
              Depth           24
              Modes           "1280x800"
      EndSubSection
EndSection

Section "ServerLayout"
      Identifier      "MonitorLayout Layout"
      Screen          "MonitorLayout Screen"
      InputDevice     "Generic Keyboard"
      InputDevice     "Configured Mouse"
      InputDevice     "stylus" "SendCoreEvents"
      InputDevice     "cursor" "SendCoreEvents"
      InputDevice     "eraser" "SendCoreEvents"
EndSection

Section "Device"
      Identifier      "Clone Device"
      Driver          "i810"
      BusID           "PCI:0:2:0"
      Option          "MonitorLayout" "CRT,LFP"
      Option          "Clone"
EndSection

Section "Screen"
      Identifier      "Clone Screen"
      Device          "Clone Device"
      #Monitor         "Color LCD"
      Monitor         "Generic Monitor"

      DefaultDepth    24
      SubSection "Display"
              Depth           24
              Modes           "1280x800"
      EndSubSection
EndSection

Section "ServerLayout"
      Identifier      "Clone Layout"
      Screen          "Clone Screen"
      InputDevice     "Generic Keyboard"
      InputDevice     "Configured Mouse"
      InputDevice     "stylus" "SendCoreEvents"
      InputDevice     "cursor" "SendCoreEvents"
      InputDevice     "eraser" "SendCoreEvents"
EndSection

Section "Device"
      Identifier      "Xinerama Device"
      Driver          "i810"
      BusID           "PCI:0:2:0"
      Screen          0
      Option          "MonitorLayout" "CRT,LFP"
EndSection

Section "Device"
      Identifier      "Xinerama Device (2)"
      Driver          "i810"
      BusID           "PCI:0:2:0"
      Screen          1
      Option          "MonitorLayout" "CRT,LFP"
EndSection

Section "Screen"
      Identifier      "Xinerama Screen"
      Device          "Xinerama Device"
Monitor  "Generic Monitor"

      #Monitor         "Color LCD"
      DefaultDepth    24
      SubSection "Display"
              Depth           24
              Modes           "1280x800"
      EndSubSection
EndSection

Section "Screen"
      Identifier      "Xinerama Screen (2)"
      Device          "Xinerama Device (2)"
Monitor  "Generic Monitor"
      #Monitor         "Color LCD"
      DefaultDepth    24
      SubSection "Display"
              Depth           24
              Modes           "1280x1024"
      EndSubSection
EndSection

Section "ServerLayout"
      Identifier      "Xinerama Layout"
      Screen          "Xinerama Screen"
      InputDevice     "Generic Keyboard"
      InputDevice     "Configured Mouse"
      InputDevice     "stylus" "SendCoreEvents"
      InputDevice     "cursor" "SendCoreEvents"
      InputDevice     "eraser" "SendCoreEvents"
      # RightOf LeftOf Above Below
      Screen          "Xinerama Screen (2)" RightOf "Xinerama Screen"
      Option          "Xinerama"
EndSection

Section "ServerFlags"
#       Option          "DefaultServerLayout" "Default Layout"
#       Option          "DefaultServerLayout" "MonitorLayout Layout"
#       Option          "DefaultServerLayout" "Clone Layout"
      Option          "DefaultServerLayout" "Xinerama Layout"
EndSection

5 Minutes with Dell PowerConnect Firmware

So over my my old employers blog (in the comments) you'll see some whining from SCADA end users about how controller/PLC/RTU vendors don't which RTOS they use. Since this is probably not unique to the SCADA world, I figured I'd see I what I could learn in 5 minutes after downloading firmware for Dell PowerConnect switches.

strings PowerConnect_34XX_boot-10101.rfb > powerconnect.txt

Then fire up TextMate and extract the juicy bits.

Even within a strings | head you would find:

CRER
4Copyright 1984-2003 Wind River Systems, Inc.;

And more interesting...

VxWorks
5.5.1
VxWorks5.5.1
Apr 10 2005, 10:47:25
usrRoot: MMU configuration failed, errno = %#x
strchr
Motorola MPC82xx ADS - HIP4
Motorola MPC82xx ADS - HIP7
Motorola MPC8260 ADS
1.21.38
FlashInit failed
motfcc(0,0)host:C:/Tornado_2_2/target/config/Carrier/vxWorks h=50.1.1.1 e=50.1.1.6 u=vxworks pw=vxworks
Out of Buffers in I2cInit(), Rx data Buffer
Out of Buffers in I2cInit(), Tx data Buffer

This is not rocket science!