BlogFranz

Good Bye BlogFranz and 20th Century

2011-09-04T12:32:00.002-06:00

Hello franzgedank.

Finally decided to give WordPress and try and leave this klunky place behind.

The chroot Hack to making Debian kFreeBSD a Usable Firewall

2011-08-14T18:09:00.009-06:00

A Rant Against Debian kFreeBSD
Don't get me wrong, I love the fact that there is a Debian distribution based on FreeBSD but if you are going to mention PF as a reason for using it, you should at least have a version of tcpdump that can read packet drops and build of pflogd so they can be logged. How useful is a firewall that you can't even tell when there are policy violations?

Not very. So I'd run across this blog and even sort of tried to apply the patches. Yes, that is the right answer (and I've been meaning to build my own Debian kFreeBSD package repository, but I'm lazy) but I today on the kFreeBSD FAQ that showed how you could run a native FreeBSD binaries in a chroot. Incidentally you can do the reverse as well, meaning run Debian *BSD chroots on top of the real FreeBSD.

Here is a quick and easy way to get tcpdump and pflogd working on a Debian kFreeBSD without having to compile anything.

Preparation
I won't repeat all the instructions from the FAQ because they just work but here are the high level steps:

Pull down the .iso for FreeBSD 8.2 (obviously pick the right architecture)
Mount the iso using mdconfig (this is basically like loopback filesystem)
Extract the sources from the install CD
Strip out the libraries and files that you need

I decided to create an /opt/native for my chroot jail

/opt/native/etc:
total 92
drwxr-xr-x  2 root root   512 Aug 14 13:26 .
drwxr-xr-x 10 root root   512 Aug 14 13:21 ..
-rw-r--r--  1 root root  1667 Aug 14 13:25 aliases
-rw-r--r--  1 root root   429 Aug 14 13:21 group
-rw-------  1 root root  1433 Aug 14 13:25 master.passwd
-rw-r--r--  1 root root  1329 Aug 14 13:22 passwd
-rw-r--r--  1 root root 40960 Aug 14 13:26 pwd.db
-rw-------  1 root root 40960 Aug 14 13:26 spwd.db


/opt/native/lib:
total 2848
drwxr-xr-x  2 root root     512 Aug 14 13:12 .
drwxr-xr-x 10 root root     512 Aug 14 13:21 ..
-r--r--r--  1 root root 1432616 Aug 14 12:08 libcrypto.so.6
-r--r--r--  1 root root   32104 Aug 14 12:08 libcrypt.so.5
-r--r--r--  1 root root 1155172 Aug 14 12:09 libc.so.7
-r--r--r--  1 root root  182668 Aug 14 12:08 libpcap.so.7
-r--r--r--  1 root root   56832 Aug 14 13:12 libutil.so.8


/opt/native/libexec:
total 244
drwxr-xr-x  2 root root    512 Aug 14 12:15 .
drwxr-xr-x 10 root root    512 Aug 14 13:21 ..
-r-xr-xr-x  1 root root 220628 Aug 14 12:15 ld-elf.so.1


/opt/native/sbin:
total 24
drwxr-xr-x  2 root root   512 Aug 14 12:11 .
drwxr-xr-x 10 root root   512 Aug 14 13:21 ..
-r-xr-xr-x  1 root root 18924 Aug 14 12:11 pflogd


/opt/native/usr:
total 8
drwxr-xr-x  4 root root 512 Aug 14 12:06 .
drwxr-xr-x 10 root root 512 Aug 14 13:21 ..
drwxr-xr-x  2 root root 512 Aug 14 12:06 bin
drwxr-xr-x  2 root root 512 Aug 14 12:10 sbin


/opt/native/usr/bin:
total 4
drwxr-xr-x 2 root root 512 Aug 14 12:06 .
drwxr-xr-x 4 root root 512 Aug 14 12:06 ..


/opt/native/usr/sbin:
total 644
drwxr-xr-x 2 root root    512 Aug 14 12:10 .
drwxr-xr-x 4 root root    512 Aug 14 12:06 ..
-r-xr-xr-x 1 root root 627792 Aug 14 12:10 tcpdump


/opt/native/var:
total 10
drwxr-xr-x  5 root root 512 Aug 14 13:24 .
drwxr-xr-x 10 root root 512 Aug 14 13:21 ..
drwxr-xr-x  2 root root 512 Aug 14 13:23 empty
drwxr-xr-x  2 root root 512 Aug 14 13:26 log
drwxr-xr-x  2 root root 512 Aug 14 13:30 run


/opt/native/var/empty:
total 4
drwxr-xr-x 2 root root 512 Aug 14 13:23 .
drwxr-xr-x 5 root root 512 Aug 14 13:24 ..


/opt/native/var/log:
total 10
drwxr-xr-x 2 root root  512 Aug 14 13:26 .
drwxr-xr-x 5 root root  512 Aug 14 13:24 ..
-rw------- 1 root root 4290 Aug 14 14:12 pflog


/opt/native/var/run:
total 6
drwxr-xr-x 2 root root 512 Aug 14 13:30 .
drwxr-xr-x 5 root root 512 Aug 14 13:24 ..
-rw-r--r-- 1 root root   5 Aug 14 13:30 pflogd.pid

Create your chroot scripts on the host filesystem

They should look like this:

root@debian:/opt# ls -al /usr/local/bin/native*
-rwxr--r-- 1 root staff 74 Aug 14 13:30 /usr/local/bin/nativepflogd
-rwxr--r-- 1 root staff 47 Aug 14 12:12 /usr/local/bin/nativetcpdump

and inside

root@debian:/opt# cat /usr/local/bin/native*

exec chroot /opt/native/ /sbin/pflogd -s 250 -i pflog0 -f /var/log/pflog 

exec chroot /opt/native /usr/sbin/tcpdump "$@"

Make sure the chroot jail can get to devices

# mount -t devfs devfs /opt/native/dev

This should show up as

# mount
devfs on /opt/native/dev (devfs, local, multilabel)

Confirm that it actually works

You can start pflogd by running /usr/local/bin/nativpflogd

root@debian:/opt# ps aux | grep pflog
root      9115  0.0  0.1   3240  1264 ?        S+   14:18   0:00 grep pflog
64        8965  0.0  0.1   9680  1508 ?        S    13:30   0:00 pflogd: [running] -s 250 -i pflog0 -f /var/log/pflog
root      8964  0.0  0.1   9680  1448 ?        Ss   13:30   0:00 pflogd: [priv]
root@debian:/opt# 

# nativetcpdump -nr /var/log/pflog 
reading from file /var/log/pflog, link-type PFLOG (OpenBSD pflog file)
17:30:47.166732 IP 192.168.56.1.64248 > 192.168.56.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
17:30:47.437045 IP 192.168.56.1.64248 > 192.168.56.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
17:30:47.707640 IP 192.168.56.1.64248 > 192.168.56.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST

The 5 Minute Guide to Running OpenVZ on CentOS 6.x

2011-07-30T16:34:00.008-06:00

As much as I like Debian (and the fact that it has OpenVZ kernels in the repos, for now...) CentOS is really the best distribution for running this container-based virtualization environment. This assumes a minimal CentOS 6.0 install.

1) Add the OpenVZ repos

See the Quick Installation guide and ensure that:

[root@opti330 yum.repos.d]# cat /etc/yum.repos.d/openvz.repo
[openvz-utils]
name=OpenVZ utilities
#baseurl=http://download.openvz.org/current/
mirrorlist=http://download.openvz.org/mirrors-current
enabled=1
gpgcheck=1
gpgkey=http://download.openvz.org/RPM-GPG-Key-OpenVZ

[openvz-kernel-rhel6]
name=OpenVZ RHEL6-based kernel
#baseurl=http://download.openvz.org/kernel/branches/rhel6-2.6.32/current/
mirrorlist=http://download.openvz.org/kernel/mirrors-rhel6-2.6.32
enabled=1
gpgcheck=1
gpgkey=http://download.openvz.org/RPM-GPG-Key-OpenVZ

2) Install the packages (after updating of course)

# yum install openvz-kernel-rhel6 vzctl vzquota bridge-utils

3) Update /etc/sysctl.conf

net.ipv4.ip_forward = 1
net.ipv4.conf.all.proxy_arp = 1

This ARP proxying is really only needed if you are doing veth networking (the default we'll use below)

4) Reboot

You kernel should now be:

[root@opti330 yum.repos.d]# uname -a
Linux opti330 2.6.32-042stab024.1 #1 SMP Tue Jul 26 15:23:12 MSD 2011 x86_64 x86_64 x86_64 GNU/Linux

5) Update /etc/sysconfig/iptables to allow traffic to/from venet0 (only if you are using venet)

-A FORWARD -i venet0 -o eth0 -j ACCEPT

-A FORWARD -i eth0 -o venet0 -j ACCEPT

Or something close. You will also want to update your iptables policy or disable it or disable it.

6) Now you can create your VE's per the instructions on my OpenVZ Wiki Page.

Creating Debian Packages with FPM for the Impatient

2011-07-28T19:20:00.006-06:00

So I've been aware of FPM for a while, but I actually hadn't got around to using it. This is an example on Debian but it should work on Ubuntu just fine.

1) Use the package maintainers version of Ruby and RubyGems.

root@debian64-10:~/tmp# apt-get install ruby ruby-dev rubygems
[snip]
Get:1 http://ftp.us.debian.org/debian/ squeeze/main rubygems1.8 all 1.3.7-3 [202 kB]
Get:2 http://ftp.us.debian.org/debian/ squeeze/main rubygems all 1.3.7-3 [66.7 kB]
Fetched 269 kB in 0s (518 kB/s)
Selecting previously deselected package rubygems1.8.
(Reading database ... 50363 files and directories currently installed.)
Unpacking rubygems1.8 (from .../rubygems1.8_1.3.7-3_all.deb) ...
Selecting previously deselected package rubygems.
Unpacking rubygems (from .../rubygems_1.3.7-3_all.deb) ...
Processing triggers for man-db ...
Setting up rubygems1.8 (1.3.7-3) ...
Setting up rubygems (1.3.7-3) ..

2) Install FPM

root@debian64-10:~/tmp# gem update; gem install fpm
Updating installed gems
Nothing to update
Building native extensions.  This could take a while...
Successfully installed json-1.5.3
Successfully installed fpm-0.3.7

3) This installed it but where does the script go? It isn't in /usr/local/bin. Nah that would be too obvious.

root@debian64-10:/# find / -name "fpm"
/var/lib/gems/1.8/bin/fpm
/var/lib/gems/1.8/doc/fpm-0.3.7/rdoc/files/lib/fpm
/var/lib/gems/1.8/gems/fpm-0.3.7/bin/fpm
/var/lib/gems/1.8/gems/fpm-0.3.7/lib/fpm

4) Update your path to included the gem wrappers and confirm that it runs

root@debian64-10:/# export PATH=$PATH:/var/lib/gems/1.8/bin
root@debian64-10:/# fpm
Missing package target type (no -t flag?)
Missing package source type (no -s flag?)
There were errors; see above.

Usage: fpm [options]
  -p, --package PACKAGEFILE        Th

To make it interesting, let's create a package that requires a dependency. The app I'm doing this for at work requires Java unfortunately, but I'll do Mono here for fun.

(I'm skipping the part here where I google for a lame Hello World C# app because I haven't touched the language since 2005 but the bottom line is I end up with .exe that will go in /usr/local/bin) so we have something like.


root@debian64-10:~/hellomono# pwd
/root/hellomono
root@debian64-10:~/hellomono# ls -al usr/local/bin/hellomono.exe 
-rwxr-xr-x 1 root root 4096 Jul 28 21:39 usr/local/bin/hellomono.exe

All we are doing here is specifying the source (a directory) and a destination target (.deb). I assume we could do the same thing for an RPM or even creepier build a Ruby gem that contained a Mono app.


root@debian64-10:~/hellomono# fpm -s dir -t deb -d mono-runtime .

This created a .deb based on your current directory


tar: ./build-deb-hellomono_1.0_amd64.deb/data.tar: file is the archive; not dumped
Created /root/hellomono/hellomono_1.0_amd64.deb
root@debian64-10:~/hellomono# ls
hellomono_1.0_amd64.deb  usr

And if we open it up


root@debian64-10:/tmp# ar -x hellomono_1.0_amd64.deb 
root@debian64-10:/tmp# ls
control.tar.gz data.tar.gz  debian-binary  hellomono_1.0_amd64.deb  hsperfdata_root
root@debian64-10:/tmp# tar xzvf control.tar.gz 
control
md5sums
root@debian64-10:/tmp# cat control
Package: hellomono
Version: 1.0
Architecture: amd64
Maintainer: 
Depends: mono-runtime
Standards-Version: 3.9.1
Section: default
Priority: extra
Homepage: http://nourlgiven.example.com/no/url/given
Description: no description given
root@debian64-10:/tmp# ls
control  control.tar.gz  data.tar.gz  debian-binary  hellomono_1.0_amd64.deb  hsperfdata_root  md5sums
root@debian64-10:/tmp# tar xzvf data.tar.gz 
./
./build-deb-hellomono_1.0_amd64.deb/
./usr/
./usr/local/
./usr/local/bin/
./usr/local/bin/hellomono.exe

Now I scp it over to an Ubuntu LTS box and install


virtual@ubuntu14:~$ sudo dpkg -i hellomono_1.0_amd64.deb 
[sudo] password for virtual: 
Selecting previously deselected package hellomono.
(Reading database ... 73505 files and directories currently installed.)
Unpacking hellomono (from hellomono_1.0_amd64.deb) ...
dpkg: dependency problems prevent configuration of hellomono:
 hellomono depends on mono-runtime; however:
  Package mono-runtime is not installed.
dpkg: error processing hellomono (--install):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 hellomono

Fix the dependencies


virtual@ubuntu14:~$ sudo apt-get install -f
Reading package lists... Done
Building dependency tree       
[snip]
Setting up hellomono (1.0) ...
Setting up libmono-system2.0-cil (2.4.4~svn151842-1ubuntu4) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place

And it runs!


virtual@ubuntu14:~$ /usr/local/bin/hellomono.exe 
Hello World
virtual@ubuntu14:~$ dpkg -l | grep hello
ii  hellomono                       1.0                                             no description given

BCFG2: A non-religious Open Source Configuration Management Tool?

2011-06-18T09:00:00.006-06:00

Yesterday I was reading a pissing match between Chef and Puppet and I'm starting to think that the only thing more religious than scripting languages (Ruby vs. Python) or web frameworks (Rails vs. Django) is configuration management tools. Just as the case with religious sects that spun off from each other due to theological (or personality) battles, dissatisfaction with cfengine begat Puppet which in turn begat Chef.

Enter BCFG2 which I discovered on Wikipedia's page on Open Source configuration management tools and this great page which compares and contrasts the 4 leading tools. Unlike these tools, this was an organic development project that was developed for internal use at Argonne National Labs and it is written in Python. Also it is non-commercial, unlike it's three other competitors. I don't think these factors are accidental.

Anyway, if you've never heard of it check out the BCF2 project page and some of the videos and see if you agree. I'm only a couple of hours in, but it definitely seems to have some interesting features that differentiate it from the alternatives, besides being a little less religious.

Meanwhile I'm testing it out on a few VMs. What I'm most interested in, is how it supports multiple Linux distributions simultaneous, in particular Debian, Ubuntu, and CentOS.

Python Boto + SSH SOCKS = Easy Cheap Coffeeshop VPN

2011-03-15T06:31:00.007-06:00

A while back (well actually after reading Unencrypted Wifi Must Die) I started using SSH SOCKS tunnels to a small AMI I created with using the AWS Console. I end up paying less than a dollar a month for this. You'll need to install Python Boto for this, but on Ubuntu 10.04 and later this is as simple as


apt-get install python-boto

The code is pretty simple

First connect to EC2


e = boto.connect_ec2()

Create a new instance where AMI_TYPE is set to 't1.micro' (the smallest cheapest AMI)


r = e.run_instances(AMI,key_name=KEY_NAME,instance_type=AMI_TYPE)
i = r.instances[-1]

You basically have to create the reservation and then pull the instance from the list of reservations. Wait for the instance to come up so you can find out the dns name


print "Host name:",i.public_dns_name

--
ec2unnel.py is available on GitHub. You obviously need to change the name to your SSH key and set your AWS environment variables or hard code them in the script.


mfranz@mfranz-xp60sublts:~$ ./ec2unnel.py stop
Connecting to EC2...
Host name: ec2-75-101-195-59.compute-1.amazonaws.com
State: shutting-down
State: shutting-down
State: shutting-down
State: shutting-down
State: shutting-down
State: shutting-down
State: terminated
mfranz@mfranz-xp60sublts:~$ ./ec2unnel.py start
Connecting to EC2...
Creating instance in: us-east-1
Launch time: 2011-03-15T12:15:05.000Z
State: pending
State: pending
State: pending
State: pending
State: pending
State: running
Host name: ec2-50-17-19-144.compute-1.amazonaws.com

mfranz@mfranz-xp60sublts:~$ ssh -v -i duo.pem -D 1080 ec2-user@ec2-50-17-19-144.compute-1.amazonaws.com
OpenSSH_5.3p1 Debian-3ubuntu5, OpenSSL 0.9.8k 25 Mar 2009
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to ec2-50-17-19-144.compute-1.amazonaws.com [50.17.19.144] port 22.
debug1: Connection established.
debug1: identity file duo.pem type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3p1 Debian-3ubuntu5
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) lang =" en_US.utf8">

And you see it takes a while for the terminated instances to go away.


mfranz@mfranz-xp60sublts:~$ ./ec2unnel.py
Connecting to EC2...
2011-03-15T12:02:23.000Z terminated
2011-03-15T12:15:05.000Z terminated

Obviously you'll need to set your proxy

Large Scale Packet Dump Analysis with MongoDB

2011-02-13T10:02:00.007-06:00

So when dealing with hundreds of MBs (or even several GBs) of packet captures spread across dozens of files, wireshark sort of breaks down, even with a fast CPU. In my case I have a laptop (with LUKS encrypted drive) so it is pretty slow. Yeah you can split them into smaller files but then you lose visibility into the complete picture when performing your queries. I think you could also write some .lua wireshark but you still have the bottleneck of tshark. So what to do? Let's back up a bit.

Over the years I've written a variety of Perl, Python, or Ruby scripts for processing .pcaps. Some that use C (or pure Python) .pcap parsers--or when I first started doing this over a decade ago just parsing the output of tcpdump and building hashes or dictionaries. Not only is this slow but you have to persist your hashes via pickling. And the challenge of the pcap libraries is they typically don't have any application layer decoding and they require a C version of the library which isn't very cross platform.

Enter pdml. I first described this in a Digital Bond blog post back in 2006. I can remember doing this on a lowly Powerbook G4 and it just worked. A year ago I was looking for a project to use MongoDB. So I wrote some code to automate the process of creating the .pdml files using wireshark and extracting the fields of interest and inserting them into a MongoDB database. I have a configuration file that specifies which PDML fields I want to extract.


[frame.len]
type = decimal

[ip.ttl]
size = 8
type = decimal

[ip.src]
size = 4
type = ipaddr

[ip.dst]
size = 4
type = ipaddr

[tcp.dstport]
size = 16
type = decimal

Because MongoDB doesn't support periods in key names (I learned this the hard way last year) I change the name of the field from ip.src to ip_src. Anything that wireshark knows about I can extract and it will become a key for that packet.

While I was still importing packets (I had close to 2 million packets in the database) I issue a query to see the unique source IPs. I can do this for any field that is supported in the PDML.


>> db.raw.distinct("ip_src")

Sun Feb 13 11:20:05 [conn17] query art0.$cmd ntoreturn:1 command: { distinct: "raw", key: "ip_src", query: {} } reslen:4457 1607ms

Or let's say I wanted to look at what are the unique TTLs.


>> db.raw.distinct("ip_ttl")

11:31:17 [conn17] query art0.$cmd ntoreturn:1 command: { distinct: "raw", key: "ip_ttl", query: {} } reslen:452 1779ms

Not bad for 2.7 million packets (and counting).

This example is pretty uninteresting because it is just standard TCP/IP headers and if you just wanted session data you could just use netflow but this far more flexible.

The downside is speed of import. Creation of the .pdml file by running tshark is very slow and the parsing of XML in Python is also not the speediest. I'm up to close to 3.1 million packets in about an hour that I've successfully imported into my database, but once they are in it is lightning fast and you are free. Where I'm (hopefully) headed today is some scripts that will create Graphviz representations of all the communications of interest perhaps like those available with Afterglow. Or I can use this to analyze and reconstruct streams from some of the proprietary protocols that I was most interested in. Or I can use this as an exercise to write a Node.js app to browse this data. The point is getting it into a useful database that allows flexible and fast queries and offloads a lot of manual tasks I would normally have to do.

Dark Horse (1999)

2011-01-22T15:01:00.009-06:00

(Something I wrote last week thanks to whoever it was in Room 227 at the Holiday Inn Express I was staying at down in Virginia had their TV turned up way too loud, before I got the nerve to call the front desk and tell them to turn it down)

When you are twenty, you have all the time in the world, but nothing to say. When you are forty, you have something to say but no time (or space) to say it. Tonight, I ran across across an set of old CDs dating back to college. They were mixed in with CDRs (backups I will never ever recover from and long unsupported OS install disks) inside one of those thick translucent CDW bags from the their showroom in Vernon Hills, IL. I was surprised they even worked. Or at least some of them did. Despite four moves in five years, I have held on to them. Mostly for the bag and the memories of driving up Milwaukee Avenue at lunchtime when I worked at Hewitt.

I've always thought we left Chicagoland way too soon, even though I know why. For once, there is no imminent move to bring on the clutter-purge of contractor bags, the sound of Legos vacuumed off carpet, that smell of Murphy's Oil Soap on hardwoods. No more "staging" for the buyer's agents or cleaning the rental and holding your breath during the moveout inspection, wondering how much will get subtracted from your security deposit.

The beltway was eerie tonight in the fog. A brief warming in between twenty degree days.. It was strange to see the spires of Mormon Temple illuminated by floodlights, which normally means you have made it, that traffic will get better, regardless of your direction.

But tonight there was no such emotion. Traffic was moving and moving fast. Only the solace of motion with a musical assist. Not unlike my flights West from Austin to San Jose in the early 2000s, staring out the window watching over the West Texas plains, the Rockies, then the Sierras, then that perilous landing over downtown.

With three kids solitude only comes during the commute or travel. I will be in a hotel room tonight. I will write.

It only took forty five minutes to get Tyson's Corner. The Cure's Pornography played “Strange Days” and it has held up over time. Most of the rest have not. Brute force existentialism seems quaint. Now you have no time for the faux nihilism, critical theory, or other youthful excesses. The soundtrack from the Until the End of the World has held up through the years, however, and “Knocking at Death's Door” seemed strangely beautiful tonight. The movie, itself, not so much. However, the vision of us all glued to our small electronic screens reading our dreams was prophetic and perhaps the EMP blast silencing everything electronic.

I'm so glad I only discovered the Internet in my last year at A&M. I am thankful for all those hours in the computer lab, printing draft after draft of my poems and short stories. Some of these I have kept to. A Liberal Arts major out of place. Marking them up at wherever I went to review them in the pre-Starbucks age. Reading T.S. Eliot. Was it just the other day on the beltway en route to Tyson's I had the urge to read Proofrock?

And Wim Wenders. It was 1992, wasn't it? The Spring of poetry readings, after getting my first short story (and that one poem) published in the campus literary magazine. Readings at coffeehouses. Night stocking at the College Station Kroger on University Avenue. Oscar, the tyrannical foreman, kneepads around the shins of his stubby legs. “Faster, white boy. Faster.” Perhaps I've conflated him with a scene from My Own Private Idaho. Curses in Spanish. The semester my wife and I started dating. January, or was it February? March is the most beautiful month in Texas, weather-wise.

Later, I would learn that in between the 2nd and 3rd week of February is when Spring came to Austin. The twin Elms of our 3/1 barely 1100 square foot house in Brentwood, had blossomed. The night before, returning home from Russia, landing at the Bergstrom, a family, home, for the first time. I trudged to the cheap lot leaving Amelia with a screaming, traumatized toddler running away from her, screaming, while I looked for our Red Dodge Dakota and struggled with a car seat for the first time. There would be a lot of screaming, but by the time we turned onto Anderson Lane (or was it North Lamar) it had stopped. Kolya looked around the small house and knew he was home. "Kolya Dom," we repeated and he pointed at the ceiling fans, bookshelves, and the popcorn ceiling.

Only two days earlier, we had left Moscow. Four days before, had left the snowy Volga at night, our facilitator emptying a whole section of the Tupelov with only whisper, so the Americans (and a lone Scottish couple) could leave with their newly acquired children. Did she buy them off? Did she threaten them? In the airport lounge we learned how Russians deal with hysterical, stressed out women. Phenobarbital.

Just like yesterday, it was snowing the morning as we left the orphanage. Perhaps because we were the youngest couple (we had not suffered through years of failed fertility treatments before deciding to adopt,) standing outside in front of the Detsky Dom, we were interviewed by Russian television. Did that ever air? Paranoia causing us to lie and say we still lived in San Antonio because that was what was on our paperwork. It was the final days of Yeltsin. They knew Putin was coming, or so we were told over homemade fruit-flavored vodka.

"The dark horse," the only English-speaking Russian on our floor in the suburban Moscow flat, called him. Our final night in Moscow we stayed a block from Red Square but only ventured out one time to a supermarket for kefir and chocolate. It was too cold and we were too exhausted.

"Twitter Segmentation": or why you need at least three accounts

2010-12-31T14:30:00.007-06:00

After I'd been using Twitter for a few months (ok, I really don't remember how long it had been) I realized that I needed two accounts for a number of reasons: privacy, personal branding, and audience. Well, when I created @seclectech I'm up to three accounts. Here is why. Besides that I take my time on Twitter way too seriously.

As much as I love shooting the shit with folks as @frednecksec that is just one side of me. And arguably not my best side. I know it. It is the the snarky, sarcastic, cynical side that can't help but mock the endless FUD about SCADA and Smart Grid Security and the never ending drumbeat of Sinophobia and generally react by picking positions I may or may not believe, just for the fun of it, and just to mess with people.

But I don't want to engage in that all the time and it is not healthy either. Sometimes, I don't want to follow people that engage in that sort of nonsense (or cause me to engage in that sort of discourse) I just want to learn about cool technical stuff (whether security or non-security related). Folks like @jonoberheide @jordansissel @jedisct1 @rgaidot consistently provide solid technical information about the topics I'm interested in (and might be interested in) without the snark (i.e. security and technology politics) of other folks I follow on @frednecksec who might be more entertaining.

If twitter had better filtering options (for example that would allow me to block hashtags such as #stuxnet or #wikileaks or #tsa) I might not need to do that, but until that happens I need two different public accounts depending on the tweets I want to produce or consume.

How did your technical skills fare in 2010?

2010-12-24T12:38:00.014-06:00

Well, my first full year at SAIC is coming to an end and it is time to take stock of what I've learned, technically, over the past year. I hope to have another blog on management, project, and customer engagement skills, because, truth be told and like it or not, I've grown more in that area than technically in the past year.

Now your day job shouldn't be the only thing determining the level of your technical skills, but it is obviously a major factor. In the last quarter, I picked up leadership/management responsibilities so I will mindful in 2011 to ensure that I at least maintain the skills I've got and not become "soft" and hands-off. This point has been made crystal clear to me as I've been reviewing folks resumes (but have seldom interviewed them) that have let their skills go. A cautionary tale. And some that that I have managed to interview, I've heard the line "I could learn that again if I needed to" one too many times. Folks hold on to the skills they hold precious, even if you have to do it on your own time or change jobs. It is just too easy, especially if you are doing well and making an impact for your organization to let things go.

Coding
The last year has been no different that other periods. Historically, coding occurs in fits and stops. At the beginning of the year I was involved in a research project where I was doing a bunch of Python development with MongoDB, but that ended early on. Near the end of the year more light scripting mostly using dpkt, scapy, and nfqueue-bindings for some protocol testing testing that will hopefully end soon so I can stop the dreaded commute two times a week to Tysons Corner.

Networking & Security Products
I really enjoyed working with ScreenOS on the lower end Juniper SSGs. Most of my firewall experience was Cisco (or PF) so it was a refreshing change of pace. It was a bit frustrating at first, both the ScreenOS CLI and WebUI are preferable to anything from Cisco has ever developed. On the other hand, I did not enjoy working with Garretcom industrial switches, especially their screwed up way of configuring trunk ports and and VLANs. Not to mention their screwed up Flash UI. Working with one of the wireless clients/bridge/APs commonly used by some of the armed services was also a mixed bag, but I learned a little bit about RF surveys and the pros and cons of Mesh vs. Bridge vs. Client/AP wireless architectures. And then there is Air Defense, which I probably haven't learned as well as I could have but there wasn't enough time. In the lab, I kept up my skills up with IOS and ASA based SSLVPNs, but this was actually something I first learned in 2009.

Design/Build/Test/Deploy
A few of my projects have been true engineering (as opposed to assessment) projects that involved specific requirements (either that were provided to us or we had to develop ourselves for the client) where we were responsible for network/system design, configuring all these components and finally bringing them to deployment and handing the components over the customer. The "big enterprise" operational/IT skills I picked up at Hewitt definitely helped here. A couple of projects have required designing and implementing a new wired and wireless network infrastructure (as well as the appropriate C&A activities necessary to connect it) and this has been an interesting challenge, sort of like putting together a puzzle as you figure out to connect all the various L2/L3 infrastructure and security devices together.

Miscellaneous Stuff
Besides fooling around with MongoDB for about a month solid, I learned that some of the RAID controllers on Compaq hardware are lame and that backing up and restoring ESX filestores can be extremely slow and painful if you have SATA drives. Hands-on experience (mostly just getting it working and defining an appropriate security architecture vs. hard core device hacking) with several different smart meters, collectors, and headends, but nothing to write home about and certainly nothing I'd pose for a newspaper for in front of my hacking gear. Last, but not least, I've had the pleasure working with a great new client of taking a deep dive into a popular SCADA system and architecting a real time vulnerability management solution that will be deployed in the new year. Didn't learn any new products here but definitely a new experience of developing a solution from scratch.

It seems like I should have done more than this, but perhaps this is even better than 2010. I learned some new products, APIs, and wrote a bit of new code. I guess it could be worse as a senior engineer for a large government contractor I could be writing technical proposals and pricing non-stop. Fortunately I've been able to dip down and keep my hands dirty now and then, in addition to the normal QA (and tough questions) to keep engineers honest and on track.

So it's been a decent year, but in 2011 I need to do better, and this will be a challenge as there is no doubt that more of my time will be involved managing employees as opposed to just leading projects.Undoubtedly I will need to do work on my own time to keep it real. Stay tuned for some goals I will define for the new year.

Greatest Hits from the Arce/McGraw Article on Cyber FUD

2010-12-04T17:39:00.003-06:00

These guys nailed it in Software [In]security: Cyber Warmongering and Influence Peddling and here are my favorite lines:

The (perhaps intentional) conceptual roll up of cyber crime, cyber espionage, and cyber war into the scariest of cyber boogeymen exponentiates the FUD factor, making an already gaping policy vacuum more obvious than ever before

Amen, I still don't even know what "CyberSecurity" really means. Back in 2003-2004 when I first heard about it I thought it was a was for "non-security folks" (putting physical security folks in that bucket) to refer to IT/Computer/Network Security. But I don't know anymore. This conflation is confusing.

The problem with these kinds of stories is that they have somehow worked their way to the halls of policymakers who repeat them without critical analysis. For every careful Dan Geer there are ten shrieking cyber security talking heads busy stirring the pot saying things like, "We may call it espionage, but it's really warfare.

The "World's Greatest Hacker" is the least of our concerns because he isn't influencing policy in the beltway.

What makes us particularly skeptical is the intentional blurring of the lines that helped to distinguish the military, the intelligence community, and the cyber security industry — a direct result of US government pouring of billions of dollars into the burgeoning maw of perpetual cyber security initiatives.

Is it any coincidence that the cyber-euphoria coincided with the US economy going to hell as IT security vendors "cyber-ize themselves." There are quite a few Austin startups (you know who you are) that have become "Cyber Security Vendors" to get that Federal money.

They point out that those beating the cyber war drums the loudest are at least partially responsible for the sorry state of affairs in computer security. Retired Director of National Intelligence (DNI) Admiral Mike McConnell bears the brunt of this criticism, as do one-time NSA Director and Deputy DNI General Mike Hayden, and one-time cyber czar Richard Clarke. We know all of these men and they are all honorable and careful. Like us, they are all capitalists as well.

Anybody that goes after "Digital Pearl Harbor" Clarke is OK in my book.

Public/private partnerships pander politically but they do no real good. As it turns out, security is not a game of ops centers, information sharing, and reacting when the broken stuff is exploited. Instead, it is about building our systems to be secure, resilient, survivable, and trustworthy.

They go after all my favorite buzzwords. The public private partnership is when the vendor and contractors (and sometimes critical infrastructure asset owners) write all the policy to their economic advantage

In conclusion, this is article is a strong defense of defense and building security in. We should let the military and the Intelligence Community do their job and the rest of us (in the Information/Network/Application/Internet Security profession) focus on ours and stop trying to play "soldier hacker." Of course the irony is some of the biggest "CyberWar Cheerleaders" have neither a background in the military, the intelligence community, or Computer Security.

Netflow on the Endpoint?

2010-11-28T16:14:00.012-06:00

So you probably most commonly think of Netflow as a router feature (where you can monitor chokepoints to identify top talkers), but over the long holiday weekend I've used it as a way to monitor behind crappy closed source SOHO APs that don't allow you to turn off NAT. I've started running netflow on some of the Linux endpoints and just for grins I enabled it on my work laptop. On the various systems I point them at a single Netflow receiver, but on my laptop I obviously point them a local receiver.

On Ubuntu/Debian it is as simple as:

apt-get install flow-tools softflowd fprobe

Softflowd and fprobe both allow you to generate Netflow datagrams to send to a netflow receiver such as flow-tools. In both cases the receivers have single configuration files in /etc/default that allow you to specify the interface to monitor and the address and UDP port of the receiver.

root@e6400:/var/flows/2010/2010-11/2010-11-28# cat /etc/default/softflowd
#
# configuration for softflowd
#
# note: softflowd will not start without an interface configured.

# The interface softflowd listens on.
INTERFACE="eth0"

# Further options for softflowd, see "man softflowd" for details.
# You should at least define a host and a port where the accounting
# datagrams should be sent to, e.g.
# OPTIONS="-n 127.0.0.1:9995"
OPTIONS="-n 127.0.0.1:3333"

root@e6400:/var/flows/2010/2010-11/2010-11-28# cat /etc/default/fprobe
#fprobe default configuration file

INTERFACE="wlan0"
FLOW_COLLECTOR="127.0.0.1:3333"

#fprobe can't distinguish IP packet from other (e.g. ARP)
OTHER_ARGS="-fip"

Since neither of these probes allow you to monitor multiple interfaces I'm having to use both to monitor my wired and wireless interfaces.

Next, I configured flow-tools by editing /etc/flow-tools/flow-capture.conf with a single line:

-w /var/flows -n 275 -N 3 127.0.0.1/127.0.0.1/3333

This stories the netflow data in the /var/flows directory and the receiver listens on 127.0.0.1:3333 which corresponds to what we had above

I found that if the directory isn't present the daemon will fail to start. This error message will show up in the logs but not on the console

When I go into work tomorrow and I plug into my dock, this should do the trick, but we'll see.

The only thing I'm not sure about is whether the daemons will correctly handled a downed interface so I may have to manually start the daemons.

Now you'll see the files are created

root@fe6400:/var/flows/2010/2010-11/2010-11-28# ls -alt | head -20
total 248
drwxr-xr-x 2 root root 4096 2010-11-28 18:10 .
-rw-r--r-- 1 root root 88 2010-11-28 18:10 tmp-v05.2010-11-28.181027-0500
-rw-r--r-- 1 root root 96 2010-11-28 18:10 ft-v05.2010-11-28.180515-0500
-rw-r--r-- 1 root root 96 2010-11-28 18:05 ft-v05.2010-11-28.180001-0500
-rw-r--r-- 1 root root 96 2010-11-28 18:00 ft-v05.2010-11-28.175448-0500
-rw-r--r-- 1 root root 96 2010-11-28 17:54 ft-v05.2010-11-28.174935-0500
-rw-r--r-- 1 root root 96 2010-11-28 17:49 ft-v05.2010-11-28.174422-0500
-rw-r--r-- 1 root root 96 2010-11-28 17:44 ft-v05.2010-11-28.173909-0500
-rw-r--r-- 1 root root 96 2010-11-28 17:39 ft-v05.2010-11-28.173356-0500
-rw-r--r-- 1 root root 96 2010-11-28 17:33 ft-v05.2010-11-28.172843-0500
-rw-r--r-- 1 root root 96 2010-11-28 17:28 ft-v05.2010-11-28.172330-0500
-rw-r--r-- 1 root root 96 2010-11-28 17:23 ft-v05.2010-11-28.171816-0500
-rw-r--r-- 1 root root 96 2010-11-28 17:18 ft-v05.2010-11-28.171304-0500
-rw-r--r-- 1 root root 96 2010-11-28 17:13 ft-v05.2010-11-28.170751-0500
-rw-r--r-- 1 root root 96 2010-11-28 17:07 ft-v05.2010-11-28.170237-0500
-rw-r--r-- 1 root root 96 2010-11-28 17:02 ft-v05.2010-11-28.165725-0500
-rw-r--r-- 1 root root 96 2010-11-28 16:57 ft-v05.2010-11-28.165212-0500
-rw-r--r-- 1 root root 346 2010-11-28 16:52 ft-v05.2010-11-28.164659-0500
-rw-r--r-- 1 root root 806 2010-11-28 16:46 ft-v05.2010-11-28.164146-0500

And most of these are empty. I should adjust the the rotation should it creates smaller files.

But I can see what sort of activity my laptop was up to while I was dealing with my youngest son's terrible in-between-two-and-three during supper.

root@e6400:/var/flows/2010/2010-11/2010-11-28# flow-cat ft-v05.2010-11-28.164146-0500| flow-print

srcIP dstIP prot srcPort dstPort octets packets
172.16.1.1 172.16.1.145 17 67 68 576 1
0.0.0.0 255.255.255.255 17 68 67 656 2
172.16.1.145 192.168.1.1 17 58772 53 61 1
192.168.1.1 172.16.1.145 17 53 34490 100 1
192.168.1.1 172.16.1.145 17 53 38384 51 1
192.168.1.1 172.16.1.145 17 53 39480 51 1
172.16.1.145 192.168.1.1 17 39480 53 51 1
172.16.1.145 192.168.1.1 17 34490 53 61 1
172.16.1.145 192.168.1.1 17 53304 53 60 1
192.168.1.1 172.16.1.145 17 53 34640 100 1
172.16.1.145 192.168.1.1 17 38384 53 51 1
192.168.1.1 172.16.1.145 17 53 58772 100 1
172.16.1.145 192.168.1.1 17 34640 53 61 1
192.168.1.1 172.16.1.145 17 53 38674 76 1
172.16.1.145 224.0.0.251 17 5353 5353 2611 9
172.16.1.145 192.168.1.1 17 38674 53 60 1

Five Quick Interview Tips for Security Folks

2010-11-14T08:33:00.008-06:00

So I've been spending 1-2 days a week for the last month or two interviewing folks for some open reqs. And yes I'm still hiring.

I've done quite a bit of interviewing (on both sides of the table) over the years in both big and small companies. In terms of the "big guys" I probably view Microsoft's process as the "gold standard." I also interviewed onsite with Amazon a while back and definitely incorporated some of what I experienced as a candidate from these west coast firms in what I do as an interviewer. Early in my days at Cisco we also did some good interviewing of candidates.

But based on recent experience, here are some things I've noticed, both in terms of tips and turn-off's. Most of these are of these are common sense.

1) Speak in concrete and specific (vs. abstract and general) terms about deliverables, responsibilities, tasks, and accomplishments. Convey a clear sense of who you are, what you like to do, and what you have accomplished. What is your career trajectory? Connect the dots for me. Even if you are "paper security" person (as opposed to a hands-on technical type) you can and should speak in specific standards, documents, and processes and data.

2) If asked about a given technology, the wrong answer is "another team did that" or "we weren't allowed to do that." Even if it is true. Find another way. This is a common problem with IT/Operational types and makes it difficult for me to envision you working in consulting, R&D, or other roles where you need to be flexible and will fill gaps where you find them.

3) Admit that you have forgotten certain technical skills if you've been "doing security" (or anything technical) for any period of time. If you say you haven't forgotten anything, you are either lying or a robot. In the long run, it is better to communicate in clear terms what you do or do not know. Plus, if you do get hired, something you claimed you were able to do (but perhaps wasn't able to be verified during the interview process, for whatever reason) may come back to haunt you as you will most certainly be passed a task.

4) When asked a seemingly factual question, the wrong answer is "I don't know" or "I could google it and find out." That is not the point. The point is to figure out what you know "around" that problem space. Plus, you are not going to get off that easy. I will take the question down a notch.

5) If asked if you can code, never, ever say you took Java/C programming in college, but haven't done any coding since then. Even if it is true. The "Modern American Poetry" classes I took are just as relevant. And resumes should err on the side of fewer skills than more skills. It makes it a pleasant surprise when you happen to know something not listed on your resume. And realistically if you put Nmap or Nessus (and most security folks do) please know what these do, because I will ask.

Hackers, Crackers, Terrorists, and Other Things You Should Care Less About

2010-09-27T18:41:00.007-06:00

@kodefupanda: Who cares who #stuxnet target was? The takeaway is that ICS security is a prob that effects us all. We need solutions not attribution.

@taosecurity: @frednecksec Attribution is necessary if you want to deal with the threat. It's not necessary if you only want to address vulnerabilities.

@frednecksec: @taosecurity Depending on who "you" are. If you are a scada admin and are behind on the vulngame, threats are somebody elses problem.

So, for quite a while now, one of my pet peeves in any security talk/whitepaper (especially a SCADA or control system security one) is when the author has a list of bullets under a headline called threats. You know, the bad guys. Typically they throw in some cheesy clip art. Even worse they will talk about the motivation: fun, profit, curiosity, world domination, etc. I always found this annoying and irrelevant. Really, who cares why someone is attacking (attempting to exploit a vulnerability in) your system and it really does not matter who they are apart from an IP address that you may or may not be able to do traceback that you may or may not be able to report to law enforcement. You focus on what you can control. Your own networks. Your own systems. Assuming you even have the time, talent, and tools to do that.

But admittedly I have a bias here. Most of my career has focused on vulnerabilities. And most of my career, I've been focused in the technical realm. Not policy or procedures. Not politics. Not targeting the bad guys (well at least after I left tactical MI, and I only targeted bad guys in warfighter command staff exercises, never in the real world.) It has been about monitoring your assets, protecting your them, ensuring devices, applications, and other hardware/software components are properly engineered so that when they are deployed operationally, they can stand up, that you've done a reasonable job reducing the attack surface, ensuring the right set of security capabilities have been implemented, that you've thought things through. You pay lip surface to threats (attack trees, threat models, etc.) but you really are only concerned about that magical moment when a threat exploits a vulnerability. That event. The goal is to prevent that or make it as unlikely as possible, or if it does happen you want to minimize the impact.

When you are concerned about technical vulnerabilities, the capabilities or intent of the threat agents don't really matter, unless there is an intersection with the assets you are responsible for monitoring or protecting--or securing prior to deployment if you are in product security or appsec. So I learn the adversary has some new tool (malware, script, or whatever) that I can detect (or not detect) that I should attempt to monitor and recover from. There is some new way of exploiting applications or network access controls or surreptitiously gaining unauthorized access. This is why you pentest, this is why you do design reviews, this is why you do operational drills. It is really not about threats. It is about your stuff. Not their stuff or them.

So if you come from this vulnerability-centric frame of mind (or at least I think I'm accurately capturing this outmoded way of thinking about the brave new world full by Cyberwar, APT, Cyberterrorism, and what my Senator this morning referred to as "Cyber Shields") you become sort of confused when folks like Bejtlich say that this no longer matters, that that this is an outmoded approach not appropriate to the 2nd decade of the 21st century. That is all failed that we must give up and go after the Chinese dragon or the Russian bear. We must stop all we are doing. Defense no longer works.

You know, sort of like a Bush Doctrine for cyberspace. Take the fight to them.

(To me there is a difference between the fact that you have to continuously stamp out vulnerabilities, over and over, Microsoft Tuesday after Microsoft Tuesday, new application or protocol. A never ending struggle that guarantees job security for a lifetime. This might be insanity, but it is not failure, but I digress)

So the big question here is who is we. What has really changed for the system administrators, the security administrators, the firewall administrators, the folks responsible for monitoring the logs, the pentesters, the application security girls, the policy and compliance weenies. They all must suddenly switch to a threat focus?

If by we, you are talking about the intelligence community if you are talking about the military, national security policy? Absolutely. Do what you need to do--or what I assumed you were already doing. Target terrorist networks with "cyber weapons" take out critical infrastructure with your cache of 0-days SCADA (or Telcom) vulnerabilities. Just do it, Cybercommand. Or whoever.

But for the rest of us, that probably aren't doing as good a job as we should monitoring our networks, patching our systems, analyzing our logs, keeping the auditors off our backs, keeping our aging systems even running as we have to do more and more with less, we are supposed to care about who the bad guys are and going after them?

For us, I say Stuxnet and Aurora (the Google one, not the smoking, shaking generator one) change nothing.

Altitude Induced Peace and Grown Up [Security] Jobs

2010-09-12T19:52:00.008-06:00

So while en route from PHX to BWI, late Thursday night after a couple of (what appear to be) successful days onsite with a new-ish client, I couldn't help but a feel a bit of satisfaction, or perhaps, more importantly, lack of restlessness that has characterized much of my infosec^H^H^H^H^Hcybersecurity career. Things actually made sense. My career had some sort of meaningful trajectory. I had not just been hopping around every 18 months for the the last decade. There was a method to the madness.

(Perhaps it is no coincidence that the we've finally settled and bought a house north of Baltimore, that I wasn't on pins and needles while out of town when talking to my wife, or that the start of the school year has gone surprisingly smoothly for my two oldest children, or that my oldest's BPD, et. al. is reasonably under control, but I digress)

Around the turn of the century (and perhaps longer), I definitely had a case of the what do I do next? What is the next big thing? How can I scratch that itch?

I got my start as a trainer, which was a monkey I had on my back for a few years, only made worse that I had a B.A. in English and History from an engineering school and that my first job out of college was as an 8th grade reading teacher of all things!

So I relished the chance to leave Trident (only after my stock options from the Veridian acquisition were safely deposited) on to a all too brief stint as a security consultant at SBC (where I mostly wrote proposals and coded tools nobody would ever use) to my 5+ years at Cisco (internal consulting, then R&D, then back again) which was just as frustrating as it was rewarding. Politics. Personalities. My own naiveté. But exposure to Big Corporate life and a hell of a lot of cool technology. Then, fleeing to a small company, which was also as rewarding as it was frustrating. Mostly the working at home thing and not having enough people to work with, which was maddening, because I found out I was more social than I thought, culminating in that fateful move to Chicago for operational IT work and a hell of a lot of Ruby coding. On call. Weekend upgrade. BSD! Then back to training because I was burned out of ops work and wanted an easy way to move back East. And I actually enjoyed teaching. Adults (in the military) and kids (when I was a middle school teacher).

What triggered my thoughts at 28,000 feet (or whatever 737-700s fly at Eastbound) was recalling how natural early in the morning it was to be up at the whiteboard in the meeting I was having: drawing network diagrams, proposing solutions, debating implementations, cracking jokes. How this was just like teaching--or at least how I liked to teach. And I realized that there was no way that I could have felt this comfortable if I hadn't had the last two (no, scratch that, three) jobs.

One of the best things about teaching at Tenable (more so with the Enterprise classes than my Nessus classes) was how I'd get all sorts of weird questions based on peculiar aspects of a student's network (or political) environment. This kept me on my toes. And before then, at Hewitt, getting just enough of a taste of the keeping-shit-running hell of operational work and large, complex, global networks that makes a SCADA system seem simple. And before that all the lessons learned from Dale (who was channeling Tom Peters) about projects and clients and who you really are. So despite the fact that I still haven't stayed in one place for more than 18 months in the last two years

But the "grown-up" part in the title? What is that about?

Perhaps because I was (or still am) only a mediocre bug-hunter (or pentester or whatever, though I loath the term) but that of vulnerability sort of work I did at Cisco (and later at Digital Bond) never seemed to make a difference. And if I'm honest about it, the learning about a new product, application, protocol, or whatever was more interesting than actually finding flaws in it. Besides, you never really knew would be fixed or not--certainly was not in the critical path, was only at the tail end. Maybe things have changed now in appsec as it has gone mainstream? Maybe I got involved in that field too early. I know I got interested in fuzzers way too early. That's for damn sure.

Although I still do some of this sort of work, these days most of what I do now involves building things, not breaking things. And it is in the critical path. Secure network & system design is in the critical path. Hell, even compliance work is in the critical path.

That cool, technical work where you could wear shorts and sandals into the office every day and never had face to face meetings with your clients and never had to record time or worry about how much time you are billing, burn rate, or profitability--that was not in the critical path.

As much as I miss Austin, that is what I'll be thinking of when I head down 95 and pull into the office in the BWI flight path tomorrow morning for the first time in a week.

A year later am I still scared? And some lessons on proposals

2010-09-02T21:04:00.009-06:00

A year ago, when I started my current job at SAIC, I referenced a Tom Peters quote about how your projects should scare you. Well, a year later, I'm no longer scared if the new job will work out, if I will hate it, if there will be enough billable work to keep me employed--or at least from spending too much time in IR&D-land. In fact I'm not scared at all, although I'm certainly being challenged. This is a good thing, obviously.

I'm at that point where I'm at thought I would be based on my standard 18 month cycle. It is interesting how this cycle has repeated itself as I've moved from job to job. About now, things are firing on all (or maybe too many) cylinders and I have the confidence that I lacked for a while. That confidence partially comes from having gotten over that "prove yourself" hump. (One of the consequences of switching jobs is that you have to prove to yourself and others at the beginning the way you don't if you stay in the same role years and years. Of course if you have a continuing stream of new projects where you have to prove yourself to new clients, you get some of that but it is not the same level of stress)

Right now, I have too many projects (this week I billed to 6-7 different charge codes, which is way too many as there is a reason you should have only 2-3 projects at a time) and tasks and they key challenge to to delegate and distribute the load so I don't burn out.

And to say no. I will never forget how during my interview at Hewitt the to-be-CISO asked me if I could say no. I don't remember my answer, but I learned why it was necessary and I certainly see that now. And the need to disengage. Tonight, I told my boss (or at least my boss for a few more days now if the rumors are true) that I was taking the night off and my wife and I watched The Edge of Heaven. Highly recommended if a bit slow moving at first. No, I wouldn't be working on that proposal tonight.

Which leads me to the original idea for this blog. So proposals? What have I learned about proposals? Right now, I'm going through one of those frustrating periods at work were I'm doing less technical work and more proposals and pricing than I'd like. These periods are a necessary evil and they are worth the thrill of the award, but they are not fun. I thought I hated technical proposals but pricing is worse. And there are things worse than pricing, but I wont go there.

So it seems like I've done a lot of proposals in the last year. Maybe I have maybe I haven't relative to folks in similar roles in similar sized companies, but there has definitely been variety. A few technical proposals I've written from scratch (more or less) and done all the pricing and am now leading the project but most where I'm part of a larger proposal team. And this is where the idea for this blog came in mind as I was stuck in traffic on 695 this afternoon.

What are some lessons of writing projects proposals (or at least participating in proposal teams) from the last year?

Who is the boss? This seems easy but it doesn't always happen. Even if there are multiple PMs or team leads involved, you need to appoint one who will run the proposal process. Ambiguity as to who is driving the process can be a disaster. It will screw it up.

Err on the side of more conference calls and less emails. I can't believe I'm saying this. Especially if there are folks that haven't worked together before or are from different organizations. You have to build up that trust. Don't hesitate to pick up the phone. Just sending out calls to contribute and review a document isn't going to cut it.

Divide and conquer. Assign tasks to specific questions in the RFP. Yeah, the folks assigned may screw it up and somebody may need to clean it up and rescue that section, but there needs to be clarity on who has to write what.

Tell folks when things are due. When does the client need it? What the internal process hurdles? When are you tasks due. Open ended tasks also won't get done. Especially if folks are using spare cycles they need to know when to perform that context switch.

Get some sleep. Yes, since proposals are typically on top of your normal project it does require nights and weekend.

And on that note.

Finally Went Android (Or, My Favorite Apps)

2010-08-09T19:35:00.007-06:00

So last month I finally broke down and got an Android phone. I'd my eye on them for a while. I've never had a desire to get an iPhone, mostly due to my aversion to AT&T, but also because I am not an iPhone guy just like I wasn't an Apple guy in high school (not suprisingly, I was a Kaypro and then a C-64 guy) although I've been through a couple of Mac phases over the years.

I went with a much maligned Motorola Cliq XT, mostly because it was cheap, small, and fairly rugged. That is my preference for cell phones over the years. A lot of folks complained about Motoblur but it really isn't that bad. Realistically you don't have to even use it. Delete the bubbles off your home desktop and you really won't know it is there.

The Cliq XT It is not perfect but hey it is just a phone! Plus I'm chained to my BlackBerry so it doesn't really matter if it reboots while I'm on a call. Someone can get ahold of me on that. Yeah I know it is still Android 1.5, but I don't care. In terms of usability for non-email tasks it is such an improvement over my BlackBerry. I also like the fact of having two different carriers. I had no issue with Sprint but their upgrade prices were awful and I got free activation from T-Mobile.

(Pro tip: if you are unfortunate to work in a large company at least check for their corporate discounts with cell providers. Not only do you save money but you get better service when you call in.)

So what are my favorite apps?

Seesmic. That is a no brainer. No ads. Works for high volume account. I tried the "Happenings" app for my low volume account. Forget about it.

Opera Mini - I really haven't given the built-in browser a chance. Haven't wanted to.

Easy Tether -- I really only use this as a backup but it works on Ubuntu 10.04 and Windows without issue. I did manage to do an adb shell, but never got the SOCKS proxy working.

Some Battery Widget on my desktop that allows me to toggle GPS/Wifi.

Connect Bot -- I still haven't generated public keys, but it is more useful than you would think. Still haven't figured out how to do a CTRL-C, so unless you know how, don't use "top"

Pandora -- everybody know what Pandora is. I use this to help put my toddler to sleep. Try the "Lucinda Williams" channel it works, although makes you miss Texas.

Android System Information - this barely makes the cut but at least I haven't deleted it.

Advanced Task Killer Free - I really haven't touch this much, but I think it is doing something?

Pictures from Texas!

2010-04-11T09:40:00.002-06:00

Spring Break 2010

218!

2010-03-21T18:32:00.002-06:00

So I've been listing too much of the raucous debate in the house about health care on C-SPAN.

Politico provides the best analysis of the behind-the-scenes action in what led to where we are:

The rebirth of the reform effort is the result of a little luck, insurance company avarice, a subsiding of post-Brown panic among party incumbents and the calculation by many Hill Democrats that going small or giving up was just as politically perilous as going big.

But the main reason the bill has made it to the floor has as much to do with the complex, occasionally tense, ever-evolving partnership between the first African-American president and the first female speaker.

and

Publicly, the White House seemed to send a different signal each day.

In the space of two weeks, Obama or his top advisers suggested breaking the bill into smaller parts, keeping it together in one comprehensive package, putting it at the back of legislative line and needing to “punch it through” Congress, as Obama himself said at one point.

At a fundraiser in early February, Obama described the “next step” as sitting down with Republicans, Democrats and health care experts, describing a process that could take weeks, if not longer. He also seemed to acknowledge for the first time that Congress may well decide to scrap health care altogether — an admission that blunted his repeated and emphatic vows to finish the job.

Behind the scenes, Obama had, in fact, already settled on a strategy.

He would invite Republicans and Democrats to a summit, to give them one last chance at compromise, knowing they wouldn’t budge. And privately, he had decided that his favored approach was a comprehensive bill.

However, what I've been thinking about over the last few days is the similarity between how health care reform, Hillary, and McCain were handled by the Obama team. See my from October 08 where I mentioned an Andrew Sullivan article of how Obama's calm (and sometimes perceived weakness) lures the opponents into a false sense of victory and incites them. It was easy to doubt him during the primary (why wasn't he more aggressive against Clinton) and during the Summer of 08 (why did he take that stupid trip to Europe) but in all cases (assume this goes through) he pulls it off.

Will this really make a difference? I don't know and I really don't care. I knew enough to support it: the exclusions about pre-existing conditions (which impacted my family first hand) was good enough for me, although I know how much I paid for my plan at Cisco a decade ago, probably 1/5 in a comparable-sized company. As if the campaign of 2008 (the selection of Palin) didn't prove it enough, the GOP continues to show it's true colors and they don't match mine anymore. Even though I have elected more Republicans than Democrats over the years (not like you really have a choice in Texas) it is hard to know whether the GOP is really that ignorant or are they just that deceptive in order to go "downscale" Republican base that Bush built. As always, I'm a political reactionary. For me, just as my vote for Bush in 2004 was more a vote against Michael Moore and his kind, my support for this bill is just as much about giving Limbaugh and Beck the finger as passing needed reforms. And the former is a lot more certain than the latter.

Installing CouchDB/CouchApp on 64 Bit Debian 5.x

2010-02-21T09:16:00.003-06:00

So the manual installation for CouchApp is mostly correct, but here were some slight modifications.

NOTE: my installation assumed everything that was custom-compiled went into /opt to keep a clean segregation of anything that is part of the distribution vs. hand compiled.

1. Install erlang from source (V5.7.4). I removed the 5-6 erlang packages but still got the dependencies. I also had to add libncurses5-dev to the list of packages for Erlang to compile.

./configure --with-ssl --prefix=/opt

2. Of course now I had to use the following when building CouchDB:

./configure --prefix=/opt --with-erlang=/opt/lib/erlang/usr/include/

3. Adjust paths accordingly to /opt/var/lib/couchdb /opt/var/log/couchdb etc.

NOTE: This VZ was running on 64bit OpenVZ

Linux debian5amd64-50 2.6.26-2-openvz-amd64 #1 SMP Thu Feb 11 01:40:09 UTC 2010 x86_64 GNU/Linux

Did the Stimulus Work?

2010-02-18T21:05:00.004-06:00

Spending 2-3 hours on the D.C. beltway (yeah it took me that long to get from Rockville to Ellicot City tonight) does not put you in best frame of mind, but I took a break from tech stuff and catch up on some politics, for a change and actually blog instead of tweet. Apart from the campaign-style movie above, there is Judging the Stimulus by Job Data Reveals Success with the key argument as

The case against the stimulus revolves around the idea that the economy would be no worse off without it. As a Wall Street Journal opinion piece put it last year, “The resilience of the private sector following the fall 2008 panic — not the fiscal stimulus program — deserves the lion’s share of the credit for the impressive growth improvement.” In a touch of unintended irony, two of article’s three authors were listed as working at a research institution named for Herbert Hoover.

Of course, no one can be certain about what would have happened in an alternate universe without a $787 billion stimulus. But there are two main reasons to think the hard-core skeptics are misguided — above and beyond those complicated, independent economic analyses.

The first is the basic narrative that the data offer. Pick just about any area of the economy and you come across the stimulus bill’s footprints.

In the early months of last year, spending by state and local governments was falling rapidly, as was tax revenue. In the spring, tax revenue continued to drop, yet spending jumped — during the very time when state and local officials were finding out roughly how much stimulus money they would be receiving. This is the money that has kept teachers, police officers, health care workers and firefighters employed.

Then there is corporate spending. It surged in the final months of last year. Mark Zandi of Economy.com (who has advised the McCain campaign and Congressional Democrats) says that the Dec. 31 expiration of a tax credit for corporate investment, which was part of the stimulus, is a big reason.

Let's hope so.

A Maze of Twisty Fuzzers All Alike

2010-02-04T20:48:00.005-06:00

Funny how a single innocent tweet can stir the pot. Not that I'm disappointed or that I mind, because the pot definitely needed to be stirred, but that certainly wasn't my intent on Monday. Really. But let's back up.

So I gave a short, not terribly technical presentation on Open Source fuzzing tools right before lunch at a conference on vulnerability discovery hosted by CERT at their office in Arlington. It was go to back there as I'd been to the SEI offices back in 2006 when I was working with them on the disclosure of some SCADA vulns.

Unfortunately, I didn't get to stick around the whole day (I missed Jared DeMott's presentation) and I was in and out during a conference call but there interesting talk by CERT, CERT-FI, Secunia, and Codenomicon.

But most interesting, and what led to my innocent tweet was a talk by Microsoft on how they use fuzzing and what were the results of different tools and approaches.

The conclusion I found to be surprising that they found that the use of "smart fuzzers" to have a lower ROI than the use of "dumb fuzzers" and their whitebox fuzzing platform called SAGE. Their point was the time it takes to define, model, and implement the protocol in a smart fuzzer is in most cases better spent having less skilled engineers run dumb fuzzers or white box tools.

They mentioned a talk at Blue Hat Security Briefings (I don't think this is the actual talk, but I don't have time to look for it) where they presented the bug results on a previously untested application were tested by a internally written (smart fuzzer), Peach (the dumb fuzzer?) and their whitebox fuzzing platform called SAGE. They mentioned an interesting technique of taking "major hashes" and "minor hashes" on the stack traces to isolate unique bugs. This is interesting because the primary focus has been on reducing the number of unique test cases but another approach is to look at the results. It may end up being more efficient. Of course this assumes the ability to have instrumented targets which may not always be the case, for example with embedded systems.

So Dale picked up on this and tried to apply this to the world of SCADA

We have two security vendors that are trying to sell products to the control system market: Wurldtech with their Achilles platform and Mu Dynamics with their Mu Test Suite. [FD: Wurldtech is a past Digital Bond client and advertiser] One of the features of these products is they both send a large number of malformed packets at an interface – - typically crashing protocol stacks that have ignored negative testing.

Mu responded within the comments in the blog and Wurldtech (far more defensively) on their own blog

In fact, our CTO Dr. Kube even gave a presentation at Cansecwest almost 2 years ago called “Fuzzing WTF” which was our first attempt to re-educate the community. To bolster the impact, we invited our friends at Codenomicon to help as they also were frustrated with the community discourse. The presentation can be found here.

Well I guess thie "re-education" (which sounds vaguely Maoist, I guess some of us need to be sent to a Wurldtech Fuzzing Re-education program) hasn't exactly worked although a satisfied Wurldtech customer did chime in on the Digital Bond blog. I actually agree that the need for better descriptions of fuzzing tools capabilities is needed and that was the entire point of my talk. I did a survey of the features available several dozen fuzzing tools and fuzzing frameworks that could be used to test.

I didn't spend as much time on the actual message generation as I should have and I was only focusing on Free and Open Source tools, but I identified a number of attributes for comparison such as target, execution mode, language, transport, template (generation, data model, built-in functions), fault payloads, debugging & instrumentation, and session handling. I'm not sure I completely hit my target but one of my goals was to develop some criteria to help folks make better choices on which Open Source tools could be used to most efficiently conduct robustness testing of your target. One of my conclusions (which I was pleased to hear echoed in the Microsoft talk) is that no single tool is best, no single approach is adequate--and that there are different types of fuzzing users that will require different feature sets. A QA engineer (that may have little to no security expertise) requires different features from those required for a pen-tester (or perhaps security analyst as part of a compliance-based engagement) which are still different from a hard core security researcher.

And the same applies to commercial tools you are paying tens of thousands of dollars for. One size does not fit all, regardless of the marketing (or mathematical) claims of the vendor. It would definitely be good to see a bakeoff of the leading commercial and Open Source fuzzing/protocol robustness tools similar to what Jeff Mercer has been doing for webapp scanners but I'm not optimistic that we will see that on the commercial tools because they are too expensive and the primarily customers for these tools (large vendors) are not going to disclose enough details about the vulnerabilities discovered to provide a rich enough data set for comparison.

It won't be me but perhaps some aspiring young hacker will take the time to do a thorough comparing the coverage of the tools that are out there against a reference implementation -- instead of writing yet another incomplete, poorly documented Open Soure fuzzer or fuzzing framework.

Hello MongoDB (Jython Style)

2010-01-13T19:44:00.003-06:00

It has been ages since I've played around with any of the Java scripting languages so I thought I'd give Jython a spin with MongoDB. I have no idea about the performance between the pure Python vs. Java driver but it would be an interesting benchmark.

This is a very quick code snippet based on the MongoDB Java tutorial.

This was done on Ubuntu 9.10 with OpenJDK in the standard repositories and assumes the jython shell script is in your path. It also assumes the Java MongoDB driver is in your path and I was lazy so I didn't bother with CLASSPATH.

#!/usr/bin/env jython
import sys
sys.path.append("mongo-1.2.jar")
from com.mongodb import *
print "Jython MongoDB Example"
m = Mongo("10.0.0.33")
db = m.getDB("grid_example")

for c in db.getCollectionNames():
print c

And the output is just what you'd expect.

mfranz@karmic-t61:~/Documents/mongo$ ./jymongo.py
Jython MongoDB Example
fs.chunks
fs.files
system.indexes

Avoiding Bracket Hell in MongoDB Queries (Python Style)

2010-01-13T12:47:00.007-06:00

To me it wasn't immediately obvious from the MongoDB Advanced Query documentation that you can string together multiple operators to perform existence, membership, and greater/than that tests. And since JSON can get very messy (and long!) and the syntax is slightly different from the Javascript in the documentation, instead of passing JSON directly to the find method of your collection pass a dictionary and assign the various conditions

For example:

myq = {}

myq["batchstamp"] = b # a timestamp

myq["modbus_tcp_reference_num"] = {"$exists": True}

cur = coll.find( myq )

Although it doesn't appear much easier than passing

{'modbus_tcp_reference_num': {'$exists': True}, 'batchstamp': 999999999}

Once start adding additional conditions (themselves which may have dictionaries it is much easier and less error prone. Trust me!

PyMongo for Dummies (using Squid logs, again)

2010-01-10T18:54:00.003-06:00

In my last blog I showed some examples form the MongoDB shell. Next, we'll go through the PyMongo API, since only crazy people code in JavaScript.

In [3]: c = pymongo.Connection("192.168.169.62")
In [4]: db = c.mongosquid
In [5]: raw = db.raw In [6]: raw
Out[6]: Collection(Database(Connection('192.168.169.62', 27017), u'mongosquid'), u'raw')

We could have also referred to our collection as db["raw"] or db[coll] if you needed to define the collection in a variable.

In [7]: raw.count()
Out[7]: 205339

You can find out the methods that belong to the database with the collection_names() method.

In [40]: db.collection_names()
Out[40]: [u'raw', u'system.indexes']

The find_one() method allows you to quickly inspect your collection and take a peek at a sample document.

In [10]: raw.find_one()
Out[10]:

{u'_id': ObjectId('4b496cddb15cb004a4000000'), u'format': u'-', u'method': u'GET', u'size': 824477.0, u'source': u'192.168.1.254', u'squidcode': u'TCP_MISS/200', u'stamp': 1263096815.7609999, u'url': u'http://netflix086.as.nflximg.com.edgesuite.net/sa0/166/1680180166.wmv/range/660083845-660907844?'}

The distinct() method does have some limitations, as I discovered the hard way, as you an see from this exception.

In [13]: raw.distinct("stamp") --------------------------------------------------------------------------- OperationFailure Traceback (most recent call last) /root/ /usr/lib/python2.4/site-packages/pymongo-1.3-py2.4-linux-i686.egg/pymongo/collection.pyc in distinct(self, key) /usr/lib/python2.4/site-packages/pymongo-1.3-py2.4-linux-i686.egg/pymongo/cursor.pyc in distinct(self, key) /usr/lib/python2.4/site-packages/pymongo-1.3-py2.4-linux-i686.egg/pymongo/database.pyc in _command(self, command, allowable_errors, check, sock)

OperationFailure: command SON([('distinct', u'raw'), ('key', 'stamp')]) failed: assertion: distinct too big, 4mb cap

So in my previous blog (using JavaScript) I introduced queries but you really can't do anything useful without using a cursor. If you've ever done any MySQL coding before you should be familiar with the concept. Basically it allows you to iterate through the results of a query.

Here we have the same expressions but you obviously need to quote the gt in Python.

In [29]: c = raw.find( {'stamp': { "$gt": 1263096815 }})
In [31]: c.count()
Out[31]: 2060

and

In [23]: c = raw.find({'squidcode':'TCP_DENIED/403'})
In [24]: c.count()
Out[24]: 2999

For the sake of this exercise, we only want to see 3 results so we call the limit() method.

In [26]: c.limit(3)
Out[26]:

Now we can iterate through the results of our query.

In [27]: for e in c:
....: print e
....:
....:

{u'squidcode': u'TCP_DENIED/403', u'format': u'-', u'stamp': 1262520969.721, u'source': u'192.168.1.254', u'url': u'http://www.bing.com/favicon.ico', u'_id': ObjectId('4b496ea4b15cb004a6000000'), u'method': u'GET', u'size': 1419.0}

{u'squidcode': u'TCP_DENIED/403', u'format': u'-', u'stamp': 1262521126.928, u'source': u'192.168.1.254', u'url': u'http://www.msn.com/', u'_id': ObjectId('4b496ea4b15cb004a600003e'), u'method': u'GET', u'size': 1395.0}

{u'squidcode': u'TCP_DENIED/403', u'format': u'-', u'stamp': 1262521127.654, u'source': u'192.168.1.254', u'url': u'http://www.bing.com/favicon.ico', u'_id': ObjectId('4b496ea4b15cb004a600003f'), u'method': u'GET', u'size': 1419.0}

So if we try again, what happens?

In [28]: for e in c:
print e
....:
....:

Nada. We have to rewind the cursor object to be able iterate again.

In [30]: c.rewind()
Out[30]:
In [31]: for e in c:
print e ....: ....:

{u'squidcode': u'TCP_DENIED/403', u'format': u'-', u'stamp': 1262520969.721, u'source': u'192.168.1.254', u'url': u'http://www.bing.com/favicon.ico', u'_id': ObjectId('4b496ea4b15cb004a6000000'), u'method': u'GET', u'size': 1419.0}

You can also manually iterate through these by calling next()

In [51]: cr.next()
Out[51]:

{u'_id': ObjectId('4b496ea4b15cb004a6000000'), u'format': u'-', u'method': u'GET', u'size': 1419.0, u'source': u'192.168.1.254', u'squidcode': u'TCP_DENIED/403', u'stamp': 1262520969.721, u'url': u'http://www.bing.com/favicon.ico'}

In [52]: result = cr.next()

Guess what, your limit will still apply so if you want to clear it you can do a cr.rewind() and cr.limit(0) and then you can manually iterate through with cr.next()

Dummies Guide to MongoDB Queries using Squid Logs (JavaScript Shell Edition)

2010-01-10T00:15:00.007-06:00

So the MongoDB develop documentation is actually pretty decent, but it doesn't really use examples with real data. For me, it made it more difficult for some of the API and shell commands to sink in.

So to generate some real world queries I created a python script that parsed the access.log file[s] generated by squid. I'll follow this blog with one that covers pymongo but I think this will be helpful, and like most of the posts will provide a good reference because when you are rapidly approaching 40 not only your eyes go, but your memory. So here goes...

First of all this assumes you are running the mongo JavaScript shell and yeah I know running from root is a bad idea and not even necessary (I don't think) but sue me.

root@opti620:~/mongodb# ./bin/mongo

MongoDB shell version: 1.2.1

url: test

connecting to: test

type "help" for help

> show dbs

admin

local

mongosquid

test

> use mongosquid

switched to db mongosquid

> show collections

raw

system.indexes

Now let's have some fun. This was actually when I just imported a few lines in from the log file so there are a relatively small number of documents. A collection is essentially like a table but since this is #nosql it really isn't a table. It is just collection of documents. We'll see those next.

> db.raw.find().count()

1029

> db.raw.find()[1029]

> db.raw.find()[1028]

{

"_id" : ObjectId("4b496cddb15cb004a4000404"),

"squidcode" : "TCP_MISS/200",

"source" : "192.168.1.254",

"stamp" : 1263102993.841,

"format" : "-",

"url" : "agmoviecontrol.netflix.com:443",

"method" : "CONNECT",

"size" : 17499

}

The JSON above is the "document." Something you'll notice is there are two different data types basically strings and floating points. The size field and timestamp are obviously floats. That hash looking thing is actually a hash or GUID that is supposedly unique.

So one of the cool built in queries is to return only the unique values for a given field. This is handled by the distinct method.

So we can see here that there were HTTP Posts.

> db.raw.distinct("method")

[ "CONNECT", "GET" ]

And because of my screwed up natting I can't tell which of my kids was going to netflix.

> db.raw.distinct("source")

[ "192.168.1.254" ]

> db.raw.distinct("url")

....

"http://netflix086.as.nflximg.com.edgesuite.net/sa0/725/1985205725.wma/range/9247565-9735184?",

"http://netflix086.as.nflximg.com.edgesuite.net/sa0/725/1985205725.wma/range/9735185-10219794?",

"http://netflix086.as.nflximg.com.edgesuite.net/sa0/725/1985205725.wma/range/985115-1469724?"

So remember when I discussed types above, if we wanted to retrieve all the transactions that were greater than 1MB we could do the following, but there are obviously more to it than that.

> db.raw.find( {size: { $gt:1000000}} )
{ "_id" : ObjectId("4b496cddb15cb004a4000162"), "squidcode" : "TCP_MISS/200", "source" : "192.168.1.254", "stamp" : 1263097489.996, "format" : "-", "url" : "http://netflix086.as.nflximg.com.edgesuite.net/sa0/166/1680180166.wmv/range/143155845-144163844?", "method" : "GET", "size" : 1008478 }
{ "_id" : ObjectId("4b496cddb15cb004a40003b0"), "squidcode" : "TCP_MISS/200", "source" : "192.168.1.254", "stamp" : 1263099100.207, "format" : "-", "url" : "http://netflix086.as.nflximg.com.edgesuite.net/sa0/166/1680180166.wmv/range/400771845-401779844?", "method" : "GET", "size" : 1008478 }

I was pleased to find that you can use regular expressions. The first query tells me there are 3199 documents that have port 443 in them and the 2nd query returns the first document. One of the things I noticed is that retrieving the document based on the "index" is really really slow. But I believe that is because it isn't really an index, but we'll get to them later.

> db.raw.find ( { url: /:443/ }).count()
3199
> db.raw.find ( { url: /:443/ })[0]
{
"_id" : ObjectId("4b496cddb15cb004a4000093"),
"squidcode" : "TCP_MISS/200",
"source" : "192.168.1.254",
"stamp" : 1263096929.091,
"format" : "-",
"url" : "agmoviecontrol.netflix.com:443",
"method" : "CONNECT",
"size" : 96222
}
> db.raw.find ( { url: /:443/ })[0:3]
Sun Jan 10 01:16:11 JS Error: SyntaxError: missing ] in index expression (shell):0

You'll notice that array slices don't work, but they do in Python, obviously which I'll blog on next.

FreeBSD 8.0 with rum0 and wpa_supplicant on Lenovo S10-2

2010-01-09T13:44:00.004-06:00

It looks like the driver for rum has changed slightly in FreeBSD 8.0 from FreeBSD 7.2 because I was not able to use the same command-line syntax as I did previously. Basically the only thing different I did was the ifconfig wlan create...

I had this card running on old Dell Optiplex acting as a bridge for my kids network (and they were watching a lot of streaming media) and I was surprisingly impressed with it. Decent performance.

mfranz-bsd8#
ugen4.3:  at usbus4
rum0:  on usbus4
rum0: MAC/BBP RT2573 (rev 0x2573a), RF RT2528

mfranz-bsd8# cat /etc/wpa_supplicant.conf
network={
 ssid="xxx"
 psk="xxxx"
}


mfranz-bsd8# ifconfig wlan create wlandev rum0
wlan0
mfranz-bsd8# ifconfig wlan0
wlan0: flags=8802 metric 0 mtu 1500
 ether 00:1c:10:e6:1a:02
 media: IEEE 802.11 Wireless Ethernet autoselect (autoselect)
 status: no carrier
 ssid "" channel 1 (2412 Mhz 11b)
 country US authmode OPEN privacy OFF txpower 0 bmiss 7 scanvalid 60
 bgscan bgscanintvl 300 bgscanidle 250 roam:rssi 7 roam:rate 1
 bintval 0
mfranz-bsd8#

mfranz-bsd8# wpa_supplicant -c /etc/wpa_supplicant.conf -i wlan0
CTRL-EVENT-SCAN-RESULTS
Trying to associate with xxxxxxxxxx (SSID='xxxxxxxx' freq=2437 MHz)
Associated with xxxxxxxxxxx
WPA: Key negotiation completed with xxxxxxxxxxx [PTK=CCMP GTK=TKIP]
CTRL-EVENT-CONNECTED - Connection to xxxxxxxxxx completed (auth) [id=0 id_str=]

And while I'm at it, I hadn't seen any who actually installed 8.0 on a Lenovo Netbook but so far so good. I've got X working (I'll blog on that later) and re seems to work well enough. Obviously the Broadcom 4312's aren't going to work, but if you have USB wifi card or a tether you will be ok.

Next step see if I can get my Novatel u727 card working. I suspect it should work just fine, because it worked well on OpenBSD, but you never know...


Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
       The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:48:17 UTC 2009
   root@almeida.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Atom(TM) CPU N270   @ 1.60GHz (1602.40-MHz 686-class CPU)
 Origin = "GenuineIntel"  Id = 0x106c2  Stepping = 2
 Features=0xbfe9fbff
 Features2=0x40c39d>
 AMD Features2=0x1
 TSC: P-state invariant
real memory  = 1073741824 (1024 MB)
avail memory = 1026433024 (978 MB)
ACPI APIC Table: 
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 1 core(s) x 2 HTT threads
cpu0 (BSP): APIC ID:  0
cpu1 (AP/HT): APIC ID:  1
ioapic0: Changing APIC ID to 4
ioapic0  irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0:  on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit> port 0x408-0x40b on acpi0
acpi_ec0:  port 0x62,0x66 on acpi0
acpi_hpet0:  iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
acpi_button0:  on acpi0
acpi_lid0:  on acpi0
acpi_button1:  on acpi0
pcib0:  port 0xcf8-0xcff on acpi0
pci0:  on pcib0
vgapci0:  port 0x60f0-0x60f7 mem 0x58280000-0x582fffff,0x40000000-0x4fffffff,0x58300000-0x5833ffff irq 16
at device 2.0 on pci0
agp0:  on vgapci0
agp0: detected 7932k stolen memory
agp0: aperture size is 256M
vgapci1:  mem 0x58200000-0x5827ffff at device 2.1 on pci0
pci0:  at device 27.0 (no driver attached)
pcib1:  at device 28.0 on pci0
pci1:  on pcib1
pcib2:  at device 28.1 on pci0
pci2:  on pcib2
pci2:  at device 0.0 (no driver attached)
pcib3:  at device 28.2 on pci0
pci3:  on pcib3
re0:  port 0x2000-0x20ff mem 0x52010000-0x52010fff,0x52000000-0x5200ffff irq 18 at
device 0.0 on pci3
re0: Using 1 MSI messages
re0: Chip rev. 0x24800000
re0: MAC rev. 0x00400000
miibus0:  on re0
rlphy0:  PHY 1 on miibus0
rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
re0: Ethernet address: 00:26:22:0b:07:28
re0: [FILTER]
pcib4:  at device 28.3 on pci0
pci4:  on pcib4
uhci0:  port 0x60a0-0x60bf irq 16 at device 29.0 on pci0
uhci0: [ITHREAD]
uhci0: LegSup = 0x0f00
usbus0:  on uhci0
uhci1:  port 0x6080-0x609f irq 17 at device 29.1 on pci0
uhci1: [ITHREAD]
uhci1: LegSup = 0x0f00
usbus1:  on uhci1
uhci2:  port 0x6060-0x607f irq 18 at device 29.2 on pci0
uhci2: [ITHREAD]
uhci2: LegSup = 0x0f00
usbus2:  on uhci2
uhci3:  port 0x6040-0x605f irq 19 at device 29.3 on pci0
uhci3: [ITHREAD]
uhci3: LegSup = 0x0f00
usbus3:  on uhci3
ehci0:  mem 0x58344400-0x583447ff irq 16 at device 29.7 on pci0
ehci0: [ITHREAD]
usbus4: EHCI version 1.0
usbus4:  on ehci0
pcib5:  at device 30.0 on pci0
pci5:  on pcib5
isab0:  at device 31.0 on pci0
isa0:  on isab0
atapci0:  port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x60c0-0x60cf irq 16 at device 31.1 on pci0
ata0:  on atapci0
ata0: [ITHREAD]
atapci1:  port 0x60d8-0x60df,0x60fc-0x60ff,0x60d0-0x60d7,0x60f8-0x60fb,0x6020-0x602f mem 0x583440
00-0x583443ff irq 17 at device 31.2 on pci0
atapci1: [ITHREAD]
atapci1: AHCI called from vendor specific driver
atapci1: AHCI v1.10 controller with 4 1.5Gbps ports, PM not supported
ata2:  on atapci1
ata2: [ITHREAD]
ata3:  on atapci1
ata3sm0:  irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: [ITHREAD]
psm0: model Generic PS/2 mouse, device ID 0
cpu0:  on acpi0
est0:  on cpu0
p4tcc0:  on cpu0
cpu1:  on acpi0
est1:  on cpu1
p4tcc1:  on cpu1
pmtimer0 on isa0
orm0:  at iomem 0xcf000-0xcffff pnpid ORM0000 on isa0
sc0:  at flags 0x100 on isa0
sc0: VGA <16 flags="0x300">
vga0:  at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: parallel port not found.
Timecounters tick every 1.000 msec
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 12Mbps Full Speed USB v1.0
usbus4: 480Mbps High Speed USB v2.0
ad4: 152627MB  at ata2-master SATA150
ugen0.1:  at usbus0
uhub0:  on usbus0
ugen1.1:  at usbus1
uhub1:  on usbus1
ugen2.1:  at usbus2
uhub2:  on usbus2
ugen3.1:  at usbus3
uhub3:  on usbus3
ugen4.1:  at usbus4
uhub4:  on usbus4
: [ITHREAD]
GEOM: ad4: partition 1 does not start on a track boundary.
GEOM: ad4: partition 1 does not end on a track boundary.
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
Root mount waiting for: usbus4
Root mount waiting for: usbus4
Root mount waiting for: usbus4
uhub4: 8 ports with 8 removable, self powered
Root mount waiting for: usbus4
Root mount waiting for: usbus4
ugen4.2:  at usbus4
Trying to mount root from ufs:/dev/ad4s2a
ugen0.2:  at usbus0
ums0:  on usbus0
ums0: 2 buttons and [XY] coordinates ID=0
drm0:  on vgapci0
vgapci0: child drm0 requested pci_enable_busmaster
info: [drm] AGP at 0x40000000 256MB
info: [drm] Initialized i915 1.6.0 20080730

Some Shallow & Superficial Reasons for Picking MongoDB for your [web]app

2010-01-07T20:34:00.006-06:00

So first got turned on to #nosql databases a little over (or under) a year ago with CouchDB but lately I've been quite enamored with MongoDB as of late.

So forgot about deep architectural reasons for using it. Here are some quite practical some practical reasons, when you are a not full-time developer (or database guru) but you find yourself doing development that involves a data store and the thought of using MySQL (so like 2000s) in your app:

Abhorrence for schemas, ORMs, and migrations - this is basically the laziness argument. Basically I want/need to store stuff. And the stuff I want to store might change and I don't want to have to deal with changing the schema (and my) app to adapt to those changes. This was document oriented databases like CouchDB and MySQL rule. If everything is a JSON object it finds a great place for you to store stuff.
Ease of Installation & Compilation -- yep CouchDB has been in the latest Ubuntu repos for a while, but I use Lenny/Hardy server side, so forget about it. Dealing with Erlang (and finding all the dependencies to build SpiderMonkey was a big pain) the ass. Beam, what the hell is beam? Mongo has 32/64 bit Linux binaries that just work and a briefly managed to get it to compile on FreeBSD 7.2. And unlike some of the others out there it doesn't require require a JRE.
Map/Reduce hurts my head - ease of use is one of the key differentiators between Mongo and CouchDB is that is the simplicity of queries. I'm not an expert yet, but having to create Map/Reduce functions to create views to get at your data, it was a slippery concept for me.
Non-HTTP Transport -- unlike CouchDB, Mongo has a binary client/server protocol and doesn't used HTTP.

There are also some really cool features like capped collection that should be useful for the app I'm working on, but these were some of the reasons why I went with Mongo. Back to coding...

Pansy or Victim?

2010-01-05T20:49:00.004-06:00

So unfortunately some who I [used to] follow over on @frednecksec cited an article over on prisonplanet.com which allowed me to check out the cool sponsors such as the one pictured above but don't forget Silverlungs.

To each his own, I inhaling gaseous gold myself. Much better preparation for the "End Times," the "New World" or whatever the "elites" have in store for us.

Installing Redmine on Debian Etch

2009-12-25T10:47:00.008-06:00

Here is a step by step summary of what I did to get to get Redmine up and running on Debian 5.x (Linux etch55 2.6.24-24-openvz #1 SMP Fri Sep 18 19:57:34 UTC 2009 i686 GNU/Linux)

If you don't know what Redmine is, it is like Trac, but better and on Rails. If you don't know what Trac is you probably wouldn't be interested in Redmine, so you can stop reading.

What it took?

1. Review this Rails on Debian guide. I'm sure there are others, but this is was a good starting point to get the nuances of running Rails apps on Debian, which can be a bit of pain if you are relying on packages. I know real rails folks use OSX but I'm not a real rails guy.

2. Install the necessary Debian packages. This is what I had to do on an OpenVZ VE, so you your packages may differ slightly: ruby ruby-dev irb sqlite3 ri libzlib-ruby libsqlite3-ruby libmysql-ruby mysql-serer mysql-client libopenssl-ruby

3. Install Rubygems the normal way. I installed 1.3.5. I created a symlink for gem1.8 just because.

4. Read the Redmine Installation Guide. Most of what you need to know is there, and I'm not going to repeat what it is there because it should just work, especially if you are familiar with rails or the configuration of rails apps.

5. Install rails and rake. I installed rails 2.1.2 based on the minimum for the 0.8x of redmine. I assume more recent versions of rails will work.

6. Download the stable release of Redmine.

7. You should have already configured mysql-server during the install, but make sure you put your password in the database.yml.

8. Create your database.yml, the session key and run the various rake scripts in the installation guide.

9. Fire up with webrick and login with admin/admin.

What went Wrong?

The main issues I had were related to not installing the right Debian packages. For example you definitely need libopenssl-ruby or the startup scripts will fail. I also screwed up the database.yml.

What Next?

Get git working following the instructions here. But first I need to work on my git skills, since I mostly have used subversion.

Sexing Up Your Boring Hardy Desktop to thwart Mac-Envy

2009-12-19T20:36:00.008-06:00

So image is important. I get that. That is why so many security folks like Mac's these days. It certainly isn't because they are more reliable, but I digress.

Karmic has decent themes and wallpapers, but 8.04LTS is bloody awful. And you can't go onsite somewhere with a bunch of non-IT folks that will already be looking over you shoulder with ugly Ubuntu brown. It is bad enough that I have to use an older Dell D630, which of course runs faster than my E6400 with XP.

So the Bisigi Project has some pretty cool themes. I picked Showtime, a nice monochrome, almost but no completely OSX like theme. Next you need a simple monochrome theme for Firefox. If you have to use Firefox, Full Flat kicks ass, especially on Netbooks with limited resolution, but I swear it is faster. Lastly you need a nice monochrome wallpaper like Dark Times (after all it is Advent remember) from gnome-look.org.

And this just in, a monochrome Chrome theme to match.

Linux Netbook Use Case: EVDO/Wifi Firewall to protect your "Big Company" XP Laptop

2009-12-19T17:53:00.009-06:00

So if you've used any large enterprise XP image you know they are awful.

The larger the company, the worse the build. They are slow. They crash all the time. They have a zillion agents running doing God knows what and they probably have the firewall disabled.

But you want to do the "right thing" and actually follow policy and NOT put Linux on the lovely Dell hardware they give like you used to do "back in the day."

And you don't want to run these in a coffee shop or an untrusted network. But I feel reasonably safe about running my Ubuntu S10-2 in relatively hostile environments.

Because most modern NICs (including the Broadcom's in most Atom-based netbooks) have auto-MDX so you can just directly plug in your laptop into the unused Ethernet on your Netbook after doing the following:

1) Configure a static address on the eth0 in (/etc/network/interfaces) making sure it is not an network you actually use (DOH!)

2) Make the appropriate change to sysctl.conf (if you have to ask...)
3) Install dnsmasq for DNS and DHCP (an apt-get away)
4) Add whatever iptables rules you want to rc.local (or run manually because if the ppp0 interface is not up it may not work)

Sometimes I share over the Wifi others I use my EVDO card.

Bottom line: it just works.

Bonus: you get to see whatever the hell all those pesky agents are doing when they phone home to your corporate network over the Internet.

Snow Photos and More

2009-12-19T06:59:00.001-06:00

In lieu of Facebook. Recent birthday party, lego club, and more!

blizzard09

WebSocket Service Fingerprinting with Curl

2009-12-09T19:39:00.006-06:00

Fingerprinting is probably a bit of a stretch, but at least I didn't use the "h" word, but using pywebsocket is probably the easiest way to learn about the protocol.

Startup the server....

franz@mfranz-s10-2:~/Documents/pywebsocket-read-only/src/mod_pywebsocket$ python standalone.py -p 8888 -w ../example/

Then the client...

mfranz@mfranz-s10-2:~/Documents/pywebsocket-read-only/src/example$ python echo_client.py -s 127.0.0.1 -p 8888
Send: Hello
Recv: Hello
Send: 日本
Recv: 日本
Send: Goodbye
Recv: Goodbye

Look at the traffic on the wire with ngrep.

interface: lo (127.0.0.0/255.0.0.0)
####
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
GET /echo HTTP/1.1..                                              
##
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
Upgrade: WebSocket..                                              
##
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
Connection: Upgrade..                                             
##
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
Host: 127.0.0.1:8888..                                            
##
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
Origin: http://localhost/..                                       
##
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
..                                                                
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
HTTP/1.1 101 Web Socket Protocol Handshake..                      
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
Upgrade: WebSocket..                                              
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
Connection: Upgrade..                                             
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
WebSocket-Origin:                                                 
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
http://localhost/                                                 
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
..                                                                
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
WebSocket-Location:                                               
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
ws://127.0.0.1:8888/echo                                          
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
..                                                                
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
..                                                                
##
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
.Hello.                                                           
#
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
.Hello.                                                           
#
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
........                                                          
#
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
........                                                          
#
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
.Goodbye.                                                         
#
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
.Goodbye.                                                         
###

Now with curl, notice the headers that you have to add to get a response. With anything less I got a 404. The origin header can be anything.

mfranz@mfranz-s10-2:~$ curl -v http://127.0.0.1:8888/echo -H "Upgrade: WebSocket" -H "Connection: Upgrade" -H "Origin: http://localhost"


* About to connect() to 127.0.0.1 port 8888 (#0)
*   Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET /echo HTTP/1.1
> User-Agent: curl/7.19.5 (i486-pc-linux-gnu) libcurl/7.19.5 OpenSSL/0.9.8g zlib/1.2.3.3 libidn/1.15
> Host: 127.0.0.1:8888
> Accept: */*
> Upgrade: WebSocket
> Connection: Upgrade
> Origin: http://localhost
>
<>
But if the URI doesn't match you get

mfranz@mfranz-s10-2:~$ curl -v http://127.0.0.1:8888/ -H "Upgrade: WebSocket" -H "Connection: Upgrade" -H "Origin: http://localhost"

* About to connect() to 127.0.0.1 port 8888 (#0)
*   Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.19.5 (i486-pc-linux-gnu) libcurl/7.19.5 OpenSSL/0.9.8g zlib/1.2.3.3 libidn/1.15
> Host: 127.0.0.1:8888
> Accept: */*
> Upgrade: WebSocket
> Connection: Upgrade
> Origin: http://localhost
>
* Empty reply from server
* Connection #0 to host 127.0.0.1 left intact
curl: (52) Empty reply from server
* Closing connection #0

Generating SVG Output (from Graphviz) in your Django App

2009-11-26T21:53:00.005-06:00

So accomplishing new coding tasks can be a challenge with interruptions and I've had a lot of interruptions this week but I finally got there. And I'm thankful!

I have an app that is storing data, meaning Django models for the uninitiated. What is in there doesn't matter, but it is something that is conducive to plotting with graphviz. So the starting point is a string that is in the .dot format. I have some code that makes queries to the database and I end up with a string.

So there is a utility function that creates this string...


def make_svg_str:
 #blah blah blah snip
 dot_string += "}"
 p = subprocess.Popen('/usr/bin/dot -Tsvg', shell=True,\
 stdin=subprocess.PIPE, stdout=subprocess.PIPE)
 (stdout,stderr) = p.communicate(dot_string)
 return stdout

So I almost got this right the first time except that I forgot the stdout in Popen() which caused the output to go to stdout (and not be assigned to the string) so I saw the .xml in the dev web server logs.

The graphviz string (dot_string) is being piped to the dot executable and then the function is returning the XML SVG as a string, and is obviously assigned to the stdout variable in the tuple.

Now the tricky part within my views.py.

My first mistake was using the Django CSV docs instead of the PDF docs because the latter is what we need. I also didn't remember that HttpResponse is a file-like object so we can can just write to it once we have the SVG text.


def svg(request):
f = Foo.objects.all()
response = HttpResponse(mimetype='image/svg+xml')
response['Content-Disposition'] = 'filename=somefilename.svg'
response.write(make_svg_str(f))
return response

So this will display your image within your browser (which is what I wanted) instead of downloading file if you the use the "attachment" in the Content-Disposition key.

The name of the game is taking shortcuts that get the job done. I'm using the admin interface to provide a good-enough UI to enter the data and now I'm using Graphviz to visualize that data without having to spend a lot of time writing UIs or nasty JavaScript.

Where's the Controversy about Shodan?

2009-11-24T19:59:00.007-06:00

So like a lot of folks I spent no more than 15 minutes this morning googling Shodan for anything interesting. I looked for SCADA protocols (there were none that I could easily find) or obvious field automation devices, so I went back to work. At best I found a bunch of VxWorks systems (and whole lot of ESX servers, shiver) and others like @chrisjager have also commented about the large number of embedded devices directly connected to the Internet, which is, indeed, frightening.

But @taosecurity just made some interesting comments, questioning how long the site will be up and hit upon in the ethical issues of a site which so obviously allows easy amplification of vulnerable systems. This was the first I've seen that even considers this angle. I'm not sure if everybody is getting ready for the holidays, trying to get the last bit of work done, or already gone but at least on the 300+ plus folks I follow on Twitter there were absolutely no questions about the site, and whether or not such as site was appropriate, ethical, etc. Just to be clear, I'm not claiming it is or is not, I'm just surprised it hasn't come up yet either way. Now if and when this happens (perhaps everyone else is so jaded and just does not want to go there) I'm sure the arguments will quickly fall into the typical cliched responses around disclosure:

The site is raising awareness so is a good thing. Administrators can actually find and fix their systems.
Anyone who has systems directly connected to the Internet with systems that vulnerable deserves to be compromised.
The site is irresponsible and we should immediately DDoS it

And so on...

I don't actually believe any of those arguments. I'm not sure what to think. And I find that troubling. After nearly a decade in information security, I've become weary of all the arguments on either side of these sorts disclosure issue so I resort to know opinion because my opinion doesn't really matter and folks will release 0-days (or not) or more interesting sites like this (or not) and what will happen will happen regardless of any international standards or documented best practice working groups.

So back to trying to find a way to graphviz to generate SVG images within a Django app. That is at least a problem I can solve.

And what exactly would we be doing differently?

2009-10-18T18:57:00.008-06:00

A blackout caused by hackers is the holy grail, the proof that extra terrestrials exist, the debunking of the Warren Commission, the final evidence that we are truly headed toward conflict with a parallel universe and shape-shifting mercury-blooded agents are among us. After Eligible Receiver after Cyber Spies Penetrated the Grid (and don't forget Aurora) after all the incidents cited in every SCADA security presentation, the hunger for one documented incident is still so strong that remote attendance won't be allowed at an upcoming SCADA Cyber Security Conference. And you can taste in the latest Call for SCADA Security Researchers from Project Grey Goose

I challenge you to try to get an answer to that question. I spent the last few weeks doing just that and ran into one brick wall after another, and I have some pretty decent connections to fall back on. It turns out that private industry, which essentially owns the U.S. power grid, enjoys a protection from public scrutiny that extends even to Freedom of Information Act (FOIA) requests, and they get to decide what falls under that protection and what does not. So who does this secrecy benefit?

Walking through .nessus files with Python xml.etree.ElementTree

2009-10-03T19:34:00.012-06:00

Back when I used to teach Tenable's Nessus course I was always surprised how most folks, if to perform additional analysis or manipulation scan results, used Excel to process NBE/NSR files rather than using XML. So I added some simple examples of how to use Python and Ruby to the course and how easy it is you write a simple parser. In my slides I believe I used expat which requires you to build a list/hash of the data you extract as you encounter the start or end of the element. This works and expat or Sax are the primary parsers I've used over the years, although I did recently discover minidom.

If you click on the capture above (blogger doesn't handle XML or code that well) you'll see that after parsing the .nessus file and starting with the top node (I'm not sure why I had to call getroot())
I navigated through the different nodes within the .nessus file starting with Report, ReportHost, and ending with ReportItem where I extracted the port, and plugin id so that when you run the script you get this for all the

192.168.20.3
- 22/tcp|0
- 1241/tcp|0
- 111/tcp|0
- 1243/tcp|0
- 111/tcp|10223
- 59370/tcp|11111
- 111/tcp|11111
- 33145/udp|11111
- 111/udp|11111
- 1241/tcp|22964
- 22/tcp|22964
- 1241/tcp|10863
- 1241/tcp|35291
- general/tcp|12634
- general/tcp|22869
- 59370/tcp|25221

This code snippet isn't terribly useful but it illustrates the API and how it is very straightforward to parse .nessus files.

Some Benchmarks
Besides being much cleaner (IMHO) the nice thing about ElementTree is that there is a C implementation. This is a 2.0 MB file that consists of 4 scans and the scans include the results from only a handful of targets.

On Python 2.5 / Cygwin on my Ideapad S10-2 (Windows XP SP3)

Pure Python
real 0m4.250s
user 0m3.155s
sys 0m0.357s

C Version
real 0m1.422s
user 0m0.405s
sys 0m0.374s

I wanted to do a comparison with Win32 on Python 2.6 on the same system but I was unable to get timeit.exe working from the Windows 2003 Resource Kit.

NOTE: ElementTree is available in Python 2.5 and later and you should be ashamed if you are using anything older than that.

Mortal Netbook Sins (or, why is it impossible for vendors to get it right?)

2009-09-11T17:06:00.004-06:00

So I'm typing this on my (well, actually my wife's) Lenovo S-10, who I admit that I covet. She inherited it after her Powerbook G4 12" (actually my favorite laptop of all time) suffered at the hands (or paws) that typically does in laptops in our household.

Even though my two main laptops (a T-61 and a Dell E6400) are both 14.1" and smallish, I still miss the two pound form-factor and the small screen/keyboard, so I've been agonizing for a while whether or not and which one should be my 2nd Netbook purchase. It should be easy, since there are literally around 50 different models on the market. But it isn't.

The Ideapad S-10 is by no means perfect (the keyboard is too small, it only has 2 USB ports and I don't use the express card slot) but compared to what is available at Best Buy, it is hard to beat. But as I've been comtemplating getting another Netbook that I want but really don't need, here is the dogma:

By far the most damning flaw, the one that cannot be reconciled ever, is an unusable touchpad and buttons. This basically removes any of Dell's offerings because they cut corners and implemented a single, rocker-style button, and the touchpad itself is awful, jumpy, and could. I actually. The Asus 1005HA is not terrible but not great either.

The second abomination is the battery bulge. As Netbooks have added 6 and 9 cell battries they have added a rear bulge that sticks straight out to the rear of the laptop. The S12 and S10-2 with the larger capacities succumb to this temptation. As well as many others.

The third sin is the appearance of cheapness. All of these systems are cheap but they should appear so. The Gateway 3103 falls prey to this. It also runs Vista and is not Linux friendly, taking it out of consideration.

A last fatal flaw is price. It should be below $350. Period. A $400 netbook makes no sense given their underpowered components, especially since I can get a T400 for right around $750, as my total credit card bill reaches $500 it makes no sense to considor an Atom processor when I can get the real thing for a few hundred dollars more. this rules out the Sony and Toshiba models

Not a showstopper, but definitely a flaw, is the inability to customize and lack of a two year warranty. Basically this means you are left with Lenovo or HP, since to the vast majority of vendors do not allow you to build custom systems with only the size of the battery.

Notice what I left out: sound, keyboard, screen, ports, ease of upgrade. None of these really matter if the essentials are met.

So right now I'm not sure, but the two choices I'm considering is a 3-Cell, Lenovo S-10-2 and an HP Mini XP with the higher resolutions 1366x768 display. Both meet all these criteria, so we'll see if I give in to temptation and place an order this weekend.

Why must I be Unnerved? (#25 from Brand You 50)

2009-08-30T20:06:00.003-06:00

About 18 Months Ago I blogged about a conversation I had with a friend and how we both were stuck in an 18 month job cycle. Several months ago (besides worrying that my 18 month clock was ticking, and for those that follow my personal twitter feed, know the egg timer rang and I'm starting a new job tomorrow) I ordered another copy of the must have Brand You 50 and started reading it again.

This was something I hadn't done for a while, but something I highly recommend.

So tonight I read #27 (TO STEER ME INC THROUGH THE WHITE WATER OF CHANGE IS TO DARE, DAILY) contained some clues to decipher my (and I assume, others) 18 month curse.

From the T.T.D. at the end of the chapter, I ran across the question "does my current project scare me shitless?" and I think Tom is really on to something here. Believe it or not I think it is possible to be confident on your abilities and experience, yet still be unnerved about the leap you are taking. If you aren't a little bit scared about what is before you, how could you possibly grow from the experiences around the corner. I can definitely remember this anxiety about various projects at past employers and unfortunately this often occurs only in the beginning of the job. Near the middle and the end things become repetitive and it becomes increasingly difficult to maintain a steep learning curve.

I'll return to the topic in 18 months to see where I'm at!

NOTE: This topic is not unrelated to the survey a month or so back that One in Two Security Pros are Unhappy in Their Job tp which my wife cynically responded that this is only because the security job market is so strong and you can bounce around...

Some Gems (however heavy) from Peter Rollins

2009-08-10T19:44:00.003-06:00

I'll take slice and dice some memorable statements from ‘Why Do I Do What I Do’, or ‘The Horror of Relationships’ (focusing more on the why do I do what I do what I do, than the relationship part, although my wife and I just celebrated our 16th anniversary this weekend) from the end:

It is today very common to see reason opposed to faith in popular literature (with reason or faith being the better depending on which side the apologist sits). The point is not that they are opposed but rather that reason is saturated with faith. In other words, all real decisions, no matter how reasonable, involve a faith act. Neither the facile liberal nor the crude fundamentalist examples mentioned above allow for the anxiety of making a real decision about love, politics or prayer. While the former only ever minimally commits (not making a full blooded decision), the latter knows what to commit to in advance of doing it (thus not making a real decision, as one can only ever make a decision when one does not know what needs to be done – thus making a choice).

Which is teed up near the beginning

The question ‘Why do I do what I do’ disturbs the smooth running of our lives because it involves a certain amount of anxiety. Yet, far from seeing its manifestation as a minor disturbance in our ongoing life, perhaps we should see it as a site of truth. As a moment in which the foundations of our decisions are momentarily manifested to us in their underlying contingency.

Most of us do not feel the full force of this question either because we never fully commit to a cause (choosing to travel through life without real investment – allowing the TV we watch and papers we read to experience life on our behalf) or because we attempt to ground our theological/philosophical/political projects, or romantic ones, in some absolute (God, Reason, Destiny, Historical Necessity etc.). In the former we never truly make a radical commitment to some cause, while in the latter we never experience the fear and trembling which such a commitment should engender.

Yep, "sites of truth." I like it.

Lessons from Netflix Culture: The 9 Behaviors & Skills

2009-08-10T18:51:00.004-06:00

Culture

View more presentations from reed2001.

While some folks were making a big deal about the fact that Netflix doesn't have a formal vacation/time off program (and I've actually worked at a place like that before) what caught me eye from the presentation was the 9 behaviors and skills.

Notice they don't use the term "values," probably because values seem to imply things that you just have (or are) vs. things you can learn or be taught.

The 9 values they define are judgment, communication, impact, curiosity, innovation, courage, passion, honesty, and selflessness.

I really encourage you to drill down and look at the bullet points. I'm sure you'll find some you do well at and others where you need some work. I know I did.

I picked 9 of the ones I found the most interesting and important. And folks that have worked with me before know that some of these I'm pretty good at, while others continue to be a work in progress:

You exhibit bias-to-action, and avoid analysis paralysis.

You listen well, instead of reacting fast, so you can better understand.

You make tough decisions without excessive agonizing.

You say what you think even if it is controversial.

You contribute effectively outside your specialty.

You smartly separate what must be done well now, and what can be improved later.

You make time to help colleagues.

You think strategically, and can articulate, what you are, and are not, trying to do

You treat people with respect independently of their status or disagreement with you.

Yep, there are some tough ones in there and there is a lot more gold these 128 slides that reinforced by my experience in small and large companies alike.

(NOTE: If you register for a slideshare account you can download a .pptx version and print them out for you cube/office wall like I did.)

CyberSpies: They are back (and we have the logs to show it!)

2009-08-05T10:29:00.007-06:00

From Cyber attacks at U.S. energy companies.

From the Loglogic Department of Statistics

“Ever since cyberspies hacked the U.S. electrical grid earlier this year, businesses have become increasingly aware that a security breach at an energy company that results in a major blackout has the potential to wreak havoc,” said Pat Sueltz, CEO at LogLogic. “We talked to leading information security professionals in the energy sector to find out how they determine the level of risk they carry and architect their security infrastructures to fortify against both internal and external attacks.”

The study surveyed information security professionals from a broad spectrum of energy corporations and government organizations ranging from less than $99 million to more than $1 billion in annual revenue. Of the respondents, two-thirds field more than 75 serious security vulnerabilities each week, with half resolving more than 150 attacks per week.

How can someone use the phrase, "Ever since cyberspies hacked the U.S. electrical grid earlier" without cracking up?

Who doesn't have 75 severe vulnerabilities a week? 75 seems a bit low, actually?

What does "resolving 150 attacks a week" even mean?

Loglogic gets the award for this one.

(CAVEAT: Loglogic is sort of a competitor of my employer, but this has nothing to do with that)

Choose Civility?

2009-08-05T04:54:00.001-06:00

Once in Austin we had a Great Horned Owl in the large Elm in our back yard. As I was watching it, my wife walked up behind me and scared me. We have a running joke about being scared of owls and birds of prey carrying off pets and small children. So last night during supper when I saw the flash of large wings through the side window and I rushed outside and brought our overweight Boston Terrier inside to spare her from this unknown bird of prey.

On our front lawn all of us (including Sam, our 20 mo old) watched as a large vulture tore up an unidentified creature into pieces no larger than a small child's fist.

It turned out to be a possum. I and the two oldest walked over to investigate, or as close as the flies would let us.

Then we went to Rita's to wash the taste of buzzard out of our mouth.

Choose Civility

Best firmware choice for WGR614L?

2009-08-04T04:48:00.004-06:00

So my WGR614L arrived yesterday and I have it running with the built-in firmware but I'd obviously let to get something new on on there that gives me a command line. I used OpenWRT a while back and definitely liked the ipkg's but am wondering what the best/most actively maintained Broadcom distro that runs well on the WGR614L these days?

Squid v. DDoS

2009-07-30T12:24:00.003-06:00

This is hardly more than worth a tweet but longer than 140 chars but Squid Defense against DDoS caught my eye.

I see two lessons here:

Using Open Source tools to respond to an incident is a crude but powerful technique that can get the job done. Try doing this with commercial products.

Environments that were properly engineered with caching and load balancing and could respond to DDoS wouldn't have to worry about the attacks in the first place.

CyberSecurity isn't new and needs domain knowledge

2009-07-14T13:46:00.004-06:00

I agree with Joe 100% on this. So much so that if you replace "Smart Grid" with "Cyber Security" everything is also true.

If all one had to draw from was the flood of conferences, webinars, and advertisements, it would appear that CyberSecurity is a very recent invention that will be achived en-masse in the near future. In reality, elements of CyberSecurity first appeared in the 1998-2000 time-frame. Additionally, decades old best practices will continue to be used in "CyberSecurity" for at least the next 5-10 years. Until about 6-8 months ago, domain knowledge was a given for those participating in the "CyberSecurity." Now, domain knowledge doesn’t seem to be a requirement.

How Chinese CyberSpies Really Compromised the Grid

2009-07-11T20:12:00.003-06:00

Now that I've got your attention. Honestly, I have no idea, but it will be really amusing to see my google analytics stats on this one, I wonder how much malware gets spread through typos in the most popular web sites. Maybe everybody else allows their browser to get them to the right place, but not me. I end up at some weird sites, or at least sites that people in Frederick, Maryland would consider weird.

BTW, the site above is from dgmail.com but it would be an interesting research project to analyze the content of fat-fingered sites. Sure, most are probably ads, but may be some goodies lurking in there.

First Impressions: HP Mini (Best Buy Style) vs. Lenovo S10

2009-06-22T18:40:00.003-06:00

So I picked up whatever the model of the HP Mini that they sell at Best Buy for $329 (the 10.1 model with a 16GB flash drive) for my mother with the goal of installing Ubuntu, since she the one family member that I've successfully converted from Windows.

Keyboard - the larger keyboard of the HP Mini's are well known. You can definitely tell the difference with the larger keys in that it allows more natural touch typing but the feel is spongier. About what you'd expect from a consumer laptop. The arrow keys are smaller size that all the other keys which is very annoying. It is difficult to see the special keys since they are light grey. On my white Lenovo they are blue so it is much easier. Overall the action is much crisper on the Lnovo

Ubuntu Netbook Remix 9.04 installation - Installation took slightly longer, I assume due to the flash drive, but the OS upgrade too so longer (scrollkeeper was pegged at 100%) I killed gdm and went into the console and did the apt upgrade's there which seemed to work better. Still really slow. Hangups at upgrade of synpatic and other packages. I assume this is all do to the flash drive. There is also a known bug in the sound support. No sound through the speakers. Haven't tried a headphone.

Wireless - Even though both use the same Broadcom chipset I had more problem with the Mini. It connected to 1/3 of the networks I tried (a WPA2 for my Verizon Westell DSL modem) but not successfully with an HP 420 WPA access point or a Cisco 851W that was wide open. Perhaps I had L2 ACLs on the latter, not sure.

Ethernet - the RJ-45 port is plugged by default. The Mini appears to use a Marvell driver (as opposed to the Reatek used most other Netbooks). I could not get a lease and was getting PHY errors.

Touchpad - the buttons are on the side which are really annoying but I could probably get used to them. But the touchpad is definintely better than the Dell Mini 10. I prefer the buttons on the buttom that are much crsiper.

Screen - the 576 vertical resolution is definitely a pain since 600 of most Netbooks is too small. The screen seems somewhat brighter than the Lenovo.

Ports & Form Factor - these I knew about so wasn't suprised. No VGA. Ethernet is plugged. Two USBs (like the Lenovo). It is too narrow, IMHO. Sitting side by side the top of the screen is a full 3/4" shorter than the Ideapad. These sacrifices are needless in my opnion because it makes the form factor too small and thin.

Upgradeability - RAM bay is easy on the back, takes up to 2GB but you have to remove the keyboard to upgrade the drive.

Noise: It is definitely seems quieter that the Lenovo. Not sure whether it is the driver or the fan.

Bottom line: I've very happy with my Lenovo even though it hurts my hands and the keyboard is small. Overall Linux runs much better. I don't see the need for a slow (if quiet) flash drive. The form factor of the Mini is just too weird for me. The Ideapad feels like a small version of a real laptop.

Are 6 cell batteries ruining Netbooks (or why you should return your Dell Mini 10)

2009-06-11T19:00:00.004-06:00

So based on these pictures of the new Lenovo S-10-2 it looks like the S-10-2 which otherwise looks like a winner, has the same ugly, bulky, downward-extending battery as the Dell Mini-10, which my parents ended up not liking (and hopefully will be able to return)

Here are my beefs on the Dell Mini-10 (with Ubuntu) most which relate to the touchpad:

Given that it is Ubuntu 8.04 the Xorg (synaptics) touchpad driver is not the same as in 9.04 and it is impossible to make the touchpad usable, despite all the tweaking of the mouse settings. This may be both a software as well as a hardware issue but it is does not bode well for Linux.
The touch pad and mouse buttons are all-in-one. It is nearly impossible to click.

As I would expect from Dell, sloppy engineering shortcuts, both in hardware and software.

And now Lenovo only sells the S10-2 with these bulky 6-cell monsters and has the ugly shiny finish.

Netbook Broadcom (43xx) Cards with Debian Lenny

2009-06-06T16:16:00.006-06:00

So with Ubuntu 9.04 (and possibly earlier) the Broadcom Wireless NIC in your Netbook (mine happens to be a Lenovo Ideapad S10) should just work. But obviously this will not happen with Debian 5.0. Because very little in Debian just works.

So the first thing to know is to ignore an articles such as these that tell you to mess with firmware. Also ignore whatever is on the Debian.

You do NOT have to use the fwcutter tools. Do it this way.

First, install your kernel headers (I use an OpenVZ kernel)

# apt-get install linux-headers-`uname -r`

Download the module source for the Linux STA driver from Broadcom.

Create a directory and uncompress the tarball (mine was hybrid-portsrc-x86_32-v5_10_91_9.tar.gz)


debian-s10:~/bc# pwd
/root/bc
debian-s10:~/bc# ls
built-in.o                 Makefile     src       wl.mod.o
hybrid-portsrc-x86_32-v5_10_91_9.tar.gz  modules.order     wl.ko       wl.o
lib                     Module.symvers  wl.mod.c

The above is what you should see when you after you compile the module using the step below. Execute the command below from wihtin the directory that has the Makefile


# make -C /lib/modules/`uname -r`/build/ M=`pwd`

The resulting module you care about is wl.ko (assuming you have the ieee80211 module installed you will be able to insmod this and see the following in dmesg)

[  922.523743] ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI 18 (level, low) -> IRQ 18
[  922.523997] PCI: Setting latency timer of device 0000:05:00.0 to 64
[  922.622849] ieee80211_crypt: registered algorithm 'TKIP'
[  922.623123] eth1: Broadcom BCM4315 802.11 Wireless Controller 5.10.91.9

and with a lshw

description: Wireless interface
product: BCM4312 802.11b/g
vendor: Broadcom Corporation
physical id: 0
bus info: pci@0000:05:00.0
logical name: eth1
version: 01
serial: 00:21:00:7e:7a:7d
width: 64 bits
clock: 33MHz
capabilities: pm msi pciexpress bus_master cap_list ethernet physical wireless
configuration: broadcast=yes driver=wl0 driverversion=5.10.91.9 ip=192.168.1.24 latency=0 module=wl multicast=yes wireless=IEEE 802.11bg

so I modified /etc/modules so that it looks like

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
# Parameters can be specified after the module name.
loop
ieee80211

So that ieee80211 gets loaded and then added the following line to my rc.local file (before the exit 0, obviously)


insmod /usr/local/lib/modules/`uname -r`/wl.ko

After copying the module there and creating the directory (remember mkdir -p is your friend)

Now NetworkManager should work just fine. And WPA2 worked just fine with my crappy Westell AP.

I tried putting in somewhere in lib/modules/`uname -r` with no luck, but this works for me...

* * *

NOTE: Don't click on the image PCI Express Card image. It contains Chinese Ghostnet Malware that will turn your Mac (and only your Mac) into a Zombie botnet enabling a complete blackout or extortion of the power grid.

Additional keywords: NERC, FERC. SCADA. Project Grey Goose. Cyberwar. ISN. TASE.2

Best Linux Virtualization for Netbooks?

2009-06-06T09:48:00.005-06:00

So I use my Lenovo Ideapad S10 as my main Linux box nearly 40% of the time. I've 1.5GB of RAM and 120GB drive so this a decent machine. My current setup is two Linux partitions, one for Ubuntu 9.04 and the other for Debian 5.0. Ubuntu is my production distro and Debian is for bleeding edge stuff. My main requirement is to run Linux VM's (of other distros than what I run on the host) because if I need to run Windows or Solaris or whatever I can connect to a remote system. For Linux systems I want "server virtualization" meaning I don't have to have a console up. Realistically there is no single solution that will meet my requirements, but here are my thoughts on the alteratives for running on a Linux Atom-based Netbook.

1) OpenVZ - this would be my first choice. Unfortunately there are only kernel for Ubuntu 8.04 LTS and Debian for the these and Ubuntu LTS is too old to work well for a desktop on netbooks. I have yet to get the Broadcom drivers working yet on Debian and the latest stable OpenVZ kernel patches are 2.6.18. I guess the real issue is if I could get the Broadcom drivers working on the stock kernel that would be the way to go.

2) VMware Player - I don't want to put VMWare Server 2.x on my laptop and this seems like the logical choice. I already have this for BSD or Windows.

3) lguest - this is something new that I've just discovered. Can I run a CentOS VM under this. Not sure.

I don't care for VirtualBox and Qemu is too damn slow. Is there anything else I'm missing?

Painless, Distro-Agnostic Cisco Webex on Linux

2009-04-12T17:07:00.008-06:00

For true cross-platform web conferencing, Cisco Webex is the only way to go. GotoMeeting only recently added OSX support, and Linux, forget about it?

My experience on getting it work with the built-in components on my Thinkpad... forget about it!

Ubuntu 8.0.4 worked sporadically and and on 8.10 Firefox crashed. Hard.

I Googled a bit and didn't find any quick workarounds, so I decided to try it the old fashioned way. So here is what I came up with to get it working reliably. I assume this works on other distros as well.

(All of this assumes you create another user for just webex so you don't corrupt your local .mozilla and .adobe files etc.)

Download Components

1. Get the tarball of Adobe Flash (10.0.22.87 tested)
2. Download Firefox 3.x (3.0.8 tested)
3. Download JRE .bin installer (jre6u13-linux-i586.bin used)
4. Create a webex directory and move all of these to it
5. Uncompress them there

Configuration

1. Remove ~/.mozilla and ~/.adobe
2. Run ~/webex-local/firefox/firefox then quit
3. Run the ./flashplayer-installer script
4. Run the java installer binary
5. Create the symlink for the java plugin within $HOME


ln -s ../../webex-local/jre1.6.0_13/plugin/i386/ns7/libjavaplugin_oji.so

Testing Webex

1. Run your local firefox
2. Confirm you can execute java applets by visiting http://java.sun.com/applets/jdk/1.4/demo/applets/Clock/example1.html
3. Click on the test meeting http://support.webex.com/support/support-overview.html

SCADA CyberSpy Reverse Forensics Contest

2009-04-09T04:05:00.006-06:00

So given the hoopla on Chinese/Russian CyberSpy Hacking the Power Grid Story I figured it was time to break Blog-silence.

I had the misfortune of hearing Siobhan Gorman on NPR yesterday on my commute so I was still fuming yesterday about the vermin in the Intelligence Community that leak classified threat data on "background" to reporters to influence policy. This data cannot be repudiated not only because most journalists don't have the technical wherewith all to know better but because the leakers cannot be held accountable. The "good guys" in the IC (those that follow the rules and don't disclose secrets) cannot challenge (or confirm) it. It is a one-sided game that leads to bad policy, scaring the public, and bad legislation. Does anyone not remember Iraq and WMDs?

But I digress.

What was interesting about the Gorman interview was that she mentioned network forensic data that showed how control systems not only had been penetrated and were being remotely monitored and possibly controlled.

So some readers may remember the HoneyNet Projects Reverse Challenge. Basically a contest to analyze malware, if you never heard of it

What I think would be cool is some aspiring folks with the skills and time (I have some of the former but none of the latter) to basically create some forensic data, let's say packet captures that show the power grid being mapped, HMI's and PLCs being monitored, ICCP traffic being captured and retransmitted back to our Chinese and Russian masters so they can "monitor power flows" like Gorman mentioned in her interview. Remember be sure to visit APNIC and pick your IPs to spoof wisely.

The minimum entry can just be some packet captures, but you are guaranteed to at least place if you release actual tools used by our Chinese and Russian overlords to blackmail us at will and cause us to resort to cannibalism.

You get bonus points if you actually show some slight knowledge of Mandarin or Russian.

But here's the rub, don't release it on your blog don't talk about it at the next Con because there will inevitably be lots of presentations on the topic. Silently release your own "evidence of Chinese Russian control over the power grid" into a P2P network, or better yet let your laptop get stolen in an airport (make sure you have the right colored classification stickers on your laptop) and wait for your "data" to make the news.

An Oldie But a Goodie

2009-03-10T05:38:00.003-06:00

Yeah, back during the 01 layoffs at Cisco (or was it afterwards, during one of the never-ending reorgs can't recall) I had a DIVX of Office Space that I would watch while I worked to remain productive and we'd take lots of orbits around the parking lot and watch for this old guy with a beard down to his knees that would get off his shift at 3:30 at the Tyco fab next door and walk to his Corvette. Happy days!

Time to leave the SBUX and get to work...

"Cyber Katrina" or "Digital Pearl Harbor" (which is a more loathsome term?)

2009-03-04T20:03:00.006-06:00

Every time you hear 9/11 or Cyber Katrina you should reach for your wallet.

Does anyone find this sort of hyperbole rhetorically effective?

Chairperson

House Permanent Select Committee on Intelligence

Washington, D.C.

RE: Establishment of North American Urgent Radiological Information Exchange

Madame Chairperson:

While we do not believe that this is a matter that rightfully falls under the province of your Committee, in the interest of cooperation, this letter will address the events leading up to the establishment of the North American Urgent Radiological Information Exchange (NAURIE).

As you know, on the 10th year anniversary of 9/11, all of our nation’s nuclear power plants were targeted in a massive distributed denial of service attack orchestrated by the Conficker III botnet which had grown to a heretofore unheard of 30,000,000+ infected PCs.

While US CERT teams as well as regional DOE cyber security personnel were focused on combating this external threat, each plant’s internal firewall separating the Command and Safety System Networks from the Site Local Area Network was breached from the inside due to the use of pirated hardware with malicious embedded code that passed server control to external users.

Of even more concern is the fact that all of these plants were targets of a carefully planned, longterm social engineering attack which relied on human error and the broad-based appeal of Social Network sites. As DOE employees broke protocol and downloaded phony social software apps, malicious code worked its way into secure networks and lay dormant until activated by the attacking force.

This led to a number of consecutive failures in our safety mechanisms resulting in partial to complete core meltdowns at 70% of our plants. When these plants went offline, the nation’s power requirements couldn’t be met. Grids were overwhelmed and blackouts began occurring in our most heavily populated urban areas. Once criminal gangs realized that overburdened police departments were unable to respond to every 911 call, looting of businesses began in earnest as did home invasions in the wealthier neighborhoods.

One year later, we still do not have a final count on the number of deaths and casualties but most responsible estimates place them in the tens of thousands. If we extrapolate out for the as yet unknown future effects of radiation poisoning on the victims, the count goes into six figures.

While this is clearly a tragedy on every level, I feel I must point out that the NNSA, as late as 2009, in a letter to the Los Alamos National Laboratory, did our part in improving security by determining that the loss of 83 LANL laptops should no longer be considered just a “property management” issue, but a cyber security issue as well.

Also, that our G3 physical security model (Gates, Guards, Guns) was not compromised, and that cyber security compliance has never been a mandatory policy; that instead it was an ongoing negotiation among various other considerations.

Sincerely,

Director, National Nuclear Security Agency

(BTW, this is far less salacious than the scenario we came up with for CyberStorm 2005 in the Energy sector)

So. Am I just a reactionary? Is this sort of FUD a necessary evil to make "progress on cybersecurity" or just another boondoggle.

ASA5505 SSLVPN Port Forwarding

2009-03-01T07:30:00.004-06:00

So as I've been chronicling over on @frednecksec I've been pleasantly surprised with the new ASA5505 I got for my classroom network. Although I'm looking forward to replacing iptables the main reason for the purchase was the WebVpn. In particular the ability to do port forwarding. Yes this is just like SSH local port forwarding.

Here is config snippet for ASA 7.2(4) to allow you to get port forwarding working.

Enable WebVPN


webvpn                                                                   
 enable outside

Actually if you stop here you would be able to do URL redirection and get to web servers behind the ASA, although this doesn't show up anymore now that port forwarding is setup.


 port-forward SSH 2223 192.168.55.100 ssh     

group-policy first internal                                                  
group-policy first attributes
 vpn-tunnel-protocol webvpn
 webvpn                                                                      
  functions port-forward auto-download                                       
  port-forward value SSH

Some gotchas here. "port-forward" and "auto-download" have to be on the same line together. It wasn't immediately obvious to me that I had to do the "port-forward value" line. My general approach for Cisco CLI work is to just brute force it to find the minimal config. But this was the key thing I ran across. Unless you had this line, the session won't show up in the UI (see above) although the applet will download.

The steps below are pretty straightforward once you have the group-policy created (above)


username vpnuser password ... encrypted                         
username vpnuser attributes                                                  
 vpn-group-policy first                                                      
tunnel-group test type webvpn                                                
tunnel-group test general-attributes                                         
 default-group-policy first

This works on Ubuntu 8.10 (Java6) and Firefox 3.x, OSX 10.4 with Safari 3, and Windows XPSP3 both Firefox 3.x and IE (who knows what versions).

Personal or Professional (or, why one Twitter account is Not Enough?)

2009-02-22T13:28:00.007-06:00

So I just hit my 400th tweet on @mdfranz but am scarcely up to 20 @frednecksec and I've learned a few things about how I like to use this addictive service in the past few months. And yeah (if you went there) my updates are protected, but more or that later.

When I was first started following people, I was annoyed by technical people (whose blogs I read or knew personally) that only tweeted about personal stuff, so I didn't follow them. I could give a shit about what what sort of decadent food they were cooking, what they were doing with their wife, or their kids accomplishments. But I was interested in 140 characters of wisdom on some technical/technology topic. If there was at least a 50:50 ratio of personal to professional context I kept following, otherwise I dropped them.

Personal Branding
As Tom Peters would say, "you are your customers." Your personal brand is reflected in the those that you do business with and those that do business with you. The same applies to you twitter followers and folks you tweet with. If people that follow you tweet about stupid shit (to put it crudely, but probably characterizes some large % of tweets) that reflects poorly on you, since one of the first things I do when I follow someone (or someone follows me) is I check out the people they follow and their followers. It is the same principle as only connecting with "people you trust" on LinkedIn. On my public account I'm more open to follow somebody I don't know well enough or let anybody follow me, including spambots. But on my private account I approve all followers.

Privacy
Frankly, a lot of stuff you tweet on has no business on public Internet (and all the various bots that follow you) where you shop, what you eat, the activities you do with your family, where you are geographically is none of the damn business of people that you don't really know, let alone twitter's public timeline. This is why I protect my updates on @mdfranz but don't on @frednecksec. Several weeks ago I registered for a demo version of some webapp and a product manager/sales person started following me. Creepy. I don't want sales people following me. And during the inauguration I wondered about how well Sprint's EVDO network would hold up and I had somebody in customer server ping me. She was nice/professional enough but I don't want that sort of interaction. I also don't want people I don't know to where I frequent.

Different Media for Different Messages
I've found that there are also two kinds of tweets: those personal, biased observations, and more objective factual statements that answer the original twitter question, "what are you doing?" More specifically what I'm am I reading that might be of interest to my readers. More reflective, opinionated tweets go on my personal account while the others (especially that are narrowly security related) go on my public account. This is the reason I've moved most of my high volume twitter lists (that mostly shared links and article) over to my public account. Public content stays public, private content stays private and I can also see on my public account when something I've read about, seen has already been tweeted on. I think RT is lame since the whole point is to post original content or content that reflects a certain perspective or range of interests.

So what Twitter client allows you to use multiple accounts at once, twhirl. Or use multiple browsers which is generally a good idea.

Installing OpenSolaris on Lenny dom0 (sort of)

2009-02-19T21:04:00.003-06:00

Here is my domain config file (open1.py)


mfranz-61lenny:/alt/xen/domains/opensol# cat open1.py 
name = "solaris"
memory = "1024"
disk = [ 'file:/alt/isos/osol-0811.iso,6:cdrom,r', 'file:/alt/xen/domains/opensol/disk.img,0,w' ]
vif = [ '' ]
bootloader = '/usr/lib/xen-3.2-1/bin/pygrub'
kernel = '/platform/i86xpv/kernel/unix'
ramdisk = '/boot/x86.microroot'
extra = '/platform/i86xpv/kernel/unix - nowin -B install_media=cdrom'

And here is proof that I did it


mfranz-61lenny:/alt/xen/domains/opensol# xm create -c open1.py 
Using config file "./open1.py".
Started domain solaris
                      v3.2-1 chgset 'unavailable'
SunOS Release 5.11 Version snv_101b 32-bit
Copyright 1983-2008 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
Hostname: opensolaris
Remounting root read/write
Probing for device nodes ...
Preparing live image for use
Done mounting Live image
USB keyboard
 1. Albanian                      22. Latvian                       
 2. Belarusian                    23. Macedonian                    
 3. Belgian                       24. Malta_UK                      
 4. Bulgarian                     25. Malta_US                      
 5. Croatian                      26. Norwegian                     
 6. Czech                         27. Polish                        
 7. Danish                        28. Portuguese                    
 8. Dutch                         29. Russian                       
 9. Finnish                       30. Serbia-And-Montenegro         
10. French                        31. Slovenian                     
11. French-Canadian               32. Slovakian                     
12. Hungarian                     33. Spanish                       
13. German                        34. Swedish                       
14. Greek                         35. Swiss-French                  
15. Icelandic                     36. Swiss-German                  
16. Italian                       37. Traditional-Chinese           
17. Japanese-type6                38. TurkishQ                      
18. Japanese                      39. TurkishF                      
19. Korean                        40. UK-English                    
20. Latin-American                41. US-English                    
21. Lithuanian                    
To select the keyboard layout, enter a number [default 41]:

[snip]

User selected: English
Configuring devices.
Mounting cdroms
Reading ZFS config: done.

opensolaris console login: root

Now what do i do?

Grampa, where did you live during the "Long Depression" that started in 2008?

2009-02-15T19:37:00.013-06:00

Richard Florida's article How the Crash Will Reshape America captures a lot of what has been on my mind (and occasionally blogged about) since we left Skokie last June.

From the packing up and leaving what my son called "best house ever" (the shaky footage is his) without a sold sign out front to some motel thoughts on the trip to premonitions of the impending collapse back to July (more specifically the non-sustainability of suburban sprawl) to the "discussions" my wife share on how long we should rent and when and where we should buy a common thread is that where you live matters.

It matters a lot. A hell of a lot. Your future may depend on it.

In October, less than a month after the financial markets began to melt down, Moody’s Investor Services published an assessment of recent economic activity within 381 U.S. metropolitan areas. Three hundred and two were already in deep recession, and 64 more were at risk. Only 15 areas were still expanding. Notable among them were the oil- and natural-resource-rich regions of Texas and Oklahoma, buoyed by energy prices that have since fallen; and the Greater Washington, D.C., region, where government bailouts, the nationalization of financial companies, and fiscal expansion are creating work for lawyers, lobbyists, political scientists, and government contractors.

Back in September, in the early days of the of the financial crisis, I thought about it a lot as I would look out into the Catoctins from the little park in our subdivision while my kids played (oblivous to what was on the radio) and I started to feel the first hint of Fall, that reminded me of 1987, my first Fall back in the states after living in Malaysia for 2 years.

It was scary. With dozens of showings since our house went on the market on April 15th, 2008 yet not a single offer the constant talk of the forelosure crisis on the NPR, to say that it was stressful, it was an understatement.

Why did we buy in Chicago when we did? Why did we buy in one of the most overpriced suburbs on the Chicago North side? Well, because we couldn't afford Evanston or Winnetka and because of Skokie's diversity. We did not want our adopted Chinese daughter to be "the Chinese girl" in her pre-school class. Yes the schools were full of industrious recent immigrants. Immigrants that didn't care for the red brick Cape Code with the master bedroom where I sometimes bumped my head on the ceiling or the hardwood floors. All they wanted was space.

Multiple generations would fill the ugliest split level boxes you can imagine adjacent Crawford or Dempster. We heard from our realtor only the split levels were selling and how the demographics of the visitors were "different" (she was trying to abide by some regulations) between our house and the those on nearby streets. Yes, so much was different between Austin and Skokie. Our first house two blocks from North Lamar (and where you could hear the music from Threadgills) sold on the first day. Those were different times and different places.

Mostly white folks in their late 20s and early 30s. The hint of pot when you walked down the streets, some which had sidewalks some that did not. Certainly a larger percentage of Gay/Lesbian couples than in our previous neighborhood in San Antonio. Small three bedroom houses (if you were lucky) built in the late 40s and early 50s during the postwar boom. Aging water mains under streets and periodic electric outages. DSL was just starting to roll out.

I remember seeing the street literally explode in front of our house on Brentwood: the last affordable neighborhood south of Anderson lane. Stay at home moms with graduate degrees. At the Elementary School meetings I felt out of place because I was sans tattoo. If I recall, our first night in Austin was on Halloween of 1999. The dotcom boom was in still in full swing and drcoop.com still held prime real estate overlooking MOPAC.

We sat in the swings in Pease park that Thanksgiving, childless, awaiting what would be our final referral from Russia that would arrive in a matter of weeks and we would discuss it at the Little Deli on my lunchbreak, short 5 minute drive from the Southwestern Bell office on Huntland, adjacent to I-35.

And all that we now know was still ahead of us: the letdown of Y2K, the NASDAQ crash in the Spring (I remember a day-trading colleague who had just joined from Dell) losing a lot that spring. I joined Cisco right after the last stock split in May 20o0 and I remember someone from SBC saying something about how I "was set" and could "retire."

Yes, Austin is/was one of Florida's poster children for these new creative cities.

Thirty years ago, educational attainment was spread relatively uniformly throughout the country, but that’s no longer the case. Cities like Seattle, San Francisco, Austin, Raleigh, and Boston now have two or three times the concentration of college graduates of Akron or Buffalo. Among people with postgraduate degrees, the disparities are wider still. The geographic sorting of people by ability and educational attainment, on this scale, is unprecedented.

The University of Chicago economist and Nobel laureate Robert Lucas declared that the spillovers in knowledge that result from talent-clustering are the main cause of economic growth. Well-educated professionals and creative workers who live together in dense ecosystems, interacting directly, generate ideas and turn them into products and services faster than talented people in other places can. There is no evidence that globalization or the Internet has changed that. Indeed, as globalization has increased the financial return on innovation by widening the consumer market, the pull of innovative places, already dense with highly talented workers, has only grown stronger, creating a snowball effect. Talent-rich ecosystems are not easy to replicate, and to realize their full economic value, talented and ambitious people increasingly need to live within them

Returning home right before noon on that fateful blue September morning in 2001 after we watched both towers Fall live in the conference room. Watching the Reserve and National Guard Intelligence units gradually get activated and a getting a call from my XO down in San Antonio saying my name was on the list to deploy to Fort Belvoir. Packing my duffel bag, sitting around the table of our retro kitchen table and trying to explain to a toddler that Dada was going to go away for a while. Of course the orders were rescinded but my son and I were baptised in the Episcopal church anyway that Fall. There was more to it than that of course, but that was Austin.

Years later I would grow restless and leave Cisco and work (virtually) from coffee shops on Lamar, Burnet, and Anderson and from my hot home office that used to be a kitchen and where supposedly a previous resident had died. From the heat and the solitude I would slowly go crazy and would start looking to find a new job where I could work in an office again. I would strike out in Seattle (yeah I wrote that after bombing my AMZN interview, badly) but cool clarity would come soon.

Giving up on big West coast software companies, I dug into the SCADA Plugins and a recruiter from Hewitt called, offering crazy money, a chance to run/develop Open Source security boxes in a large company and not do vuln work which I'd grown, a sweet relocation package, and a way to escape the Texas heat.

* * *

We will see if it is true but one of the more important (and troubling) charactization of our current malaise, elswhere Florida says our depression has more in common with the "Long Depression" of the late 19th century than the Great Depression:

Economic crises tend to reinforce and accelerate the underlying, long-term trends within an economy. Our economy is in the midst of a fundamental long-term transformation—similar to that of the late 19th century, when people streamed off farms and into new and rising industrial cities. In this case, the economy is shifting away from manufacturing and toward idea-driven creative industries—and that, too, favors America’s talent-rich, fast-metabolizing places.

And on the importance of geography:

To a surprising degree, the causes of this crash are geographic in nature, and they point out a whole system of economic organization and growth that has reached its limit. Positioning the economy to grow strongly in the coming decades will require not just fiscal stimulus or industrial reform; it will require a new kind of geography as well, a new spatial fix for the next chapter of American economic history.

Suburbanization was the spatial fix for the industrial age—the geographic expression of mass production and the early credit economy. Henry Ford’s automobiles had been rolling off assembly lines since 1913, but “Fordism,” the combination of mass production and mass consumption to create national prosperity, didn’t emerge as a full-blown economic and social model until the 1930s and the advent of Roosevelt’s New Deal programs.

And, among other things, the foolishness of the American Dream of owning your own house. Thank God we're still renting.

On one level, the crisis has demonstrated what everyone has known for a long time: Americans have been living beyond their means, using illusory housing wealth and huge slugs of foreign capital to consume far more than we’ve produced. The crash surely signals the end to that; the adjustment, while painful, is necessary.

But another crucial aspect of the crisis has been largely overlooked, and it might ultimately prove more important. Because America’s tendency to overconsume and under-save has been intimately intertwined with our postwar spatial fix—that is, with housing and suburbanization—the shape of the economy has been badly distorted, from where people live, to where investment flows, to what’s produced. Unless we make fundamental policy changes to eliminate these distortions, the economy is likely to face worsening handicaps in the years ahead.
Suburbanization—and the sprawling growth it propelled—made sense for a time. The cities of the early and mid-20th century were dirty, sooty, smelly, and crowded, and commuting from the first, close-in suburbs was fast and easy. And as manufacturing became more technologically stable and product lines matured during the postwar boom, suburban growth dovetailed nicely with the pattern of industrial growth. Businesses began opening new plants in green-field locations that featured cheaper land and labor; management saw no reason to continue making now-standardized products in the expensive urban locations where they’d first been developed and sold. Work was outsourced to then-new suburbs and the emerging areas of the Sun Belt, whose connections to bigger cities by the highway system afforded rapid, low-cost distribution. This process brought the Sun Belt economies (which had lagged since the Civil War) into modern times, and sustained a long boom for the United States as a whole.

But that was then; the economy is different now. It no longer revolves around simply making and moving things. Instead, it depends on generating and transporting ideas. The places that thrive today are those with the highest velocity of ideas, the highest density of talented and creative people, the highest rate of metabolism. Velocity and density are not words that many people use when describing the suburbs. The economy is driven by key urban areas; a different geography is required

Of course a lot of this (as well as the stimulus package) hinges on whether or not the era of manufacturing jobs is truly gone.

Twitter / FredneckSec Updates

2009-02-15T12:27:00.003-06:00

For better or worse, I'm now up to 2 twitter accounts, having created @frednecksec with the goal of trying (once again) to form a Security networking group in the Frederick area along the lines of CharmSec or NoVA Sec except for us country folks that live too far out to make it into (or stick around after work) to the DC/Baltimore area.

Yeah, so this is definitely cutting into my blogging. Apart from a regional focus I hope to tweet on stuff you won't see elsewhere on any of the twitter, even if it tends to border on the obscure.

FredneckSec was something a couple of us (unsuccessfully) tried to do last Summer but am hoping with power of twitter and some new folks I've met here in the New Market area to tried to get this rolling again real soon now.

Twitter Starts to Grow Up

2009-01-29T06:43:00.003-06:00

Looks like they are actually starting to address twamming or or whatever (tweet-spam) is called. Cause I went to block JENNY and got this image.

Good for them. About time.

How long until they get non-Base64 authentication?

Is jennydddggeee too hot for you? (or, Automated Twitter Spam Blocking?)

2009-01-27T20:45:00.009-06:00

If you are reading this blog, you don't know anyone like this, don't want to know anyone that looks like that -- and certainly don't want either of them following your every move.

So it should be pretty easy to write less than 25 lines of Python using Twyt that automatically removes any followers that have a single post.

But there have to be tools that already do this. Or any Twitter clients that will automatically block spam followers.

Another Post-Twitter Poor Excuse for a Blog Entry

2009-01-25T18:44:00.005-06:00

After getting hooked on Twitter, I really don't blog anymore (but in the tradition of Luv Them Firewalls, another picture of my my daughter on our now defunct Macbook) here was a picture tonight. Was barely able to get her unglued from Gcompris. It is funny how kids get attached to certain items of clothing. This year it was this Santa hat. Back in Austin it was this pink pair of snow boots.

Khe Sanh?

2009-01-20T20:41:00.002-06:00

Yep, that stood out for me as well, but this is as good as an exchange as I heard/read today.

Visit msnbc.com for Breaking News, World News, and News about the Economy

Childish Things and Hand Me Downs

2009-01-20T20:04:00.005-06:00

I really wanted to blog on our new President's comments about "putting away childish things" behind us, but I'm too tired. I woke up 4-ish again, and made the drive in at 6:15 to avoid the traffic that never came. So instead I'll post a picture of my youngest child.

Appropriate, since my wife has some new project where she is rifling through old physical photos in plastic tubs.

Slightly more than nine years ago, my oldest son wore this same snowsuit in Samara and Moscow (yes, both are in Russia) but he never got played in the snow.

We returned home with him in the 2nd week of February in 2000 to our 1st "Green House" in Austin and Spring had sprung. This snowsuit was not worn again. At three months older and barely walking, my Chinese daughter, in March of 2005, was too small to wear this snowsuit on a ski trip to Utah.

But we kept it. And I remember packing it up in June, when I was single-handedly packed our 4 bedroom house in Skokie.

Yesterday, we had the first decent snow here in New Market (but not nearly as much as when his was born) but it was enough.

My wife found the snow suit and Sam wore it.

Inside the Gmail Login Sequence (or, has anyone documented all the parameters and JSON response codes)

2009-01-14T20:09:00.003-06:00

I generally don't like Wiley books, but near the end of a chapter on how Gmail works actually isn't that bad.

I'm sure there has to be more stuff like


/gmail?
ik=344af70c5d
&view=cv
&search=inbox
&th=101865c04ac2427f
&lvp=-1
&cvp=0
&zx=9m4966e44e98uu

As you can see, this the message ID of the message I clicked on.
But the others are mysterious at the moment. At this point in the
proceedings, alarms went off in my head.Why, I was thinking, is
the variable for message ID th—when that probably stands for thread.
So, I sent a few mails back and forth to create a thread, and loaded
the Inbox and the message back up

elsewhere dissecting the URL parameters, but I haven't found it apart from looking at the libgmail source, the constants file in particular. Has nobody documented this stuff or is google burying any documentation on reverse engineering Gmail?

It is sort of curious that the author is using tcpflow. Fine tool, but using an interceptor proxy like paros or something like firebug is a hell of a lot more efficient than sniffing.

Another word for stakeholders

2009-01-12T19:17:00.006-06:00

This is probably more worthy of a tweet (funny how tweeting has cut down on my blogging) but Alex Payne writes about the challenges of securing twitter (a relevant topic given my Twitter usage lately)

The thing about security is that it requires stakeholders. I have a security background, but Twitter’s security isn’t my job. In fact, my job is pretty much the opposite: I open up as much of Twitter’s functionality as I can without (hopefully) making the system insecure. So while I’ve usually been a “first responder” to security incidents because of my background, it requires a major mental context switch from the work I normally do.

Several months after I joined Twitter in early 2007, I suggested to the team that we do a full internal security audit. Stop all work, context switch to Bad Guy Mode, find issues, fix them. I wish I could say that we’ve done that audit in its entirety, but the demands of a growing product supported by a tiny team overshadowed its priority. Now we‘re in an unwelcome position that many technical organizations get into: so far into a big code-base that’s never seen any substantial periodic audits that the only way to really find all the issues is to bring in some outside help – something I sincerely hope we end up doing, but is not my call.

This post is depressing on a number of levels, mainly because it reminds me of the attitudes (and my own personal frustrations) from back in the early years of doing product security at Cisco.

I hear thing have actually have improved (however slowly) there, but obviously in the supercool world of 2.0 and social networking, they are still pre-2001.

Stakeholders, yeah I'll tell you another word for stakeholders: people that give a shit.

I remember a certain Director of Marketing in the Security & VPN BU. These guys have long since cashed out their options (and the product is killed off), so I don't feel any reservations about blogging about it. Yeah, he was a stakeholder all right, he told us (a small, understaffed, security testing team with no power or authority) that his remote access VPN product was a communication product so security didn't really apply. (Leaving out the far more interesting & cynical quote from a GSR Director of Marketing)

So I understand the frustration, but the idea (that even even if you are a developer, product manager, system administrator) that suddenly you put on your security security hat, stop the presses, fix everything is a quaint notion alongside that 20th century concept that your application, device, or TCP/IP enabled Kleenex box (a big shout out to the Hewitt appsec crew!) is behind a firewall (or not on the Internet) so therefore security isn't a big deal.

You are the stakeholder. And to paraphrase The Wire, "Is you up or is you not?"

Security is not about losing the big battles. It is about winning the small ones. The one's you can win. You do what you can and don't whine about it. If it is not your call, then it is not your problem. Worry about what is your call. That is all you can do.

Been there and done that, you are wasting a lot of time and energy. Trust me.
If you don't believe me, read Unfettered. Bless his heart, Joe is still preaching (nobody gets it, nothing is being done, etc.) the same way he did the first conference of his I attended on SCADA security back in 2003.

Either they get it or they don't and maybe if they don't appear to get it, it is because it really isn't that important in the grand scheme of things. Or maybe you aren't explaining it well enough. If it is really important it work itself out in the long run.

Verisign's Big Takeaway (And Pigs Need Wings, Too)

2009-01-07T05:10:00.004-06:00

How unsuprising is Tim Callan's "big takeway" for anyone that has had experience on the disclosure front either inside a vendor or as a finder:

The big takeaway for me from this incident is that we need an environment where researchers and security vendors can trust each other. Alexander has explained why his team did not feel they could place that trust in VeriSign. I have explained why I feel they could have. We at VeriSign would like to see an environment where researchers need not mistrust security vendors and vice versa. We're committed to doing our part to bring back that environment, and we encourage security researchers in the future to reach out directly to us. We promise to treat you fairly and respectfully.

Yup, we'll see when that happens.

Moodle External-Internal Authentication/Enrollment Synchronication

2009-01-06T11:48:00.002-06:00

This is something I'd wish I'd blogged about because I wasted time remembering how to do this.

This assumes an Ubuntu 8.10 install

First you must sync up the users:

php -c /etc/php5/apache2/php.ini /var/www/moodle/auth/db/auth_db_sync_users.php

Then your must sync up the enrollments

php -c /etc/php5/apache2/php.ini /var/www/moodle/enrol/database/enrol_database_sync.php

Will flesh this out later...

Hello CouchDB (or, does Erlang make it Cool?)

2009-01-03T18:45:00.006-06:00

Today I ran across CouchDB in Ten reasons why CouchDB is better than MySQL (provides a high level overview but not terribly interesting) and a more interesting discussion among a bunch of database guys (which I'm obviously not,) about what sort of problems this (and similar approaches) are most suited for. I must say after having played around with ORM (ActiveRecord, Django, Hibernate) over the years and more recently had my nose is Moodle databases I'm sympathetic to the idea that maybe not everything should be stored in tables, rows, and fields and having to design (or discern) the relations. There is just something sort of contorted about the process of viewing the world (or the data we are trying to capture in the world) this way. I have also definitely felt the pain of having to adjust your schema (as you realize new requirements) and perform "migrations" so there is definitely something intriguing about CouchDB.

Some of the aspects I found interesting

In an SQL database, as needs evolve the schema and storage of the existing data must be updated. This often causes problems as new needs arise that simply weren’t anticipated in the initial database designs, and makes distributed “upgrades” a problem for every host that needs to go through a schema update.

and

CouchDB is a peer based distributed database system. Any number of CouchDB hosts (servers and offline-clients) can have independent “replica copies” of the same database, where applications have full database interactivity (query, add, edit, delete). When back online or on a schedule, database changes are replicated bi-directionally.

And there is also OReilly book in progress that provides a much more readable introduction and Standalone Applications with CouchDB is also definitely worth reading.

(Oh and on Ubuntu 8.10 it is in the repo so it is an "apt"-get away)

start-stop-daemon on CentOS/RHEL5 (and probably other RedHat derivatives)

2009-01-03T11:32:00.003-06:00

Some packages depend on the start-stop-daemon command which is not available in CentOS (or the dag repositories, AFAIK) but the solution is to pull down the source for dpkg and compile it. Basically do a ./configure at the top level of the package and then do a make, make install the utils directory


[root@moodle_dev utils]# make
gcc -std=gnu99  -g -O2  -Wl,-O1 -o start-stop-daemon  start-stop-daemon.o ../get
opt/libopt.a
[root@moodle_dev utils]# ls
enoent    enoent.o  Makefile.am  start-stop-daemon    start-stop-daemon.o
enoent.c  Makefile  Makefile.in  start-stop-daemon.c
[root@moodle_dev utils]# make install
make[1]: Entering directory `/root/dpkg-1.13.26/utils'
test -z "/usr/local/lib/dpkg" || mkdir -p -- "/usr/local/lib/dpkg"
 /usr/bin/install -c 'enoent' '/usr/local/lib/dpkg/enoent'
test -z "/usr/local/sbin" || mkdir -p -- "/usr/local/sbin"
 /usr/bin/install -c 'start-stop-daemon' '/usr/local/sbin/start-stop-daemon'
make[1]: Nothing to be done for `install-data-am'.
make[1]: Leaving directory `/root/dpkg-1.13.26/utils'

Unmasking the Mysteries of the Moodle User/Course Database: Part I

2008-12-31T07:38:00.014-06:00

CAVEAT: From this blog post you'll [correctly] conclude I have way too much time on my /hands, but heh, when you work on your day off you can be as inefficient as you like!

(Oh yeah and all of this, running moodle, mysql, mysql GUI tools, gimp, etc. was all done on my Netbook. These are decent little machines. I'm glad I bought a hard drive instead of a flash because you can use them for real apps. All in a 2 pound package. I would recommend an external mouse/trackball if you want to save you thumbs). A bit hotter than


mfranz@mfranz-s10:~$ uptime
 09:23:44 up 15:39,  4 users,  load average: 0.53, 0.63, 0.59

mfranz@mfranz-s10:~$ free
             total       used       free     shared    buffers     cached
Mem:       1543920    1505324      38596          0     106532     571888
-/+ buffers/cache:     826904     717016
Swap:      1983988        668    1983320

A bit hotter (CPU-wise) than I'd like but Opera was the only thing that bogged down a bit.

The Problem: How do you programmatically find out which students are enrolled in a given moodle course? Since the new authorization/enrollment model implemented in Moodle 1.7 (IIRC) this becomes a little more difficult because the data is spread across a number of tables in the moodle database

Basically you want to find out something like this.

I'm the only student in CF102.

So we start with mdl_user (the table we retreived the metadata on in a previous blog)

Remember, my id is 3

Now to look at the courses (mdl_course)

Remember that CF102 has an id of 3 as well.

Here is where it starts to get interesting. The role_assignment table shows that my user has a roleid of 5 and a contextid of 11. Both of these are necessary to understand what a given use can or cannot do/view in terms of course content.

The role_capabilities table defines what roleid 5 is.

The roleid of 5 corresponds to a student and and the capability is self-explanatory.

Now back to the contextid (from the role_capabilities table), which is the indirect link to the course through the mdl_context table. For once I actually highlighted the correct row. In this case we are interested in a contextid of 11.

I cut the field names off, but the third field is instanceid (which is 3) and points us back to the courseid which corresponds to CF102.

Simple, eh?

In the next blog post on this topic I'll write some Python/SQLAlchemy code to retreive a list of users that are enrolled in a given course or which courses a student is enrolled in.

Why use Python to access your Moodle User Database?

2008-12-30T19:37:00.005-06:00

Well, besides that PHP is an absolute shit for brains language and basic stuff like yaml, displaying syntax errors in imported modules and other sane things you would expect after using Python or Ruby just ain't there.

And oh yeah, and it is is butt ugly ($, ->, ::, ?> etc.)

Not only that because because I was able to whip this up cool script with SQLAlchemy (no I'm not using the ORM, just want to avoid MysqlDB)


mfranz@mfranz-s10:~/crap$ cat alctest.py 
#!/usr/bin/env python
from sqlalchemy import *
from pprint import pprint
e = create_engine("mysql://moodle:blackboard@127.0.0.1/moodle")
m = MetaData(e)
user_table = Table('mdl_user',m,autoload=True,autoload_with=e)
pprint(user_table.columns.keys())
mfranz@mfranz-s10:~/crap$ ./alctest.py 
[u'id',
 u'auth',
 u'confirmed',
 u'policyagreed',
 u'deleted',
 u'mnethostid',
 u'username',
 u'password',
 u'idnumber',
 u'firstname',
 u'lastname',
 u'email',
 u'emailstop',
 u'icq',
 u'skype',
 u'yahoo',
 u'aim',
 u'msn',
 u'phone1',
 u'phone2',
 u'institution',
 u'department',
 u'address',
 u'city',
 u'country',
 u'lang',
 u'theme',
 u'timezone',
 u'firstaccess',
 u'lastaccess',
 u'lastlogin',
 u'currentlogin',
 u'lastip',
 u'secret',
 u'picture',
 u'url',
 u'description',
 u'mailformat',
 u'maildigest',
 u'maildisplay',
 u'htmleditor',
 u'ajax',
 u'autosubscribe',
 u'trackforums',
 u'timemodified',
 u'trustbitmask',
 u'imagealt',
 u'screenreader']

Now that I've got that off my chest.

So what I was trying to do, since Moodle is PHP (and I'm stuck with Moodle) and we are a PHP shop and I thought I would do the right thing and try to use PHP even though I hate it, know it is evil, etc.

The app is in PHP and there are obviously some higher-level APIs/ for accessing Moodle tables, so it makes sense I should write my scripts in PHP?

And there were.So I started using DML (although I was using Pre-2.0 has awful documentation on the wiki, so I basically had to look at the source, which at least has decent internal documentation) to provide external (meaning not through the Moodle web UI) to the Moodle user database.

But that took way too long. Of course it has been years since I've touched any PHP, so I'll admit that was part of the problem. Mainly, forgetting semi-colons. What kind of insane language requires semi-colons as statement separators?

I was contemplating some a weird hack (which I know works just fine, because I've done it before) of sending YAML over SSH (in lieu of XMLRPC, which is a pain in the ass to secure) but php-syck is completely broken with CentOS and I wasn't able to build the PHP module manually, which I shouldn't have to, anyway.

So the long and short of it. I completed in Python (and my Python is rusty) in an hour what took me 3-4 in PHP so Python it is. Honestly, much of the time could have been saved If PHP had an interactive interpreter like Ruby or Python so could quickly test out the new APIs I was learning, inspect objects, etc.

Verisign: Hardly (or, do we have a new disclosure model here?)

2008-12-30T18:52:00.008-06:00

Now I only caught that last 5-10 minutes of the Q&A from the big talk this morning and what I heard (especially about the differences among browser implementations) was pretty interesting. Wish I would have heard the whole thing.

The whining form vendors (or so it is said in the blogs) about "wish they had been told earlier" has been amusing. Waaah.

And I like this new disclosure model (which turns the existing model upside down, vendors have to sign NDAs instead of the the researchers, brilliant!) end the fact that there is a real exploitation (however limited) prior to a fix which brings the end-user community into the disclosure dance.

However, I can't help but think this was sort of a letdown (and I don't think it is just because crypto puts me to sleep) and I liked this summary from This morning's MD5 attack - resolved

Q: Is Internet security broken?
A: Hardly. The presenters of this morning's paper stressed that it took them a long time and a great deal of computational power to succeed in their collision attack. VeriSign has already eliminated the attack as a possibility.

It bothered me that this was positioned as "critical internet infrastructure" attack/vulnerability/compromise to me which pretty much means routing or nameservice or some other collosal failure in the transport layer or below. Which this was not. Web security completely broken I could buy but Internet security, let alone Critical Internet Infrastructure security.

Hardly is a pretty good summary.

Some Non-Speculation on the CCC Breaking the CII Talk Tomorrow

2008-12-29T17:55:00.009-06:00

I'll fess up right away. I have no interest in playing hermenuetical games with redacted texts or trying to divine the flaws that will be released tomorrow.

And the first time I read the talk writeup I thought, "Oh, God, here we go again... more preconference disclosure bullshit."

And of course they were allready at it over on Dailydave. BGP. Crypto. Everybody loves BGP and Crypto. Some new DoS?

Get ready for the FUD machines to start. Time to get ill. Get the bucket ready. But after reading HD's blog (which was based on knowledge of the vulnerability) a second time (once wasn't enough) I'm thinking perhaps this one is different:

Their research combined a known weakness in one area with a massive resource investment in another to show that a third party was vulnerable to a practical attack that affects the security of all Internet users. Security researchers often release code and technical documentation to demonstrate a flaw, but in this case, they went a step further and used the attack in the real world to obtain proof that it works.

Not in terms of the vulnerability (or vulnerabilities) to be disclosed (although that very well be) but as way of disclosing critical vulnerabilities that does neither trivializes nor desensitizes flaws that need to be addressed by vendors and the end-user community. The current model isn't working so well.

As you can already see, if folks within the hacker/researcher community (who should know better) conflate all the scary Internet infrastructure vulnerabilities of course folks technology journalists will.

In the broader IT press, Kaminsky's DNS will be treated the same as Watson's TCP as Gont's ICMP as Oulu's ASN.1/SNMP as Guardent's TCP as Lee' TCP, etc. ad naeseum.

(If I weren't typing on this damn Netbook I'd add links but google them yourself if you are interested. But you get the point)

Within the mainstream media, each of these will be covered with approximately the same number of words, the same oversimplification and carefully selected, out of context quotes, regardless of the technical merit of the research, regardless of the scope of the flaws, and the professionalism (or lack thereof) of the finders.

And each time there where will be the Oh-My-God-the-Internet-is-Doomed-thank-God-for-the-Hackers-that-Saved-It narrative.

(Compare the recent wired article on Summer DNS flaws with the coverage of the 2003 TCP vulnerability discovered by Paul (Tony) Watson (aka the man that saved the Internet) and you will see an eerie similarity.)

Another wasted news cycle, and despite the claims of the finders, the security of th e Infrastructure is not improved. End users are either confused or cynical. It is conference season again. It is just too easy to dimiss the research as an individual trying to make a name for themselves and climb the corporate security ladder, a consulting company marketing its services or a vendor hawking their wares in the guise of a BlackHat talk.

Unless there is proof.

And that is where it looks like this will be different. There is a huge difference between what you can prove with a few boxes in your basement, a one-rack testbed with 50-100k of gear, an ISP with live users, or the larger Internet.

Each environment to demonstrate attack vectors and vulnerabilities is increasingly less contrived and more and more like reality. Each is an environment less out of the control of the attacker/adversary/researcher which is where it starts to get interesting. Meaning attacks on an Internet scale.

That is why real incidents (i.e. the smurf attacks of 98, the DDoS of 2000, the worms) teach far better lessons. They provide real data. They impact the bottom lines of vendors and users and impact operational best practicies.

Compare that with flash in the pan vulnerability presentations and you'll see why in the long run I wish more researchers would go beyond proof of concept and operationalize their exploits and discovered vulnerabilities.

Regardless of the technical details of the disclosure, it will be interesting to watch what happens. Will this be more of the same or the start of something new?

Libgmail, Twitter, Ideapad S10 Hardware Info on Linux

2008-12-28T19:08:00.003-06:00

Not sure why but (under Ubuntu) the fan has been running nonstop for the past hour (not sure why, perhaps because the temperature is 61 C, who knows) even booted into fluxbox. I have a little project/tool involving gmail that I'm hoping to get done before the end of the year (not a vuln or anything, don't get too excited) that I've had my head in libgmail for the past 24 hours as well as starting to sort of get the point of Twitter (notice my tweets on the side) but I started poking around at /proc and lshw/dmidecode, etc on the S10. There is definitely something funky with power management. A couple of times (even under XP) the battery value doesn't show up properly and gnome-power-manager doesn't start up properly 20-25% of the time.

description: VGA compatible controller
product: Mobile 945GME Express Integrated Graphics Controller
vendor: Intel Corporation
physical id: 2

description: Ethernet interface
product: NetLink BCM5906M Fast Ethernet PCI Express
vendor: Broadcom Corporation
capabilities: pm vpd msi pciexpress bus_master cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd autonegotiation
configuration: autonegotiation=on broadcast=yes driver=tg3 drive rversion=3.94 latency=0 link=no module=tg3 multicast=yes port=twisted pair

description: Wireless interface
product: BCM4312 802.11b/g
vendor: Broadcom Corporation
capabilities: pm msi pciexpress bus_master cap_list ethernet physical wireless

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 28
model name : Intel(R) Atom(TM) CPU N270 @ 1.60GHz
stepping : 2
cpu MHz : 800.000
cache size : 512 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yesflags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx constant_tsc arch_perfmon pebs bts pni monitor ds_cpl est tm2 ssse3 xtpr lahf_lm
bogomips : 3191.95
clflush size : 64
power management:
cpuid level : 10
wp : yes00:00.0 Host bridge: Intel Corporation Mobile 945GME Express Memory Controller Hub (rev 03)
00:02.0 VGA compatible controller: Intel Corporation Mobile 945GME Express Integrated Graphics Controller (rev 03)
00:02.1 Display controller: Intel Corporation Mobile 945GM/GMS/GME, 943/940GML Express Integrated Graphics Controller (rev 03)
00:1b.0 Audio device: Intel Corporation 82801G (ICH7 Family) High Definition Audio Controller (rev 02)
00:1c.0 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 1 (rev 02)
00:1c.1 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 2 (rev 02)
00:1c.2 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 3 (rev 02)
00:1d.0 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #1 (rev 02)
00:1d.1 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #2 (rev 02)
00:1d.2 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #3 (rev 02)
00:1d.3 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #4 (rev 02)
00:1d.7 USB Controller: Intel Corporation 82801G (ICH7 Family) USB2 EHCI Controller (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev e2)
00:1f.0 ISA bridge: Intel Corporation 82801GBM (ICH7-M) LPC Interface Bridge (rev 02)
00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE Controller (rev 02)
00:1f.2 IDE interface: Intel Corporation 82801GBM/GHM (ICH7 Family) SATA IDE Controller (rev 02)
00:1f.3 SMBus: Intel Corporation 82801G (ICH7 Family) SMBus Controller (rev 02)
02:00.0 Ethernet controller: Broadcom Corporation NetLink BCM5906M Fast Ethernet PCI Express (rev 02)
05:00.0 Network controller: Broadcom Corporation BCM4312 802.11b/g (rev 01)

Belief comes last

2008-12-24T07:14:00.000-06:00

My wife turned me on to this...

Are you down with OSCP?

2008-12-24T06:46:00.006-06:00

I'm generally weary (not wary!) about anything related to SSL or MITM (and particularly SSL MITM's) but Traffic for Revoked TLSv1 Certificate is actually pretty interesting drink coffee while only the 1 year old is up and toddling around, catch up on blogs on your new Netbook activity.

And Richard's traffic dissection, reminded its been weeks (months?) since I've fired up Wireshark. Tcpdump, every other day, but Wireshark, not so much lately.

Merry Christmas!

Ideapad S10 Upgrade Complete

2008-12-23T21:10:00.003-06:00

So I only ended up working a half-day today, so this afternoon (in between watching the kids and doing some last minute shopping for Christmas) I put in my old 120GB drive from my wife's dead MacBook and started the arduous process of copying a drive image from the original 80GB drive that ships with it, to the new.

(The added bonus was that I Ubuntu automatically mounted the HFS+ so I was able to recover a bunch of picture from iPhoto and phone booth)

And none of this would have been possible without UNetbootin.

It is definitely the hero of the day.

So the S10 has a weird partitioning layout. It only uses the first half of the drive (in my case 80GB) for the XP Home (FAT32) partition there are 3-4 other partitions, some more or less hidden for the backup features that I would know about if I had bothered to break the seals on the product documentaiton.

Of course dd|gzip (then back again) on an Atom processer takes forever so there was a lot of downtime. 40 GB images. I think it compressed down to 7GB or so, but to and from an external USB drive. You get the point. Drive imaging is slow, but now I have a dual boot (8.10 and XP Home) S10. This is the 2nd time I installed 8.10 and the long and short of it it is supported pretty well. No more quirks than on other Laptop hardware.

First Ideapad Blog

2008-12-22T17:55:00.004-06:00

Well my Lenovo Ideapad S10 came today, several days ahead of schedule. I guess one of the benefits of a global economic downturn is fulfillment centers chock full of crap that people shouldn't be buying in the first place.

It turns out I had ordered a white one. Oh well, couldn't remember. I thought that might be the case. That's OK need to find some small stickers (OpenBSD, not!) to put on it to look cool.

Still running XP for now (will probably leave it on the 80GB that came with it) but I did boot off 8.10 from a USB key and the only thing that didn't appear to work was the batter life indicator. I didn't try WPA, only briefly hopped on a neighbor's open AP to test the browser because I couldn't remember my WPA2 key and haven't added the MAC to the router yet. Popped in the RAM from my wife's MacBook and up to 1.5.

If you ordered the sleeve (and are a man who is not manly enough to not care if you had a pink notebook sleeve) it is reversible.

So here are the first impressions:

It is definitely smaller than I expected, and the keyboard is more cramped than I expected. TAB key is very tiny making shell expansion difficult under cygwin/remote system. But it is usable (typing on it now)
Performance is snappier than I thought, given the lousy benchmarks I read about. Chrome on Atom is a nice platform.
It definitely feels solid, not like a toy, as I envisioned the Dell Mini 9 to be. Hinges are nice and stiff.
Get rid of Norton and their stupid phishing toolbars which suck up screen geometry. Lots of apps barely fit.
The touchpad rocks. Hell of a lot better than crappy Dell touchpads (at least the ones I used on Latitude/Precision.
It is sort of tricky removing/putting the expansion cover (for HD and RAM) on and off. I was afraid I was going to break it.
The is a noticible audible blowing sound (the fan I believe)
Speakers are about what I expect. Not great. Not terrible.
Watched the start of the latest episode of Chuck on Hulu and was decent.

Forget about OWASP, go for Webkin Application Security!

2008-12-21T13:37:00.008-06:00

So my daughter received an early Christmas present, a Webkins Clydesdale horse, and when I registered this "adopted" pet (as an adoptive parent, I'm always find stuff like this mildly offensive) I was shocked to see the number of disclaimers, guidance on passwords security, protecting your secret code, etc. during the initial registration far exceeds many security products and public web portals, online banking sites, etc.

And most effective was the animated goose (with glasses propped down on her nose) scolding you about the dangers of a weak password or sharing your secret code.

Of course coming up with the squid proxy whitelist was sort of painful because they use a lot hardcoded IP address in their app like below:

1229885588.050 117 192.168.10.103 TCP_MISS/200 1081 GET http://www.webkinz.com/XML/InstanceFactory/InstanceFactoryData.xml? - DIRECT/66.114.49.27 text/xml
1229885588.056 139 192.168.10.103 TCP_MISS/200 2406 GET http://www.webkinz.com/XML/L10N/TransList.xml? - DIRECT/66.114.49.27 text/xml
1229885615.945 136 192.168.10.103 TCP_MISS/200 851 GET http://www.webkinz.com/XML/WEBSTAT/call_config.xml? - DIRECT/66.114.49.27 text/xml
1229885615.977 953 192.168.10.103 TCP_MISS/200 6437 GET http://www.webkinz.com/XML/vnum_API.xml? - DIRECT/66.114.49.27 text/xml
1229885616.007 177 192.168.10.103 TCP_MISS/200 1713 POST http://66.48.69.99/sindex.php - DIRECT/66.48.69.99 text/xml
1229885617.630 101 192.168.10.103 TCP_MISS/200 359 POST http://66.48.69.99/getdate.php? - DIRECT/66.48.69.99 text/html
1229885617.647 116 192.168.10.103 TCP_MISS/200 672 POST http://66.48.69.99/sindex.php - DIRECT/66.48.69.99 text/xml
1229885617.656 136 192.168.10.103 TCP_MISS/200 374 POST http://66.48.69.123/sindex.php - DIRECT/66.48.69.123 text/xml
1229885617.902 122 192.168.10.103 TCP_MISS/200 470 POST http://66.48.69.123/sindex.php - DIRECT/66.48.69.123 text/plain
1229885617.956 175 192.168.10.103 TCP_MISS/404 630 GET http://66.48.69.104/DAS/2008_12_21.xml? - DIRECT/66.48.69.104 text/html

Is conntrackd really pfsync+CARP for Linux?

2008-12-19T17:42:00.012-06:00

Say it aint' so Joe, but conntrack-tools says it "provides and equivalent of OpenBSD's pfsync."

What can do the conntrack-tools for me?

Lots of cool things. conntrackd covers the specific aspects of stateful Linux firewalls to enable high availability solutions and it can be used as statistics collector of the firewall use as well. The command line interface conntrack provides an interface to add, delete and update flow entries, list current active flows in plain text/XML, current IPv4 NAT'ed flows, reset counters atomically, flush the connection tracking table and monitor connection tracking events among many other.

This is something I've been wondering about for a while and it looks like this project has been around since 2006.

Here is a presentation on the capabilities of this. There isn't much test data here, but based on the stats in the talk, the performance of conntrackd (my testing/production observations was done on similar hardware DL-145G3) look a significantly worse than FreeBSD/OpenBSD with PF+pfsync+CARP. Note that the CARP/VRRP functionality is performed by keepalived.

*BSD can be used in the Enterprise for high availability gigabit packet filtering, but it would be interesting to see if anyone is using iptables+conntrackd+keepalived for this?

Update
This presentation about a successful migration from Linux to OpenBSD confirmed my suspicions about conntrackd not being ready for prime time. And This USENIX article provided an interesting comparison between OpenBSD and Iptables.

Linux is, in general, more efficient than OpenBSD. In both router and bridge configurations, it spends less time forwarding packets. Furthermore, iptables filters packets more quickly than PF, with only one exception (in our testing): if the transport-layer protocol of the transit packet, say, UDP, differs from the specifiedtransport-protocol type of a sequence of rules—“protocol type” set to “TCP”in this example—PF ignores those rules and confronts the packet only with the rest of the set, acting more efficiently than Linux, which confronts the packet with all the rules in the set.

This feature of PF is very interesting. UDP-based attacks are very insidious, and most firewalls have rules to prevent many types of UDP datagram from accessing the network. Nevertheless, most traffic from and to a protected network is made up of TCP streams (protocols such as HTTP, SMTP, and FTP all use TCP). In such a case, PF may be more effective: it does not spend processing time comparing TCP packets with the set of rules destined to block UDP datagrams, avoiding delay in processing legitimate packets. Finally, unlike iptables, PF performs automatic optimization of the rule set, processing it in multiple linked lists [7, 8]. A way to optimize the search on the rule set for iptables is to resort to the “jump” parameter [18] for jumping to a subset of rules (i.e., a chain) reserved for TCP or UDP packets, depending on protocol type.

Of course even discounting performance, the iptables rulesets are much less elegant than ipf/pf.

Webjob

2008-12-19T10:16:00.003-06:00

WebJob looks more interesting that you would think from the description:

WebJob downloads a program or script from a remote WebJob server and executes it in one unified operation. Any output produced by the program/script is packaged up and sent to a remote, possibly different, WebJob server. WebJob is useful because it provides a mechanism for running known good programs on damaged or potentially compromised systems. This makes it ideal for remote diagnostics, incident response, and evidence collection. WebJob also provides a framework that is conducive to centralized management. Therefore, it can support and help automate a large number of common administrative tasks and host-based monitoring scenarios such as periodic system checks, file updates, integrity monitoring, patch/package management, and so on.

When you look at the use cases:

To date, WebJob has been successfully used to:

* Automatically harvest argus, ifconfig, lsof, netstat, ndd, patch, ps, tcpdump, (name your utility), etc. data
* Automatically update cron tabs, DNS records, password files, snort rules, web sites, (name your application), etc.
* Automatically update system binaries when their MD5s do not match expected values
* Conduct massive searches for credit card numbers, social security numbers, and suspect hashes
* Deploy FreeBSD, Linux, Solaris, and Windows packages
* Drive GUI-based Windows utilities via AutoIT scripts
* Harvest evidence and diagnostic information from hundreds (300+) of systems in parallel
* Harvest system information to perform security audits or compliance verification
* Implement a Virtual Evidence Locker (VEL)
* Implement and maintain a Poor Man's Compile Farm (PMCF)
* Implement and maintain a distributed malware test harness
* Perform integrity monitoring with FTimes
* Periodically perform administrative tasks on a 950+ node Content Delivery Network (CDN) and the list goes on and on...

I haven't tried it (and it would be interesting to see if it really can scale) but I will!

Andy v. Alan: Two Man Enter

2008-12-18T06:53:00.001-06:00

Having been on both sides of the fence this is amusing.

Come with me on a little journey. What if she had convinced him to buy her product? Well, that would only happen in one of a couple of ways. First, he decided to make the decision on his own not knowing what the business requirements for this product are. He has no business being CIO. Second, he comes to me and tells me that he wants it and asks for my input. I tell him we don't need it at the moment, there are more pressing projects and I haven't decided on a vendor. He still buys it. He has no business being CIO. So we now have a product that we don't currently need, may not meet all of our requirements, may not be the best fit or the best value for us and I have another piece to force into my security program.
Who wins?
Not me. I've now got another product forced on me and I am learning that my input and opinion are not really valuable to the company so why not move on.
Not my CIO. He has lost my respect and possibly my services. Now he has to find someone else to come in and learn the environment, business and everything else.
Not my company. They just spend a lot of money that wasn't necessary and may not meet their needs.
Not the sales person. She has damaged relationships with a potential customer down the road.
Not the vendor. They have now sold a product that if it doesn't do as expected or doesn't meet the business requirements will only cause the customer to have a bad taste in their mouth.
All of this could have been avoided if the sales person simply chose to wait until next year when a "real" decision could be made.

Lenovo Ideapad S10 it is

2008-12-17T21:07:00.003-06:00

Well I did my small part in propping up post-late-capitalism-2.0 by purchasing necessary consumer electronics devices, by finally ordering an Ideapad S10 Netbook (512MB/80GB) I think it was a black one, can't remember.

A number of stars aligned and it was a tossup at the end because there was Dell Mini-9 package for around $410 with 8GB SSD with Ubuntu and I really wanted to support a vendor that preinstalled Ubuntu (because I will be running it afterall) but here are the reasons I went with the Lenovo:

I hate Dell Laptops, although the M4300 I'm using right now is tolerable but I love my Thinkpad T-61. I had positive experiences with Lenovo support, the unknown of Dell support scares me. They don't call it "Dell Hell" for nothing
Expandability - the Lenovo case is really easy to pop open and the drive is a SATA, so I can pop in the 120GB from my wife's dead MacBook. And I can wait until prices some down on SATA SSD's. I think the 1GB stick from the MacBook should work, too. Dell just has 1 SODIMM slot. PCI-Express port., probably more for a SSD than wifi since I have a USB, although the 2 USB ports could come back an bite me.
Price - with the Lenovo Corporate discount, it was $30-40 cheaper.
Aesthetics - Lenovo is boxy with a matte finish vs. rounded and shiny. Looks more professional
OS/Storage - I can install my Own Damn Linux and it is not a bad thing to have another OS with XP on it. It will be interesting to see how lame the Atom is, but the larger hard has the potential of making it usable as a real computer (or so I think) vs. only as Linux netbook. I will either dual boot or pop in a new drive.
Screen Size - 10" vs. 8.9 with my eyes really blurry right now I need every larger pixel I can get.

The battery life is going to suck compared to the Mini-9 but there is a 6-cell that is available.

OpenVZ Virtual Ethernet Devices

2008-12-16T01:25:00.004-06:00

By default, OpenVZ uses the venet devices which on the network have the same mac address as the host (VE0/CT0). This actually proved to be a problem when I was trying to [Nessus] scan OpenVZ containers from the host.

(Basically I'm trying to migrate Linux VM's some of which are targets that we scan in class away from VMWare Server, and the behavior was that students were only able to scan the VE's if they were connected to a Nessus scanner that was not on the same physical system as the other containers. Got it?)

So I had seen an eth0 within the container and wondered what it was and how it is configured. Well the virtual ethernet device wiki page has the answers although I was not unable to get this working after waking up at 1:30 AM and being unable to go back to sleep. Will try again tomorrow.

CyberSecurity Sanity We Can Believe In

2008-12-15T13:29:00.005-06:00

With everybody and their pocket yoyo trumpeting the need for a Cyber-Czar it was good to see Dale's comments over on Digital Bond

1. The reorganization of responsibility will introduce delay and is unlikely to improve the situation

Let’s say the National Office for Cyberspace comes to be early in the Obama administration. We are in for an ineffective time period and disruption while the new organization is ’stood up’ and everyone figures what their new role is in this organization. Is it six months, a year or longer before the new organization is effective? Anyone who has dealt with government stand up efforts and associated bureaucracy is probably shaking their heads.

Many loyal blog readers have been involved in one or more re-orgs of large organization, especially with arrival of new management. How often has that really made a dramatic difference? I don’t see the organizational structure being even close to the biggest impediment to date.

2. This whole consolidation / czar concept that is the rage is flawed, at least as related to information security.

We like to think that we can bring in a superstar with charisma to become the czar, e.g. drug czar, education car czar, cyber security czar, …, and all will be well. In this control system cyber security effort I’d argue the key is the people three, four and five levels down from this charismatic czar.

We don't need to be creating new organizations.

We don't need a Cyber Defense Agency (or a Control Systems CERT for that matter).

Just do your F-ing jobs, people.

Major Vendor Netbook Pricing Showdown

2008-12-13T20:24:00.004-06:00

I've been longing for a Netbook for a while and am still watching craiglist like a hawk as well as pricing from Netbook vendors.

HP Mini HP 1000/1GB Ram/8GB SSD with 2 year warranty (WinXP obviously) - $459
Dell Inspiron Mini 9/512GB/8GB SSD with 2 year warranty (Ubuntu) - $463
Lenovo S10 IdeaPad/512GB/80GB SATA with 2 year Warranty (XPSP3) - $408

I'm sort of anal about getting warantees on laptops and I don't trust Acer, Asus, etc.

Ubuntu reportedly runs on these, but the Lenovo looks really tempting (especially because I'm somehow on their employee discount list). It looks easy to upgrade, like to put in a SSD when they get a little cheaper and it also has a express card slot which I would be more interested more for storage although looking at the BIOS it doesn't look like you could boot from it. It would be trivial to dual boot on the SATA drive, but it seems like the whole point of having at netbook is to have a SSD drive.

XDot Graphviz Viewer

2008-12-13T08:24:00.002-06:00

The last time I was using Graphviz I was mostly using the Mac, which had a nice viewer, but XDot seems to be the best option for Linux. It is a single Python script and all the dependencies are available in the repos.

Chuvakin waits for the "Retarded" SCADA 09 Predications

2008-12-12T07:15:00.003-06:00

I'm with Anton Chuvakin on this one:

“SCADA anything REALLY bad” (here) – to be really honest, I have not really seen it yet this year so no link, but it will come. Help yourself to previous year embarrassments :-)

Forest and Trees?

2008-12-11T08:11:00.002-06:00

Along with the neverending drumbeat from ElasticVapor on CyberSecurity, now even Tom Peters is bullish on the idea of a CyberSecurity czar and an increased focusing on "CyberSecurity" in the next administration.

While Mr. Bush did increase spending on cyberthreats, much, much more emphasis is called for—and the topic is too important to bury in DHS.

But if if you "create a new White House office to protect cyberspace from hackers, thieves and foreign agents, coordinating security efforts across U.S. military, intelligence and civilian agencies" isn't that creating another DHS?

The problem is not coordination (of all things) we don't need another figurehead or another advocate for "CyberSecurity."

The issue is implementation. This is dirty, tedious work that creation of another agency or czar is not going to solve.

Remember CP/M ladder?

2008-12-09T19:19:00.005-06:00

At bedtime, my son always asks me about what it was like when I was a kid and tonight I was telling him how much more powerful his Nintendo DS was than the first computer we owned, a Kaypro II but I remembered my favorite game Ladder that I was very pleased to find a Java implementation.

Pure genius.

Correction, my favorite game was Rats! when I was able to go into my dad's lab.

Whisky, Tango, Foxtrot, over?

2008-12-08T13:44:00.007-06:00

From U.S. is Losing Global CyberWar. '

What the hell does that even mean?

To compile the report, which is entitled "Securing Cyberspace in the 44th Presidency," commission members say they reviewed tens of thousands of pages of undisclosed documentation, visited forensics labs and the National Security Agency, and were briefed in closed-door sessions by top officials from Pentagon, CIA, and British spy agency MI5. From their research, they concluded that the U.S. badly needs a comprehensive cybersecurity policy to replace an outdated checklist of security requirements for government agencies under the existing Federal Information Security Management Act.

The report calls for the creation of a Center for Cybersecurity Operations that would act as a new regulator of computer security in both the public and private sector. Active policing of government and corporate networks would include new rules and a "red team" to test computers for vulnerabilities now being exploited with increasing sophistication and frequency by identity and credit card thieves, bank fraudsters, crime rings, and electronic spies. "We're playing a giant game of chess now and we're losing badly," says commission member Tom Kellermann, a former World Bank security official who now is vice-president of Security Awareness at Core Security.

So the need to replace their old checklists with new checklists and start testing for vulnerabilities.

Which genius thought that up?

Chess? Win vs. Lose?

If only they knew about the Petraeus doctrine

Define the problem in these terms, and winning battles becomes less urgent than pacifying populations and establishing effective governance.

War in this context implies not only coercion but also social engineering. As Nagl puts it, the security challenges of the 21st century will require the U.S. military “not just to dominate land operations, but to change entire societies.”

If you can't even define victory in Iraq (conventionally or unconventionally) how can you define victory in "Cyberspace?"

(And of course over on ElasticVapor they think this is a great article which confirms my suspicions)

Please tell me what all this is going to accomplish or attempt that has not already been tried in the last decade?

But Is Intrepid Ubuntu Enough?

2008-12-07T09:18:00.002-06:00

Several years ago my wife was using Hoary Hedghog on her laptop and it suited her just fine, but I really haven't tried switching family members away from Windows to Linux. Actually I have counseled them to get Mac's but they haven't gone for it.

But this week my sister in law got some nasty malware from Facebook (this appears to be going around) and spending a few hours descending into safemode trying to get rid of of it with McAfee, Windows Defender, and ClavAV. No luck. Yesterday I had booted up with a Fedora 10 LiveCD and my brother liked the look of that and given the hell of dealing with malware they are willing to give Ubuntu a shot. They already use OpenOffice and Firefox sould it is the ideal case. But we'll see if it is painless enough.

So I added Ubuntu 8.10 to the Dell Optiplex 330. Added medibuntu repos, installed flash, Realplayer, and the codecs for mp3's even added the Gnome weather applet to the toolbar. Hell, even imported their pictures into F-Spot. What I was most pleased about what that the several year old D-Link PCI Wireless card worked out of the box, even with WPA.

What didn't work well was the GUI partition resizing, which failed silently, but I was able to drop down and do an ntfsresize -i to see that it was unclean NTFS partition.

So there is no way that an average user could do Dual Boot or setup all the repos and install the software themselves without doing a whole of lot of reading, but now we will see how painful it is. I'm particularly concerned about printing. My sister in law still needs access to some windows programs so I'll try Dual Booting and/or possibly VMWare Player.

If this is successful I want to replace XP Home on my mom's 3-4 year old Dell laptop because it is starting to die a slow death.

Fedora 10 on T-61 Passes the Sniff Test (But Fails the Virtualization Test)

2008-12-04T22:08:00.006-06:00

Did the install from the LiveCD. First go around I had forgot to unmount /dev/sda1 (where I was going to install it) so it failed but on the 2nd try booted just fine. Installed Flash from the repositories. Added another repo so I Gstreamer could handle mp3 streams. Sound works. Haven't tried WPA but that should work too.

Still not as smooth as the last two (or three) Ubuntu releases for but still usable, but the two things I do like

1) Fedora GNOME install has mini-commander. Yeah!!!
2) Fonts are bigger for an old guy like me ;)

But easy virtualization (compared to Ubuntu) forget about it

No easy OpenVZ. KVM (which sucks with QEMU)/Virtual Machine Manager didn't work out of the box (or at least not in 5 minutes the way it does on Ubuntu) and no dom0.

Lame. I really wanted a Xen-friendly distro to dual boot for my Thinkpad, since Centos/RHEL kernel is so old it won't support the hardware.

Trisano: Open Source & National Security

2008-12-04T21:35:00.002-06:00

As opposed to my previous blog Trisano would be a real example (if it does what it say) that Open Source improves national security:

TriSano™ is an open source, citizen-focused surveillance and outbreak management system for infectious disease, environmental hazards, and bioterrorism attacks. It allows local, state and federal entities to track, control and ultimately prevent illness and death.

Open Source Cliches of the Day

2008-12-04T19:38:00.017-06:00

First it was ludicrous article Open source is dead long live open source (don't get me started about the notion that Open Source code is so good that it doesn't need support, put the crackpipe down!) and then Bejtlich Cited in Economist.

While kudos go out to Richard (and I'm quite jealous) about being cited in The Economist I wish it would have been about NSM and not Open Source:

One way for governments to do this [to become resilient to cyber attack], says Richard Bejtlich, a former digital-security officer with the United States Air Force who now works at GE, an American conglomerate, might be to make greater use of open-source software, the underlying source code of which is available to anyone to inspect and improve. To those outside the field of computer security, and particularly to government types, the idea that such software can be more secure than code that is kept under lock and key can be difficult to accept. But from web-browsers to operating systems to encryption algorithms, the more people can scrutinise a piece of code, the more likely it is that its weak spots will be found and fixed. It may be that open-source defence is the best preparation for open-source attack.

Besides being included in article on my non-favorite topic of late (cyber-anything makes me ill) I think Richard is repeating one (or maybe two) security cliches: the "more eyes mean greater security" and the oft-repeated negation of "security through security."

OpenBSD is not PHP.
Linux is not Apache.
Tomcat is not BIND.
Debian is not OpenSSH.
Fedora is not SELinux.
Firefox is not Ruby on Rails.

Each of these may or may not be "more secure" than the other--or compared to an individual development team within a vendor we know and love/hate.

Software security is about tools, talent, and techniques not whether code of open or closed. Developer culture and committed project leadership are what make software secure, not whether the code exposed to clueless masses to find security flaws. Furthermore, there is great diversity in code quality, development (and business/sponsorship) models among Open Source pojrects that make it very difficult to make these sort of generalizations about the security of Open Source, let alone the role of Open Source in "resisting a Cyberattack" and much better case could be made based about the Open Source network security toolset that Richard champions in thwarting attackers -- as opposed to the inherent robustness and integrity of the Open Source codebase. With the exception of the Intel community, how many government personnel (or their contractors) are spending time scrutinizing the Linux kernel source? Jakarta Struts?

Not too many, I would guess.

While I am certainly a huge advocate of Open Source (and have had the [mis]fortune of developing/operating Open Source-based security platforms performingcritical functionality within a large Enterprise to back it up) security would not be at the top of the list as the reason to develop on (or deploy) an Open Source stack. For me it is about control, customization, and cost. Probably in that order. Yes, transparency, can result in improved code security (meaning fewer vulnerabilities per line of code) and better decision making in terms of deciding when and whether to patch (if I can look at the diff I don't have to guess about the true impact of the cryptic Cisco/Microsoft advisory or worse) but this is only a potential that in many (perhaps) Open Source projects don't live up to in reality.

I dunno where my wife finds this stuff

2008-12-03T18:47:00.002-06:00

Money quote.

Black Jesus: "You ever hear of Steve Jobs?"

A Nice Xen vs. OpenVZ Comparison

2008-12-02T19:32:00.002-06:00

Why OpenVZ and not XEN has a nice summary of some of the differences that are relevant to some of the comments made in response to OpenVZ fever

OpenVZ has one strong limit compare to XEN, it is not a full visualization and therefore you're limited to Linux only containers. People working with Sun will recognize Solaris zones concept, that was introduced few years ago. Like for Solaris every OpenVZ zones shared the same kernel, which at OVH translate in a Linux-2.6.24.7 kernel. This being said, it is important to understand that Linux distributions are independent of kernel, you can therefore run any Linux distributions you want under a unique kernel. While OVH ships Debian Etch with OpenVZ hyperviseur, you can chose any other distribution for your zones, new version of Fridu mostly operated with Ubuntu, but nothing prevents you from running multiple distributions. OVH ships template for Debian, CentOS, Gentoo and Ubuntu, but if this is not enough you can either create your own template or download one from Internet (OpenVz-WIKI)

OpenVZ includes a set of scripts to create/manage virtual machines, unlike Xen that is shipped naked and where I had to write more or less equivalent scripts by myself (cf: Fridu Xen Quick Start). Furthermore OVH ships OpenVZ with a web console from Proxmox, not that I'm a big fan of having a GUI, but as you can see on the video, it is great to make sexy demos.This console allows you to create a new virtual instances literally in a mater of seconds :) It allows you to start/stop change ram size, IP adresses, etc. on any instances without forcing you to remember any special commands. While Proxmox console misses few features like an SSH applet, a firewall config, or a java VPN. I must say that I get used to it and create every virtual machine through the web GUI.

OpenVZ is very light weight, not only it shares the same kernel, but also the same filesystem and networking stack. Direct result is that, on a given server you can run more OpenVZ zones than you could run XEN virtual-machines. From a user point of view when a zone is up, wether you run OpenVZ or XEN is fairly transparent, this being said they are nevertheless some fundamental differences: