Finally Went Android (Or, My Favorite Apps)

So last month I finally broke down and got an Android phone. I'd my eye on them for a while. I've never had a desire to get an iPhone, mostly due to my aversion to AT&T, but also because I am not an iPhone guy just like I wasn't an Apple guy in high school (not suprisingly, I was a Kaypro and then a C-64 guy) although I've been through a couple of Mac phases over the years.

I went with a much maligned Motorola Cliq XT, mostly because it was cheap, small, and fairly rugged. That is my preference for cell phones over the years. A lot of folks complained about Motoblur but it really isn't that bad. Realistically you don't have to even use it. Delete the bubbles off your home desktop and you really won't know it is there.

The Cliq XT It is not perfect but hey it is just a phone! Plus I'm chained to my BlackBerry so it doesn't really matter if it reboots while I'm on a call. Someone can get ahold of me on that. Yeah I know it is still Android 1.5, but I don't care. In terms of usability for non-email tasks it is such an improvement over my BlackBerry. I also like the fact of having two different carriers. I had no issue with Sprint but their upgrade prices were awful and I got free activation from T-Mobile.

(Pro tip: if you are unfortunate to work in a large company at least check for their corporate discounts with cell providers. Not only do you save money but you get better service when you call in.)

So what are my favorite apps?

Seesmic. That is a no brainer. No ads. Works for high volume account. I tried the "Happenings" app for my low volume account. Forget about it.

Opera Mini - I really haven't given the built-in browser a chance. Haven't wanted to.

Easy Tether -- I really only use this as a backup but it works on Ubuntu 10.04 and Windows without issue. I did manage to do an adb shell, but never got the SOCKS proxy working.

Some Battery Widget on my desktop that allows me to toggle GPS/Wifi.

Connect Bot -- I still haven't generated public keys, but it is more useful than you would think. Still haven't figured out how to do a CTRL-C, so unless you know how, don't use "top"

Pandora -- everybody know what Pandora is. I use this to help put my toddler to sleep. Try the "Lucinda Williams" channel it works, although makes you miss Texas.

Android System Information - this barely makes the cut but at least I haven't deleted it.

Advanced Task Killer Free - I really haven't touch this much, but I think it is doing something?

Pictures from Texas!

Spring Break 2010

218!

So I've been listing too much of the raucous debate in the house about health care on C-SPAN.

Politico provides the best analysis of the behind-the-scenes action in what led to where we are:

The rebirth of the reform effort is the result of a little luck, insurance company avarice, a subsiding of post-Brown panic among party incumbents and the calculation by many Hill Democrats that going small or giving up was just as politically perilous as going big.

But the main reason the bill has made it to the floor has as much to do with the complex, occasionally tense, ever-evolving partnership between the first African-American president and the first female speaker.

and

Publicly, the White House seemed to send a different signal each day.

In the space of two weeks, Obama or his top advisers suggested breaking the bill into smaller parts, keeping it together in one comprehensive package, putting it at the back of legislative line and needing to “punch it through” Congress, as Obama himself said at one point.

At a fundraiser in early February, Obama described the “next step” as sitting down with Republicans, Democrats and health care experts, describing a process that could take weeks, if not longer. He also seemed to acknowledge for the first time that Congress may well decide to scrap health care altogether — an admission that blunted his repeated and emphatic vows to finish the job.

Behind the scenes, Obama had, in fact, already settled on a strategy.

He would invite Republicans and Democrats to a summit, to give them one last chance at compromise, knowing they wouldn’t budge. And privately, he had decided that his favored approach was a comprehensive bill.

However, what I've been thinking about over the last few days is the similarity between how health care reform, Hillary, and McCain were handled by the Obama team. See my from October 08 where I mentioned an Andrew Sullivan article of how Obama's calm (and sometimes perceived weakness) lures the opponents into a false sense of victory and incites them. It was easy to doubt him during the primary (why wasn't he more aggressive against Clinton) and during the Summer of 08 (why did he take that stupid trip to Europe) but in all cases (assume this goes through) he pulls it off.

Will this really make a difference? I don't know and I really don't care. I knew enough to support it: the exclusions about pre-existing conditions (which impacted my family first hand) was good enough for me, although I know how much I paid for my plan at Cisco a decade ago, probably 1/5 in a comparable-sized company. As if the campaign of 2008 (the selection of Palin) didn't prove it enough, the GOP continues to show it's true colors and they don't match mine anymore. Even though I have elected more Republicans than Democrats over the years (not like you really have a choice in Texas) it is hard to know whether the GOP is really that ignorant or are they just that deceptive in order to go "downscale" Republican base that Bush built. As always, I'm a political reactionary. For me, just as my vote for Bush in 2004 was more a vote against Michael Moore and his kind, my support for this bill is just as much about giving Limbaugh and Beck the finger as passing needed reforms. And the former is a lot more certain than the latter.

Installing CouchDB/CouchApp on 64 Bit Debian 5.x

So the manual installation for CouchApp is mostly correct, but here were some slight modifications.

NOTE: my installation assumed everything that was custom-compiled went into /opt to keep a clean segregation of anything that is part of the distribution vs. hand compiled.

1. Install erlang from source (V5.7.4). I removed the 5-6 erlang packages but still got the dependencies. I also had to add libncurses5-dev to the list of packages for Erlang to compile.

./configure --with-ssl --prefix=/opt

2. Of course now I had to use the following when building CouchDB:

./configure --prefix=/opt --with-erlang=/opt/lib/erlang/usr/include/

3. Adjust paths accordingly to /opt/var/lib/couchdb /opt/var/log/couchdb etc.

NOTE: This VZ was running on 64bit OpenVZ

Linux debian5amd64-50 2.6.26-2-openvz-amd64 #1 SMP Thu Feb 11 01:40:09 UTC 2010 x86_64 GNU/Linux

Did the Stimulus Work?

Spending 2-3 hours on the D.C. beltway (yeah it took me that long to get from Rockville to Ellicot City tonight) does not put you in best frame of mind, but I took a break from tech stuff and catch up on some politics, for a change and actually blog instead of tweet. Apart from the campaign-style movie above, there is Judging the Stimulus by Job Data Reveals Success with the key argument as

The case against the stimulus revolves around the idea that the economy would be no worse off without it. As a Wall Street Journal opinion piece put it last year, “The resilience of the private sector following the fall 2008 panic — not the fiscal stimulus program — deserves the lion’s share of the credit for the impressive growth improvement.” In a touch of unintended irony, two of article’s three authors were listed as working at a research institution named for Herbert Hoover.

Of course, no one can be certain about what would have happened in an alternate universe without a $787 billion stimulus. But there are two main reasons to think the hard-core skeptics are misguided — above and beyond those complicated, independent economic analyses.

The first is the basic narrative that the data offer. Pick just about any area of the economy and you come across the stimulus bill’s footprints.

In the early months of last year, spending by state and local governments was falling rapidly, as was tax revenue. In the spring, tax revenue continued to drop, yet spending jumped — during the very time when state and local officials were finding out roughly how much stimulus money they would be receiving. This is the money that has kept teachers, police officers, health care workers and firefighters employed.

Then there is corporate spending. It surged in the final months of last year. Mark Zandi of Economy.com (who has advised the McCain campaign and Congressional Democrats) says that the Dec. 31 expiration of a tax credit for corporate investment, which was part of the stimulus, is a big reason.

Let's hope so.

A Maze of Twisty Fuzzers All Alike

Funny how a single innocent tweet can stir the pot. Not that I'm disappointed or that I mind, because the pot definitely needed to be stirred, but that certainly wasn't my intent on Monday. Really. But let's back up.

So I gave a short, not terribly technical presentation on Open Source fuzzing tools right before lunch at a conference on vulnerability discovery hosted by CERT at their office in Arlington. It was go to back there as I'd been to the SEI offices back in 2006 when I was working with them on the disclosure of some SCADA vulns.

Unfortunately, I didn't get to stick around the whole day (I missed Jared DeMott's presentation) and I was in and out during a conference call but there interesting talk by CERT, CERT-FI, Secunia, and Codenomicon.

But most interesting, and what led to my innocent tweet was a talk by Microsoft on how they use fuzzing and what were the results of different tools and approaches.

The conclusion I found to be surprising that they found that the use of "smart fuzzers" to have a lower ROI than the use of "dumb fuzzers" and their whitebox fuzzing platform called SAGE. Their point was the time it takes to define, model, and implement the protocol in a smart fuzzer is in most cases better spent having less skilled engineers run dumb fuzzers or white box tools.

They mentioned a talk at Blue Hat Security Briefings (I don't think this is the actual talk, but I don't have time to look for it) where they presented the bug results on a previously untested application were tested by a internally written (smart fuzzer), Peach (the dumb fuzzer?) and their whitebox fuzzing platform called SAGE. They mentioned an interesting technique of taking "major hashes" and "minor hashes" on the stack traces to isolate unique bugs. This is interesting because the primary focus has been on reducing the number of unique test cases but another approach is to look at the results. It may end up being more efficient. Of course this assumes the ability to have instrumented targets which may not always be the case, for example with embedded systems.

So Dale picked up on this and tried to apply this to the world of SCADA

We have two security vendors that are trying to sell products to the control system market: Wurldtech with their Achilles platform and Mu Dynamics with their Mu Test Suite. [FD: Wurldtech is a past Digital Bond client and advertiser] One of the features of these products is they both send a large number of malformed packets at an interface – - typically crashing protocol stacks that have ignored negative testing.

Mu responded within the comments in the blog and Wurldtech (far more defensively) on their own blog

In fact, our CTO Dr. Kube even gave a presentation at Cansecwest almost 2 years ago called “Fuzzing WTF” which was our first attempt to re-educate the community. To bolster the impact, we invited our friends at Codenomicon to help as they also were frustrated with the community discourse. The presentation can be found here.

Well I guess thie "re-education" (which sounds vaguely Maoist, I guess some of us need to be sent to a Wurldtech Fuzzing Re-education program) hasn't exactly worked although a satisfied Wurldtech customer did chime in on the Digital Bond blog. I actually agree that the need for better descriptions of fuzzing tools capabilities is needed and that was the entire point of my talk. I did a survey of the features available several dozen fuzzing tools and fuzzing frameworks that could be used to test.

I didn't spend as much time on the actual message generation as I should have and I was only focusing on Free and Open Source tools, but I identified a number of attributes for comparison such as target, execution mode, language, transport, template (generation, data model, built-in functions), fault payloads, debugging & instrumentation, and session handling. I'm not sure I completely hit my target but one of my goals was to develop some criteria to help folks make better choices on which Open Source tools could be used to most efficiently conduct robustness testing of your target. One of my conclusions (which I was pleased to hear echoed in the Microsoft talk) is that no single tool is best, no single approach is adequate--and that there are different types of fuzzing users that will require different feature sets. A QA engineer (that may have little to no security expertise) requires different features from those required for a pen-tester (or perhaps security analyst as part of a compliance-based engagement) which are still different from a hard core security researcher.

And the same applies to commercial tools you are paying tens of thousands of dollars for. One size does not fit all, regardless of the marketing (or mathematical) claims of the vendor. It would definitely be good to see a bakeoff of the leading commercial and Open Source fuzzing/protocol robustness tools similar to what Jeff Mercer has been doing for webapp scanners but I'm not optimistic that we will see that on the commercial tools because they are too expensive and the primarily customers for these tools (large vendors) are not going to disclose enough details about the vulnerabilities discovered to provide a rich enough data set for comparison.

It won't be me but perhaps some aspiring young hacker will take the time to do a thorough comparing the coverage of the tools that are out there against a reference implementation -- instead of writing yet another incomplete, poorly documented Open Soure fuzzer or fuzzing framework.

Hello MongoDB (Jython Style)

It has been ages since I've played around with any of the Java scripting languages so I thought I'd give Jython a spin with MongoDB. I have no idea about the performance between the pure Python vs. Java driver but it would be an interesting benchmark.

This is a very quick code snippet based on the MongoDB Java tutorial.

This was done on Ubuntu 9.10 with OpenJDK in the standard repositories and assumes the jython shell script is in your path. It also assumes the Java MongoDB driver is in your path and I was lazy so I didn't bother with CLASSPATH.

#!/usr/bin/env jython
import sys
sys.path.append("mongo-1.2.jar")
from com.mongodb import *
print "Jython MongoDB Example"
m = Mongo("10.0.0.33")
db = m.getDB("grid_example")

for c in db.getCollectionNames():
print c

And the output is just what you'd expect.

mfranz@karmic-t61:~/Documents/mongo$ ./jymongo.py
Jython MongoDB Example
fs.chunks
fs.files
system.indexes

Avoiding Bracket Hell in MongoDB Queries (Python Style)

To me it wasn't immediately obvious from the MongoDB Advanced Query documentation that you can string together multiple operators to perform existence, membership, and greater/than that tests. And since JSON can get very messy (and long!) and the syntax is slightly different from the Javascript in the documentation, instead of passing JSON directly to the find method of your collection pass a dictionary and assign the various conditions

For example:

myq = {}

myq["batchstamp"] = b # a timestamp

myq["modbus_tcp_reference_num"] = {"$exists": True}

cur = coll.find( myq )

Although it doesn't appear much easier than passing

{'modbus_tcp_reference_num': {'$exists': True}, 'batchstamp': 999999999}

Once start adding additional conditions (themselves which may have dictionaries it is much easier and less error prone. Trust me!

PyMongo for Dummies (using Squid logs, again)

In my last blog I showed some examples form the MongoDB shell. Next, we'll go through the PyMongo API, since only crazy people code in JavaScript.

In [3]: c = pymongo.Connection("192.168.169.62")
In [4]: db = c.mongosquid
In [5]: raw = db.raw In [6]: raw
Out[6]: Collection(Database(Connection('192.168.169.62', 27017), u'mongosquid'), u'raw')

We could have also referred to our collection as db["raw"] or db[coll] if you needed to define the collection in a variable.

In [7]: raw.count()
Out[7]: 205339

You can find out the methods that belong to the database with the collection_names() method.

In [40]: db.collection_names()
Out[40]: [u'raw', u'system.indexes']

The find_one() method allows you to quickly inspect your collection and take a peek at a sample document.

In [10]: raw.find_one()
Out[10]:

{u'_id': ObjectId('4b496cddb15cb004a4000000'), u'format': u'-', u'method': u'GET', u'size': 824477.0, u'source': u'192.168.1.254', u'squidcode': u'TCP_MISS/200', u'stamp': 1263096815.7609999, u'url': u'http://netflix086.as.nflximg.com.edgesuite.net/sa0/166/1680180166.wmv/range/660083845-660907844?'}

The distinct() method does have some limitations, as I discovered the hard way, as you an see from this exception.

In [13]: raw.distinct("stamp") --------------------------------------------------------------------------- OperationFailure Traceback (most recent call last) /root/ /usr/lib/python2.4/site-packages/pymongo-1.3-py2.4-linux-i686.egg/pymongo/collection.pyc in distinct(self, key) /usr/lib/python2.4/site-packages/pymongo-1.3-py2.4-linux-i686.egg/pymongo/cursor.pyc in distinct(self, key) /usr/lib/python2.4/site-packages/pymongo-1.3-py2.4-linux-i686.egg/pymongo/database.pyc in _command(self, command, allowable_errors, check, sock)

OperationFailure: command SON([('distinct', u'raw'), ('key', 'stamp')]) failed: assertion: distinct too big, 4mb cap

So in my previous blog (using JavaScript) I introduced queries but you really can't do anything useful without using a cursor. If you've ever done any MySQL coding before you should be familiar with the concept. Basically it allows you to iterate through the results of a query.

Here we have the same expressions but you obviously need to quote the gt in Python.

In [29]: c = raw.find( {'stamp': { "$gt": 1263096815 }})
In [31]: c.count()
Out[31]: 2060

and

In [23]: c = raw.find({'squidcode':'TCP_DENIED/403'})
In [24]: c.count()
Out[24]: 2999

For the sake of this exercise, we only want to see 3 results so we call the limit() method.

In [26]: c.limit(3)
Out[26]:

Now we can iterate through the results of our query.

In [27]: for e in c:
....: print e
....:
....:

{u'squidcode': u'TCP_DENIED/403', u'format': u'-', u'stamp': 1262520969.721, u'source': u'192.168.1.254', u'url': u'http://www.bing.com/favicon.ico', u'_id': ObjectId('4b496ea4b15cb004a6000000'), u'method': u'GET', u'size': 1419.0}

{u'squidcode': u'TCP_DENIED/403', u'format': u'-', u'stamp': 1262521126.928, u'source': u'192.168.1.254', u'url': u'http://www.msn.com/', u'_id': ObjectId('4b496ea4b15cb004a600003e'), u'method': u'GET', u'size': 1395.0}

{u'squidcode': u'TCP_DENIED/403', u'format': u'-', u'stamp': 1262521127.654, u'source': u'192.168.1.254', u'url': u'http://www.bing.com/favicon.ico', u'_id': ObjectId('4b496ea4b15cb004a600003f'), u'method': u'GET', u'size': 1419.0}

So if we try again, what happens?

In [28]: for e in c:
print e
....:
....:

Nada. We have to rewind the cursor object to be able iterate again.

In [30]: c.rewind()
Out[30]:
In [31]: for e in c:
print e ....: ....:

{u'squidcode': u'TCP_DENIED/403', u'format': u'-', u'stamp': 1262520969.721, u'source': u'192.168.1.254', u'url': u'http://www.bing.com/favicon.ico', u'_id': ObjectId('4b496ea4b15cb004a6000000'), u'method': u'GET', u'size': 1419.0}

You can also manually iterate through these by calling next()

In [51]: cr.next()
Out[51]:

{u'_id': ObjectId('4b496ea4b15cb004a6000000'), u'format': u'-', u'method': u'GET', u'size': 1419.0, u'source': u'192.168.1.254', u'squidcode': u'TCP_DENIED/403', u'stamp': 1262520969.721, u'url': u'http://www.bing.com/favicon.ico'}

In [52]: result = cr.next()

Guess what, your limit will still apply so if you want to clear it you can do a cr.rewind() and cr.limit(0) and then you can manually iterate through with cr.next()

Dummies Guide to MongoDB Queries using Squid Logs (JavaScript Shell Edition)

So the MongoDB develop documentation is actually pretty decent, but it doesn't really use examples with real data. For me, it made it more difficult for some of the API and shell commands to sink in.

So to generate some real world queries I created a python script that parsed the access.log file[s] generated by squid. I'll follow this blog with one that covers pymongo but I think this will be helpful, and like most of the posts will provide a good reference because when you are rapidly approaching 40 not only your eyes go, but your memory. So here goes...

First of all this assumes you are running the mongo JavaScript shell and yeah I know running from root is a bad idea and not even necessary (I don't think) but sue me.

root@opti620:~/mongodb# ./bin/mongo

MongoDB shell version: 1.2.1

url: test

connecting to: test

type "help" for help

> show dbs

admin

local

mongosquid

test

> use mongosquid

switched to db mongosquid

> show collections

raw

system.indexes

>

Now let's have some fun. This was actually when I just imported a few lines in from the log file so there are a relatively small number of documents. A collection is essentially like a table but since this is #nosql it really isn't a table. It is just collection of documents. We'll see those next.

> db.raw.find().count()

1029

> db.raw.find()[1029]

> db.raw.find()[1028]

{

"_id" : ObjectId("4b496cddb15cb004a4000404"),

"squidcode" : "TCP_MISS/200",

"source" : "192.168.1.254",

"stamp" : 1263102993.841,

"format" : "-",

"url" : "agmoviecontrol.netflix.com:443",

"method" : "CONNECT",

"size" : 17499

}

The JSON above is the "document." Something you'll notice is there are two different data types basically strings and floating points. The size field and timestamp are obviously floats. That hash looking thing is actually a hash or GUID that is supposedly unique.

So one of the cool built in queries is to return only the unique values for a given field. This is handled by the distinct method.

So we can see here that there were HTTP Posts.

> db.raw.distinct("method")

[ "CONNECT", "GET" ]

And because of my screwed up natting I can't tell which of my kids was going to netflix.

> db.raw.distinct("source")

[ "192.168.1.254" ]

> db.raw.distinct("url")

....

"http://netflix086.as.nflximg.com.edgesuite.net/sa0/725/1985205725.wma/range/9247565-9735184?",

"http://netflix086.as.nflximg.com.edgesuite.net/sa0/725/1985205725.wma/range/9735185-10219794?",

"http://netflix086.as.nflximg.com.edgesuite.net/sa0/725/1985205725.wma/range/985115-1469724?"

So remember when I discussed types above, if we wanted to retrieve all the transactions that were greater than 1MB we could do the following, but there are obviously more to it than that.

> db.raw.find( {size: { $gt:1000000}} )
{ "_id" : ObjectId("4b496cddb15cb004a4000162"), "squidcode" : "TCP_MISS/200", "source" : "192.168.1.254", "stamp" : 1263097489.996, "format" : "-", "url" : "http://netflix086.as.nflximg.com.edgesuite.net/sa0/166/1680180166.wmv/range/143155845-144163844?", "method" : "GET", "size" : 1008478 }
{ "_id" : ObjectId("4b496cddb15cb004a40003b0"), "squidcode" : "TCP_MISS/200", "source" : "192.168.1.254", "stamp" : 1263099100.207, "format" : "-", "url" : "http://netflix086.as.nflximg.com.edgesuite.net/sa0/166/1680180166.wmv/range/400771845-401779844?", "method" : "GET", "size" : 1008478 }

I was pleased to find that you can use regular expressions. The first query tells me there are 3199 documents that have port 443 in them and the 2nd query returns the first document. One of the things I noticed is that retrieving the document based on the "index" is really really slow. But I believe that is because it isn't really an index, but we'll get to them later.

> db.raw.find ( { url: /:443/ }).count()
3199
> db.raw.find ( { url: /:443/ })[0]
{
"_id" : ObjectId("4b496cddb15cb004a4000093"),
"squidcode" : "TCP_MISS/200",
"source" : "192.168.1.254",
"stamp" : 1263096929.091,
"format" : "-",
"url" : "agmoviecontrol.netflix.com:443",
"method" : "CONNECT",
"size" : 96222
}
> db.raw.find ( { url: /:443/ })[0:3]
Sun Jan 10 01:16:11 JS Error: SyntaxError: missing ] in index expression (shell):0

You'll notice that array slices don't work, but they do in Python, obviously which I'll blog on next.

FreeBSD 8.0 with rum0 and wpa_supplicant on Lenovo S10-2

It looks like the driver for rum has changed slightly in FreeBSD 8.0 from FreeBSD 7.2 because I was not able to use the same command-line syntax as I did previously. Basically the only thing different I did was the ifconfig wlan create...

I had this card running on old Dell Optiplex acting as a bridge for my kids network (and they were watching a lot of streaming media) and I was surprisingly impressed with it. Decent performance.

mfranz-bsd8#
ugen4.3:  at usbus4
rum0:  on usbus4
rum0: MAC/BBP RT2573 (rev 0x2573a), RF RT2528

mfranz-bsd8# cat /etc/wpa_supplicant.conf
network={
 ssid="xxx"
 psk="xxxx"
}


mfranz-bsd8# ifconfig wlan create wlandev rum0
wlan0
mfranz-bsd8# ifconfig wlan0
wlan0: flags=8802 metric 0 mtu 1500
 ether 00:1c:10:e6:1a:02
 media: IEEE 802.11 Wireless Ethernet autoselect (autoselect)
 status: no carrier
 ssid "" channel 1 (2412 Mhz 11b)
 country US authmode OPEN privacy OFF txpower 0 bmiss 7 scanvalid 60
 bgscan bgscanintvl 300 bgscanidle 250 roam:rssi 7 roam:rate 1
 bintval 0
mfranz-bsd8#

mfranz-bsd8# wpa_supplicant -c /etc/wpa_supplicant.conf -i wlan0
CTRL-EVENT-SCAN-RESULTS
Trying to associate with xxxxxxxxxx (SSID='xxxxxxxx' freq=2437 MHz)
Associated with xxxxxxxxxxx
WPA: Key negotiation completed with xxxxxxxxxxx [PTK=CCMP GTK=TKIP]
CTRL-EVENT-CONNECTED - Connection to xxxxxxxxxx completed (auth) [id=0 id_str=]

And while I'm at it, I hadn't seen any who actually installed 8.0 on a Lenovo Netbook but so far so good. I've got X working (I'll blog on that later) and re seems to work well enough. Obviously the Broadcom 4312's aren't going to work, but if you have USB wifi card or a tether you will be ok.

Next step see if I can get my Novatel u727 card working. I suspect it should work just fine, because it worked well on OpenBSD, but you never know...

Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
       The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:48:17 UTC 2009
   root@almeida.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Atom(TM) CPU N270   @ 1.60GHz (1602.40-MHz 686-class CPU)
 Origin = "GenuineIntel"  Id = 0x106c2  Stepping = 2
 Features=0xbfe9fbff
 Features2=0x40c39d>
 AMD Features2=0x1
 TSC: P-state invariant
real memory  = 1073741824 (1024 MB)
avail memory = 1026433024 (978 MB)
ACPI APIC Table: 
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 1 core(s) x 2 HTT threads
cpu0 (BSP): APIC ID:  0
cpu1 (AP/HT): APIC ID:  1
ioapic0: Changing APIC ID to 4
ioapic0  irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0:  on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit> port 0x408-0x40b on acpi0
acpi_ec0:  port 0x62,0x66 on acpi0
acpi_hpet0:  iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
acpi_button0:  on acpi0
acpi_lid0:  on acpi0
acpi_button1:  on acpi0
pcib0:  port 0xcf8-0xcff on acpi0
pci0:  on pcib0
vgapci0:  port 0x60f0-0x60f7 mem 0x58280000-0x582fffff,0x40000000-0x4fffffff,0x58300000-0x5833ffff irq 16
at device 2.0 on pci0
agp0:  on vgapci0
agp0: detected 7932k stolen memory
agp0: aperture size is 256M
vgapci1:  mem 0x58200000-0x5827ffff at device 2.1 on pci0
pci0:  at device 27.0 (no driver attached)
pcib1:  at device 28.0 on pci0
pci1:  on pcib1
pcib2:  at device 28.1 on pci0
pci2:  on pcib2
pci2:  at device 0.0 (no driver attached)
pcib3:  at device 28.2 on pci0
pci3:  on pcib3
re0:  port 0x2000-0x20ff mem 0x52010000-0x52010fff,0x52000000-0x5200ffff irq 18 at
device 0.0 on pci3
re0: Using 1 MSI messages
re0: Chip rev. 0x24800000
re0: MAC rev. 0x00400000
miibus0:  on re0
rlphy0:  PHY 1 on miibus0
rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
re0: Ethernet address: 00:26:22:0b:07:28
re0: [FILTER]
pcib4:  at device 28.3 on pci0
pci4:  on pcib4
uhci0:  port 0x60a0-0x60bf irq 16 at device 29.0 on pci0
uhci0: [ITHREAD]
uhci0: LegSup = 0x0f00
usbus0:  on uhci0
uhci1:  port 0x6080-0x609f irq 17 at device 29.1 on pci0
uhci1: [ITHREAD]
uhci1: LegSup = 0x0f00
usbus1:  on uhci1
uhci2:  port 0x6060-0x607f irq 18 at device 29.2 on pci0
uhci2: [ITHREAD]
uhci2: LegSup = 0x0f00
usbus2:  on uhci2
uhci3:  port 0x6040-0x605f irq 19 at device 29.3 on pci0
uhci3: [ITHREAD]
uhci3: LegSup = 0x0f00
usbus3:  on uhci3
ehci0:  mem 0x58344400-0x583447ff irq 16 at device 29.7 on pci0
ehci0: [ITHREAD]
usbus4: EHCI version 1.0
usbus4:  on ehci0
pcib5:  at device 30.0 on pci0
pci5:  on pcib5
isab0:  at device 31.0 on pci0
isa0:  on isab0
atapci0:  port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x60c0-0x60cf irq 16 at device 31.1 on pci0
ata0:  on atapci0
ata0: [ITHREAD]
atapci1:  port 0x60d8-0x60df,0x60fc-0x60ff,0x60d0-0x60d7,0x60f8-0x60fb,0x6020-0x602f mem 0x583440
00-0x583443ff irq 17 at device 31.2 on pci0
atapci1: [ITHREAD]
atapci1: AHCI called from vendor specific driver
atapci1: AHCI v1.10 controller with 4 1.5Gbps ports, PM not supported
ata2:  on atapci1
ata2: [ITHREAD]
ata3:  on atapci1
ata3sm0:  irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: [ITHREAD]
psm0: model Generic PS/2 mouse, device ID 0
cpu0:  on acpi0
est0:  on cpu0
p4tcc0:  on cpu0
cpu1:  on acpi0
est1:  on cpu1
p4tcc1:  on cpu1
pmtimer0 on isa0
orm0:  at iomem 0xcf000-0xcffff pnpid ORM0000 on isa0
sc0:  at flags 0x100 on isa0
sc0: VGA <16 flags="0x300">
vga0:  at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: parallel port not found.
Timecounters tick every 1.000 msec
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 12Mbps Full Speed USB v1.0
usbus4: 480Mbps High Speed USB v2.0
ad4: 152627MB  at ata2-master SATA150
ugen0.1:  at usbus0
uhub0:  on usbus0
ugen1.1:  at usbus1
uhub1:  on usbus1
ugen2.1:  at usbus2
uhub2:  on usbus2
ugen3.1:  at usbus3
uhub3:  on usbus3
ugen4.1:  at usbus4
uhub4:  on usbus4
: [ITHREAD]
GEOM: ad4: partition 1 does not start on a track boundary.
GEOM: ad4: partition 1 does not end on a track boundary.
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
Root mount waiting for: usbus4
Root mount waiting for: usbus4
Root mount waiting for: usbus4
uhub4: 8 ports with 8 removable, self powered
Root mount waiting for: usbus4
Root mount waiting for: usbus4
ugen4.2:  at usbus4
Trying to mount root from ufs:/dev/ad4s2a
ugen0.2:  at usbus0
ums0:  on usbus0
ums0: 2 buttons and [XY] coordinates ID=0
drm0:  on vgapci0
vgapci0: child drm0 requested pci_enable_busmaster
info: [drm] AGP at 0x40000000 256MB
info: [drm] Initialized i915 1.6.0 20080730

Some Shallow & Superficial Reasons for Picking MongoDB for your [web]app

So first got turned on to #nosql databases a little over (or under) a year ago with CouchDB but lately I've been quite enamored with MongoDB as of late.

So forgot about deep architectural reasons for using it. Here are some quite practical some practical reasons, when you are a not full-time developer (or database guru) but you find yourself doing development that involves a data store and the thought of using MySQL (so like 2000s) in your app:

• Abhorrence for schemas, ORMs, and migrations - this is basically the laziness argument. Basically I want/need to store stuff. And the stuff I want to store might change and I don't want to have to deal with changing the schema (and my) app to adapt to those changes. This was document oriented databases like CouchDB and MySQL rule. If everything is a JSON object it finds a great place for you to store stuff.
• Ease of Installation & Compilation -- yep CouchDB has been in the latest Ubuntu repos for a while, but I use Lenny/Hardy server side, so forget about it. Dealing with Erlang (and finding all the dependencies to build SpiderMonkey was a big pain) the ass. Beam, what the hell is beam? Mongo has 32/64 bit Linux binaries that just work and a briefly managed to get it to compile on FreeBSD 7.2. And unlike some of the others out there it doesn't require require a JRE.
• Map/Reduce hurts my head - ease of use is one of the key differentiators between Mongo and CouchDB is that is the simplicity of queries. I'm not an expert yet, but having to create Map/Reduce functions to create views to get at your data, it was a slippery concept for me.
• Non-HTTP Transport -- unlike CouchDB, Mongo has a binary client/server protocol and doesn't used HTTP.

There are also some really cool features like capped collection that should be useful for the app I'm working on, but these were some of the reasons why I went with Mongo. Back to coding...

Pansy or Victim?

So unfortunately some who I [used to] follow over on @frednecksec cited an article over on prisonplanet.com which allowed me to check out the cool sponsors such as the one pictured above but don't forget Silverlungs.

To each his own, I inhaling gaseous gold myself. Much better preparation for the "End Times," the "New World" or whatever the "elites" have in store for us.

Installing Redmine on Debian Etch

Here is a step by step summary of what I did to get to get Redmine up and running on Debian 5.x (Linux etch55 2.6.24-24-openvz #1 SMP Fri Sep 18 19:57:34 UTC 2009 i686 GNU/Linux)

If you don't know what Redmine is, it is like Trac, but better and on Rails. If you don't know what Trac is you probably wouldn't be interested in Redmine, so you can stop reading.

What it took?

1. Review this Rails on Debian guide. I'm sure there are others, but this is was a good starting point to get the nuances of running Rails apps on Debian, which can be a bit of pain if you are relying on packages. I know real rails folks use OSX but I'm not a real rails guy.

2. Install the necessary Debian packages. This is what I had to do on an OpenVZ VE, so you your packages may differ slightly: ruby ruby-dev irb sqlite3 ri libzlib-ruby libsqlite3-ruby libmysql-ruby mysql-serer mysql-client libopenssl-ruby

3. Install Rubygems the normal way. I installed 1.3.5. I created a symlink for gem1.8 just because.

4. Read the Redmine Installation Guide. Most of what you need to know is there, and I'm not going to repeat what it is there because it should just work, especially if you are familiar with rails or the configuration of rails apps.

5. Install rails and rake. I installed rails 2.1.2 based on the minimum for the 0.8x of redmine. I assume more recent versions of rails will work.

6. Download the stable release of Redmine.

7. You should have already configured mysql-server during the install, but make sure you put your password in the database.yml.

8. Create your database.yml, the session key and run the various rake scripts in the installation guide.

9. Fire up with webrick and login with admin/admin.

What went Wrong?

The main issues I had were related to not installing the right Debian packages. For example you definitely need libopenssl-ruby or the startup scripts will fail. I also screwed up the database.yml.

What Next?

Get git working following the instructions here. But first I need to work on my git skills, since I mostly have used subversion.

Sexing Up Your Boring Hardy Desktop to thwart Mac-Envy

So image is important. I get that. That is why so many security folks like Mac's these days. It certainly isn't because they are more reliable, but I digress.

Karmic has decent themes and wallpapers, but 8.04LTS is bloody awful. And you can't go onsite somewhere with a bunch of non-IT folks that will already be looking over you shoulder with ugly Ubuntu brown. It is bad enough that I have to use an older Dell D630, which of course runs faster than my E6400 with XP.

So the Bisigi Project has some pretty cool themes. I picked Showtime, a nice monochrome, almost but no completely OSX like theme. Next you need a simple monochrome theme for Firefox. If you have to use Firefox, Full Flat kicks ass, especially on Netbooks with limited resolution, but I swear it is faster. Lastly you need a nice monochrome wallpaper like Dark Times (after all it is Advent remember) from gnome-look.org.

And this just in, a monochrome Chrome theme to match.

Linux Netbook Use Case: EVDO/Wifi Firewall to protect your "Big Company" XP Laptop

So if you've used any large enterprise XP image you know they are awful.

The larger the company, the worse the build. They are slow. They crash all the time. They have a zillion agents running doing God knows what and they probably have the firewall disabled.

But you want to do the "right thing" and actually follow policy and NOT put Linux on the lovely Dell hardware they give like you used to do "back in the day."

And you don't want to run these in a coffee shop or an untrusted network. But I feel reasonably safe about running my Ubuntu S10-2 in relatively hostile environments.

Because most modern NICs (including the Broadcom's in most Atom-based netbooks) have auto-MDX so you can just directly plug in your laptop into the unused Ethernet on your Netbook after doing the following:

1) Configure a static address on the eth0 in (/etc/network/interfaces) making sure it is not an network you actually use (DOH!)

2) Make the appropriate change to sysctl.conf (if you have to ask...)
3) Install dnsmasq for DNS and DHCP (an apt-get away)
4) Add whatever iptables rules you want to rc.local (or run manually because if the ppp0 interface is not up it may not work)

Sometimes I share over the Wifi others I use my EVDO card.

Bottom line: it just works.

Bonus: you get to see whatever the hell all those pesky agents are doing when they phone home to your corporate network over the Internet.

Snow Photos and More

In lieu of Facebook. Recent birthday party, lego club, and more!

blizzard09

WebSocket Service Fingerprinting with Curl

Fingerprinting is probably a bit of a stretch, but at least I didn't use the "h" word, but using pywebsocket is probably the easiest way to learn about the protocol.

Startup the server....

franz@mfranz-s10-2:~/Documents/pywebsocket-read-only/src/mod_pywebsocket$ python standalone.py -p 8888 -w ../example/

Then the client...

mfranz@mfranz-s10-2:~/Documents/pywebsocket-read-only/src/example$ python echo_client.py -s 127.0.0.1 -p 8888
Send: Hello
Recv: Hello
Send: 日本
Recv: 日本
Send: Goodbye
Recv: Goodbye

Look at the traffic on the wire with ngrep.

interface: lo (127.0.0.0/255.0.0.0)
####
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
GET /echo HTTP/1.1..                                              
##
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
Upgrade: WebSocket..                                              
##
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
Connection: Upgrade..                                             
##
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
Host: 127.0.0.1:8888..                                            
##
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
Origin: http://localhost/..                                       
##
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
..                                                                
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
HTTP/1.1 101 Web Socket Protocol Handshake..                      
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
Upgrade: WebSocket..                                              
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
Connection: Upgrade..                                             
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
WebSocket-Origin:                                                 
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
http://localhost/                                                 
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
..                                                                
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
WebSocket-Location:                                               
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
ws://127.0.0.1:8888/echo                                          
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
..                                                                
##
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
..                                                                
##
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
.Hello.                                                           
#
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
.Hello.                                                           
#
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
........                                                          
#
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
........                                                          
#
T 127.0.0.1:44284 -> 127.0.0.1:8888 [AP]
.Goodbye.                                                         
#
T 127.0.0.1:8888 -> 127.0.0.1:44284 [AP]
.Goodbye.                                                         
###

Now with curl, notice the headers that you have to add to get a response. With anything less I got a 404. The origin header can be anything.

mfranz@mfranz-s10-2:~$ curl -v http://127.0.0.1:8888/echo -H "Upgrade: WebSocket" -H "Connection: Upgrade" -H "Origin: http://localhost"

* About to connect() to 127.0.0.1 port 8888 (#0)
*   Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET /echo HTTP/1.1
> User-Agent: curl/7.19.5 (i486-pc-linux-gnu) libcurl/7.19.5 OpenSSL/0.9.8g zlib/1.2.3.3 libidn/1.15
> Host: 127.0.0.1:8888
> Accept: */*
> Upgrade: WebSocket
> Connection: Upgrade
> Origin: http://localhost
>
<>
But if the URI doesn't match you get

mfranz@mfranz-s10-2:~$ curl -v http://127.0.0.1:8888/ -H "Upgrade: WebSocket" -H "Connection: Upgrade" -H "Origin: http://localhost"

* About to connect() to 127.0.0.1 port 8888 (#0)
*   Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.19.5 (i486-pc-linux-gnu) libcurl/7.19.5 OpenSSL/0.9.8g zlib/1.2.3.3 libidn/1.15
> Host: 127.0.0.1:8888
> Accept: */*
> Upgrade: WebSocket
> Connection: Upgrade
> Origin: http://localhost
>
* Empty reply from server
* Connection #0 to host 127.0.0.1 left intact
curl: (52) Empty reply from server
* Closing connection #0

Generating SVG Output (from Graphviz) in your Django App

So accomplishing new coding tasks can be a challenge with interruptions and I've had a lot of interruptions this week but I finally got there. And I'm thankful!

I have an app that is storing data, meaning Django models for the uninitiated. What is in there doesn't matter, but it is something that is conducive to plotting with graphviz. So the starting point is a string that is in the .dot format. I have some code that makes queries to the database and I end up with a string.

So there is a utility function that creates this string...

def make_svg_str:
 #blah blah blah snip
 dot_string += "}"
 p = subprocess.Popen('/usr/bin/dot -Tsvg', shell=True,\
 stdin=subprocess.PIPE, stdout=subprocess.PIPE)
 (stdout,stderr) = p.communicate(dot_string)
 return stdout

So I almost got this right the first time except that I forgot the stdout in Popen() which caused the output to go to stdout (and not be assigned to the string) so I saw the .xml in the dev web server logs.

The graphviz string (dot_string) is being piped to the dot executable and then the function is returning the XML SVG as a string, and is obviously assigned to the stdout variable in the tuple.

Now the tricky part within my views.py.

My first mistake was using the Django CSV docs instead of the PDF docs because the latter is what we need. I also didn't remember that HttpResponse is a file-like object so we can can just write to it once we have the SVG text.

def svg(request):
f = Foo.objects.all()
response = HttpResponse(mimetype='image/svg+xml')
response['Content-Disposition'] = 'filename=somefilename.svg'
response.write(make_svg_str(f))
return response

So this will display your image within your browser (which is what I wanted) instead of downloading file if you the use the "attachment" in the Content-Disposition key.

The name of the game is taking shortcuts that get the job done. I'm using the admin interface to provide a good-enough UI to enter the data and now I'm using Graphviz to visualize that data without having to spend a lot of time writing UIs or nasty JavaScript.

Where's the Controversy about Shodan?

So like a lot of folks I spent no more than 15 minutes this morning googling Shodan for anything interesting. I looked for SCADA protocols (there were none that I could easily find) or obvious field automation devices, so I went back to work. At best I found a bunch of VxWorks systems (and whole lot of ESX servers, shiver) and others like @chrisjager have also commented about the large number of embedded devices directly connected to the Internet, which is, indeed, frightening.

But @taosecurity just made some interesting comments, questioning how long the site will be up and hit upon in the ethical issues of a site which so obviously allows easy amplification of vulnerable systems. This was the first I've seen that even considers this angle. I'm not sure if everybody is getting ready for the holidays, trying to get the last bit of work done, or already gone but at least on the 300+ plus folks I follow on Twitter there were absolutely no questions about the site, and whether or not such as site was appropriate, ethical, etc. Just to be clear, I'm not claiming it is or is not, I'm just surprised it hasn't come up yet either way. Now if and when this happens (perhaps everyone else is so jaded and just does not want to go there) I'm sure the arguments will quickly fall into the typical cliched responses around disclosure:

• The site is raising awareness so is a good thing. Administrators can actually find and fix their systems.
• Anyone who has systems directly connected to the Internet with systems that vulnerable deserves to be compromised.
• The site is irresponsible and we should immediately DDoS it
  

And so on...

I don't actually believe any of those arguments. I'm not sure what to think. And I find that troubling. After nearly a decade in information security, I've become weary of all the arguments on either side of these sorts disclosure issue so I resort to know opinion because my opinion doesn't really matter and folks will release 0-days (or not) or more interesting sites like this (or not) and what will happen will happen regardless of any international standards or documented best practice working groups.

So back to trying to find a way to graphviz to generate SVG images within a Django app. That is at least a problem I can solve.