Down Came the Snow, Down went RCN Cable

Coicidentally with the snow today, RCN went to hell again but my Debian EVDO Router was ready. Actually got rid of the OpenWRT box (wasn't using the wireless anyway) and switched to wvdial, which has done a great job of automatically running pppd if the connection drops.

/etc/rc.local

if mount /dev/sr0 -t iso9660 /mnt
then
       echo "Found Novatel u727"
       sleep 3
       umount /dev/sr0
       eject /dev/sr0
fi

echo "1" > /proc/sys/net/ipv4/ip_forward
iptables -A POSTROUTING -t nat -o ppp0 -j MASQUERADE
sleep 10
wvdial &

/etc/wvdial.conf

[Dialer Defaults]
Modem = /dev/ttyUSB0
Baud = 460800
Init = ATZ
ISDN = 0
Modem Type = USB Modem
Phone = #777
Username = ''
Password = ''
Carrier Check = no
Stupid Mode = yes

Obviously need to clean up the iptables rules, although I'm not terribly worried about it.

Best WRT54G (v3) for Intel 4965AGN

I've been having a hell of a time with the Wireless card in my T-61and my Linksys router (firmware v1.02.0, Jan. 16, 2007) for the last few days. No problems with the OSX on either my Powerbook G4 (Broadcom) or my wife's Macbook (Atheros). Some of these were screwups on Ubuntu but there definitely appear to be some issues with this card and some Linksys WPA configurations. Under XPSP2, I was becoming dissasociated 3-4 times an hour and with WPA2 Personal (TKIP+AES) would not even work with LInux

I believe these are the default settings which seem to work the best:

WPA Personal
AES
Group Key Renewal - 3600

If you don't believe me, at least believe Chris Rock

Although not as good as the line about him not being afraid of "the media" robbing him at an ATM machine, this isn't bad either

“I love Hillary Clinton,” he continued, “but to me she is the Democratic version of George Bush: someone who is running, and the only reason you know who this person is is because of their name.”

But seriously, Check out Obama's latest speech from Iowa

That's the kind of change that's more than just rhetoric - that's change you can believe in. It's change that won't just come from more anger at Washington or turning up the heat on Republicans. There's no shortage of anger and bluster and bitter partisanship out there. We don't need more heat. We need more light. I've learned in my life that you can stand firm in your principles while still reaching out to those who might not always agree with you. And although the Republican operatives in Washington might not be interested in hearing what we have to say, I think Republican and independent voters outside of Washington are. That's the once-in-a-generation opportunity we have in this election.

I've been pretty cynical about politics (and most things over the years) but I actually contributed a few bucks to his campaign. I haven't decided who I'll vote for (or if I'll even vote) but I do know for damn sure who won't be getting my vote: Clinton or Romney.

Novatel u727 on Debian Etch

I previously blogged on getting this card working on Ubuntu but obviously nobody tried my instructions because it wouldn't have worked. The bizarre thing is that in order to get the USB serial devices to show up, you have to first mount the "Novatel CD" device that gets detected, unmount it, and then eject it. This only has to be done once after the device is powered up ( meaning if you unplug it) so here is what I added to the /etc/rc.local an old Optiplex 100 running Etch so things get automatically setup.

if mount /dev/sr0 -t iso9660 /mnt
then
       echo "Found Novatel u727"
       sleep 3
       umount /dev/sr0
       eject /dev/sr0
       sleep 10
       pppd call sprint
fi

The only thing left is to add an iptables commnad to masquerade everything out the ppp0 interface and I have my backup EVDO gateway. Well and change the default route on a box or two -- or get VRRP working.

So the next time RCN hits the fan (must have been the weather) I'll power up this box and plugin the EVDO adapter and I'll be good to go.

Aspen: A Python Web Server You Can Get Excited About

A year ago (or at least over Christmas and New Years) I was playing a lot with Django (and reading about WSGI) so its fitting I ran across Aspen.

Aspen is designed around the idea that there are basically two kinds of websites, publications and applications, differentiated by their organization and interface models. A publication website organizes information into individual pages within a hierarchical folder structure that one navigates by browsing. In an application website, on the other hand, data is not organized into hierarchical pages but is dealt with via a non-browsing interface such as a search box.

The HTML version of this documentation is an example of a publication website: a number of hypertext documents organized into sections. If we weren't using LaTeX (or if I knew how to use it better), the sections would probably be encoded in folders. Gmail is a pure application website, one which organizes and presents information non-hierarchically. Most websites, however, are hybrids. That is, within an overall hierarchical organization you will find both individual pages of information as well as applications such as a site search feature, or a threaded discussion forum.

Publication websites are actually a subset of application websites, of course. An application site can use any interface metaphor; a publication is an application that uses the familiar folder/page metaphor to organize and present its information. Therefore, every website is fundamentally an application.

Aspen enables the full range of websites: publications, applications, and hybrids. It uses the filesystem for the hierarchical structure of publication and hybrid websites, and provides a mechanism for including applications within that hierarchy.

Based on the screencast, it looks very cool. Why? It is so un-Ruby: well documented, it supports multiple frameworks (a Python HTTP server that support PHP!) and Conan O'Brien-style talking faces. Hopefully I'll be able to squeeze some time away from baby care to play around with it.

Open Source NAC and A [Kind of/Sort Of] Agentless Endpoint Posture Assessment for Debuntu Boxes

Previously I described a quite common situation I've encountered where non-compliant laptops and OS's are used by members of security teams, frequently in violation of technology/security policy (anyone else know the term "shadow IT"?) Another use case might by highly-skilled/trusted security consultants that could own your ass if they wanted to and probably already have the keys to the kingdom. A reader noted that one solution would be just to grant an exception to policy for these [already] trusted users, but this doesn't sit too well with me.

I personally would like to have some additional layer of monitoring above and beyond good-faith adherence to policy and the desire to do the right thing. The various Open Source NAC toolsets that are out there (many which seem to be developed within Universities) such as packetfence, FreeNAC, or RINGS seem like overkill and clearly inappropriate for this sort of user base.

What I had in mind was something much lighter weight that:

• Runs with user-level privileges
• Requires minimal level of installation, perhaps simple Ruby/Python script minimal to no third party libraries
  
• Communicates to a server vs. having the server interrogate the client (i.e. no agent listening for connections back from a centralized server)
  
• Provides flexible execution (run as a startup script or within desktop environment, gnome-session something or other)
  
• User authentication against to some sort of directory server (so we can associate a given endpoint with a user)
  

An initial environment would be Debian/Ubuntu (although OSX would be nice, too) that could provide the following information to a Rails/Django web app

• Linux kernel version and running kernel modules
• Last apt-get update
  
• Hardware (both real and virtual)
• Whether or not filesystem encryption is enabled (look for /dev/mapper stuff)
  
• Information about packages (such as the output of a dpkg -l)
• Listening services (netstat or lsof)
  
• Network information (interfaces, routing)
  

This assumes that there is something NAC-like (in the sense of segregating non-compliant PC's to a certain network/tunnel) or that users will voluntarily run some sort of script upon login.

That being said, as easy as this is to imagine, I'm sort of ambivalent about the usefulness of something like this (perhaps I've heard too many of the arguments about NAC "fighting the last war"), but it doesn't seem like it would be terribly difficult or time consuming to whip up a small client script that pulled together some basic Linux system information and sent it to a CRUD webapp to provide some basic auditing and reporting. And it wouldn't be much of a stretch to add some policy definition/enforcement based on the data or tie it to a iptables/PF box with anchors to implement different access profiles. Obviously, unlike most of the commercial (or even the Open Source) NAC solutions) this is a L3, n-hop-away solution. No 802.1x, no DHCP, no VLAN assignment, but it is actually deployable and might immediately provide some useful (and perhaps even actionable) information that is higher fidelity than a scan the endpoint with Nmap/Non-Auth Nessus or use passive device/app fingerprinting which seems a waste of time for the problem at hand.

Is there anything out there along these lines?

Sprint Novatel u727 on Ubuntu 7.10

Add vendor and product options to /etc/modules

usbserial vendor=0x1410 product=0x4100

Disable automounting of USB serial devices with gnome-volume-properties

Otherwise the USB Serial devices won't show up and you would have to unmount WTF that image that is being mounted from the

Create /etc/ppp/peers/sprint
/dev/ttyUSB0 # modem
115200 # speed
921600 # works, abt 60kbytes/sec on S620
#1036800 # doesn't work
defaultroute # use cellular network for default route
usepeerdns # use the DNS servers from the remote network
nodetach # keep pppd in the foreground
crtscts # hardware flow control
lock # lock the serial port
noauth # don't expect the modem to authenticate itself
local # don't use Carrier Detect or Data Terminal Ready
user
ppp
#passive
debug
lcp-echo-failure 4 # prevent timeouts (1of2)
lcp-echo-interval 65535 # prevent timeouts (2of2)
connect "/usr/sbin/chat -v -f /etc/chatscripts/sprint-connect"

Create /etc/chatscripts/sprint-connect

TIMEOUT 10
ABORT 'BUSY'
ABORT 'NO ANSWER'
ABORT 'ERROR'
SAY 'Starting SPRINT connect script\n'

# Get the modem's attention and reset it.
"" 'ATZ'
# E0=No echo, V1=English result codes
#OK 'ATE0V1'

OK 'ATDT#777'
CONNECT

Start pppd

root@gutsy61:~# pppd call sprint
Starting SPRINT connect script
Serial connection established.
using channel 1
Using interface ppp0
Connect: ppp0 <--> /dev/ttyUSB0

I might add links to the source materials (cause I obviously didn't come up with this all on my own) but this should work.

Hello Sprint EVDO Goodbye AT&T DSL!

Although I haven't yet managed to cancel my AT&T DSL order yet (I only had 5-6 hops before giving up, which reminded me of the reason I a year ago never to use them again) but after 24 hours I've given up. The service tech was nice enough, but DSL Self Install kit never arrived and I could never get a dial tone. Maybe the three daisy-chained telephone network interfaces had something to do with it. Or maybe it was the rats nest mix of Cat 5 and 1950's era kit, but I started looking for wireless alternatives. I've only been logged with the Novatal U727 (on my Powerbook, it supposedly works with Linux, too) for 32 minutes but so far so good.

WTF is "spock power" and why does it think I know Wietse Venema?

Well some folks in my LinkedIn network are now sending me spock trust invitations. Sure why not? Live on the edge. Invite more identity theft. Some of the obvious differences (besides all the tagging) between LinkedIn are that (I guess) you can trust someone and they might not trust you and that you (and your community?) can vote on various attributes (tags?). Another nice feature was the automatically generated (via google) content that you can also vote on. For example I was able to vote down a Matthew Franz's (in Arizona) MySpace page. Maybe I should get one of those too, assuming they let folks over 30 even use it. Nah.

Smartphones, Dementia, and the Demise of the PDA Market

I've never been a fan of $300 phones that can easily be dropped [into a bathtub] by your kids, or eaten by your dog but I ordered one of the Palm Centro's (from Sprint) over the weekend. Of course I had to cancel the order because not only did boneheads at Sprint interrupt an important call with my son's Dr. yesterday to confirm the order I placed over the weekend, but they wanted me to fax a copy of my drivers license and a bank statement F--- that. Like I have time to fax something somewhere. And I'm certainly not going to give them a bank statement. Why the hell do they need that if they've already done a credit check?

But, basically, I have felt like I've been losing my mind.

I have assumed most of the administrative/transportation/[child|pet]care duties while my wife recovers from the C-section. I can't remember names. I don't have all the phone numbers of neighbors, zillions of school officials I'm dealing with, various meds, and I only can only remember the times of appointments to the nearest 4 hour granularity.

I needed a PDA! That will solve my problems. But who uses PDA's anymore? Do they even sell them? Or perhaps a phone with decently calendaring and todo lists. That's all I want. No MP3 player. No camera. No shitty web browser. Why don't these devices exist? There were a couple of Samsung's (since the first little Startec I had when I worked at SBC I have loathed Motorola devices) that might have worked, but I don't want to sign another contract to get a decent price.

So I desperation, I picked up a Z22 (you can't spit without running across a Best Buy on this side of town, but good luck finding a Sprint store, of course now I just realized they sell Centro's at Best Buy, but no matter). Small and $99 and I didn't need to get new cell service. Whether or not it actually works, the GTD principle of dumping as much of the things you have to do/decide to do onto paper (or some electronic form) to avoid thinking about them (as a means of de-cluttering and de-stressing) has always seemed appealing. And it appears to be working.

And the Z22 feels very comfortable and soothing. I've had 5-6 PalmOS devices over the last decade. My 2nd CLIE was pro bably the best (before my dog cracked the screen) but Sony exited the market 3-4 years ago. It is a shame, because palm got a so many things right: a simple desktop and graffiti. I have looked at Window Mobile, Pocket PC, or WTF it is called but it just feels klunky. And of course you can't sync to Linux or use something like JPilot or pilot-link right?

Quick Blog Break to Retain Sanity (and Ron Paul cracks $12 million)

Back in May I first blogged on Paul. BTW, the number of political blogs are directly proportional to the stresses and strains of everyday living, but who would have thought they'd play Bush as Paul crosses 12 Million (Youtube video) and gets Andrew Sullivan's endorsement

But the deeper reason to support Ron Paul is a simple one. The great forgotten principles of the current Republican party are freedom and toleration. Paul's federalism, his deep suspicion of Washington power, his resistance to government spending, debt and inflation, his ability to grasp that not all human problems are soluble, least of all by government: these are principles that made me a conservative in the first place. No one in the current field articulates them as clearly and understands them as deeply as Paul. He is a man of faith who nonetheless sees a clear line between religion and politics. More than all this, he has somehow ignited a new movement of those who love freedom and want to rescue it from the do-gooding bromides of the left and the Christianist meddling of the right. The Paulites' enthusiasm for liberty, their unapologetic defense of core conservative principles, their awareness that in the new millennium, these principles of small government, self-reliance, cultural pluralism, and a humble foreign policy are more necessary than ever - no lover of liberty can stand by and not join them.
He's the real thing in a world of fakes and frauds. And in a primary campaign where the very future of conservatism is at stake, that cannot be ignored. In fact, it demands support.

Paul is likable enough, but I'd still have side with the only other authentic candidate on the Republican side. I mean I like crazy (McCain, like Paul has that sort of crazy edge) but Paul is just too out there. But I doubt either will get the nomination, barring a miracle. Of course the amazing thing is that (about 14 hours into my wife's labor) we watched the last Republican debate and she actually liked Huckabee. (And she will vote for Hillary, if she gets the nomination) And I'm still struggling with how the Christian Right can actually support him, when he sounds socially (if probably not culturally) liberal enough. Weird.

Back to the kid-ferry.

Into the World

I've gotten hooked on Andrew Sullivan's the View from your window so I thought I'd add mine.

The snow has finally stopped and we are going home.

Welcome Samuel Austin!

12/12 @ 10:38 CST - 8 lbs 14 oz much bigger than expected!


(Would have got this up sooner but had to setup a quick squid over SSH to get through websense)

AntiDote for Bad Customer Service Experience and Icy Roads

Between dealing with RCN and a clueless United Behavioral Health rep yesterday thinking I last had service in 2001 (this is my first bad experience with them, otherwise they are awesome and kick Magellan's ass) there is a need for some humor so check out the Immanual Kant Attack Ad (by Nietzsche) and Ron Paul's favorite Super Hero and Andrew Young's absurd comments on [Bill] Clinton and Obama who claims he was the first black president because he has slept with more black women than Obama.

Oh and on the not funny (but reassuring that conservative support for Obama is not a vast right wing conspiracy) this National Review article mocking the Messianic Obama.

Unfortunately, must leave warm Panera, get on the icy roads, and go to work.

RCN Cable Internet: Fun while it lasted

I knew I should have learned more about Cable when I was at Cisco (although I vaguely remember trolling EDCS for one of my projects so the CMTS acronym sounds familiar) but my 2nd attempt at using Cable provider is coming to an end, anyway. After 10 months of nearly blip-free service (not bad for $29.95 a month) with RCN, things have gone to hell in the last week. God I miss Speakeasy, but a year after swearing never to give another dime to AT&T/SBC, signed up for AT&T Yahoo DSL and even bought one of their little gateways so I don't have to muck with PPPoE (I hope) over the weekend just in case. I don't look forward to dealing with AT&T but what can you do? Maybe two shitty $29.95 Internet Services are better than a single decent $55/month service. And we'll actually have a land line for a change.

Although working as first line support for a consumer Internet provider (even if you are offshore) must suck, it was a surreal experience dealing with them for 3 hours last night, but I did learn a little bit about these mysterious cable modems

Toshiba Cable Modem Diagnostics Page

CM Info: MODEL PCX2500 ; HW_REV 9.2.3 ; SW_REV 1.0.14
MAC Address 00-00-39-xx-xx-xx   SerialNO. 3316470xxx   Version Capability D1.0

CmStatus:todEstablished  ServerBootState:waitingForTftp
sysUptime:0d:00h:02m:15s  CMTS MAC Address:00-30-B8-C6-EB-90
Last CmStatus - prior reset:

Power Level:
Received: -13.1 dBmV  Transmitted: 45.1 dBmV

Received SNR: 28.0 dB

Frequency:
Downstream: 735.000 MHz  Upstream: 33.000 MHz

User Set Parameter:
Polling Time: No Polling

So besides the high packet loss, on all my devices (2 routers and 2 different laptops) I kept getting leases for 192.168.100.2 (the tech support folks said it must be a configuration error on my end) which reminded me AirLink Cellular Modems we used in the SCADA Honeynet, where the modem itself has a DHCP server which temporarily assigns you a private address before forwarding your DHCP requests and then turning into bridge mode (or whatever) and then your interface finally gets a public address. So I unplugged the coax and sure enough I got a private address (192.168.100.1 was the router) did a quick TCP scan and found the web server up (see the display above) Didn't bother with UDP, would probably find TFTP and some other stuff. Of course one of the bizarre things was that at some point during all my troubleshooting I saw the 172.30.88.1 (the tech said this was also the Cable modem) attempting to ping a 208.x.x.x address. But I saw that on the Ethernet side? Something clearly must not have been well on the modem. And try as I could, I ended up hanging up, because there was obviously going to be no resolution.

Saint Barack of Iowa

So after reading the latest cover story on Obama it's starting to get creepy how how the conservative press (and perhaps even certain kinds of conservatives, which I am probably one) are fawning over Obama. What is up with this? Is this support real or is a cynical Anything But Hillary agenda based on the foregone conclusion that the Republicans have no chance in '08.

In my case, I've only voted in two Presidential elections since I was of age (1988 and 2004) and I voted for a Bush in both, but real soon now you are likely to see Obama '08 bumper stickers on both our blue Hondas and I might even contribute 25 bucks. Both would be a first for me and this might be the explanation of why someone who can't help find Rumsfeld and Cheney amusing (and not frightening) would even consider Obama

This is the Obama trick, and it explains why, despite his very liberal voting record in the Senate (and in the Illinois Senate before that), he is not viewed as a left-wing ideologue. When a student asks Obama for his views on the Second Amendment, he reminds his audience that he taught constitutional law at the University of Chicago and is thus familiar with the arguments regarding the right to bear arms. He acknowledges "a tradition of gun ownership in this country that can be respected," and says that his academic studies convinced him gun ownership "is an individual right and not just the right of a militia."

Or perhaps I'm just politically schizophrenic, since I certainly do not agree with his entire platform -- particularly on Iraq, which I'm much more in line with McCain. Of course my strange enthusiasm for Obama (although I'm ambivalent about his speech at Google but I did like his "not bubble sort" answer to a Google interview question) leads to some interesting discussions with my wife who "likes" Obama (and has actually read his memoir, I have not) but is willing to settle for Hillary because she thinks Obama is unelectable. And she thinks the "Republican machine would crush him." She also thinks McCain will be the Republican candidate. I wish that were the case (and I liked McCain in 2000) but it ain't gonna happen.

Python from the Browser Redux / How about JRuby?

I previously lamented PDP's descent into Web-2.0 Hacking Hell but I was pleased to see this update on his toolset for running Jython within a browser. Given the slow pace of Jython development and the lag behind CPython this is less cool than it was in 2006.

I wonder what it would take to do the same with JRuby

SCADA Compromise in 08? Bring it On!

So as yet another sign that SCADA is out of the closet, it made Hoff's 2008 [In]Security Predictions.

Be-Afraid-A of a SCADA compromise...the lunatics are running the asylum! Remember that leaked DHS "turn your generator into a roman candle" video that circulated a couple of months ago? Get ready to see the real thing on prime time news at 11. We've got decades of legacy controls just waiting for the wrong guy to flip the right switch. We just saw an "insider" of a major water utility do naughty things, imagine if someone really motivated popped some goofy pills and started playing Tetris with the power grid...imagine what all those little SCADA doodads are hooked to...

Call me cynical, but was has changed to make things worse in the last five years that would increase the liklihood of a "SCADA Compromise" (WTF that means). While things are probably different (meaning better, more rational) inside in large asset owners, in public forums the IT vs. Control System debate is as unealthy as it was back in 2003. Many control systems folks are still intent on making broad generalizations based their own bad experience with "IT".

What we're seeing here is a clash of technological focus and philosophies. IT departments don't do risk analysis the way Control Engineers do. Often things are replaced only because they're going to be out of date real soon now. Many throw software and servers at the wall until something useful sticks. I've heard estimates that up to 1/3 of all IT projects are regarded as failures. Few seem to see anything wrong with this. They take the risk anyway, knowing that the payoff can be very lucrative. Conversely, the control engineer tends to run a risk analysis on everything before making a move. They're very conservative and often don't change anything unless there are no parts for it any more and they've run out of spares. Their bosses are penny pinchers. They won't spend money to invest in anything that isn't broken.

and

And the fundamental difference between the IT department and the industrial control system engineer is that the engineers usually work at the application level. There is very little knowledge of the OS under the hood.

It pretty easy to come of with counterexamples for these. For every Areva admin that has no clue about Windows 2000 and TPKT/COTP, I'll bet there is an Oracle DBA that is equally clueless about Solaris 2.8 and TCP/IP.

But the more interesting question of about high visibility critical infrastructure compromise scenarios is why they aren't happening vs. how they could happen.

Conveniance Laptops/Operating Systems in the Enterprise

In most of the [security] teams I've been a part of in large companies, the first step an engineer would do upon receipt of new hardware (whether lease or purchase) was to immediately purge the box of the official corporate (always Windows) install and install your own OS (typically some Linux flavor, but perhaps BSD). More recently, you might purchase you own hardware (often a Mac) possibly in clear violation of the corporate security policy.

If you needed a rationale, it was because the standard IT image didn't allow you to do your job. Yeah you might be able to build and run libdnet/libpcap based apps on Windows, but why would you want to. You needed the right network, development, and security tools -- which in an of themselves are most definitely a violation (that is if the policy applied to you, since you were special, you were a security genius!). A fringe benefit was that you were free of the nasty IT-installed agents that suck the life out of laptops and the corporate spyware monitoring your every move. Oh yeah, and you also were more up to date on security packages than the IT build.

So with NAC and other endpoint control regimes designed to stamp out these rogue systems, there is the real possibility of controlling access to campus or remote access networks to "supported systems." What is a passive aggressive security guy to do? Sure, there are ways to subvert these controls, but you want to do the right thing, sort of. It is one thing to simply ignore policies that really weren't designed for you in the first place. It is another to actively thwart countermeasures. And then there is stuff in between like reverse engineering the "software token" so that a Linux user can enjoy the benefits of hardware token free authentication that the Windows users enjoyed.

And no this last example wasn't me (I'm not smart or motivated enough) but you know who you are!

Control System Security: Two Man Enter One Man Leave

Because I know a lot of the players and because its sort of quaint, I continue to follow the trials and tribulations of "SCADA Security" community I used to be part of while I was at Cisco and later, Digital Bond. I'd love to do the color commentary, but I'll just hit the highlights and let you make up your own mind. But believe me I am biting my tongue.

I'm assuming this whole spat started with Dale's Blog on a Wonderware NetDDE Vulnerability which led to Joe's Weiss Cybersecurity disclosures– the game everybody can play:

The way that the cybersecurity establishment has presented the Wonderware disclosure on the Digital Bond website clearly shows the lack of control system expertise in the cybersecurity “industry.” It IS an industry, and it is filled with people from IT security and cryptographic analysis backgrounds who have rarely, if ever, set foot in a control room for a process plant, refinery, or power plant.

It isn’t enough to be able to understand a vulnerability. It is every bit as important to understand the relative danger of the vulnerability IN CONTROL SYSTEMS. For example, the Wonderware disclosure isn’t very dangerous. Why not? Because the vulnerability disclosed is limited to a very small population of control systems using an outdated version of the Wonderware software. Like the ICONICS issue, revealing a vulnerability without a corresponding assessment of its impact is not only detrimental, but could be viewed (and certainly would be by Wonderware and ICONICS, for example) as unnecessarily injurious to their brands.

Which was followed by an exchange between Dale and Walt that almost didn't happen.

We have a serious problem in cybersecurity in control systems…we don’t have enough “cybersecurity experts” who know anything about process control or factory automation. We have a bunch of soi-disant experts who descended on control systems (remember, they’re the guys who thought every control system was “SCADA”?) because they saw a big market, and have been spreading FUD ever since. Recently, a Wonderware vulnerability has been disclosed, and the disclosure is making the rounds. Several months ago, an ICONICS vulnerability was disclosed, causing ICONICS significant distress. Why? Well in both cases, the vulnerability was, although accurately described, not dangerous.

Followed by Walt's attempt to trick Dale (and preach to the choir) on the Australian SCADA Mailing List

Since you have referenced the exchange Dale and I have had on my blog, I'm curious to hear YOUR answer to the question I kept asking Dale, and he kept not answering.

Here's what I asked, repeatedly. "Do you disagree with my premise: that in order to adequately advise people about cybersecurity in the process industries, significant familiarity with those industries and control systems is required?"

Dale didn't answer. I'd be delighted to hear others' answers.

What's behind all of this. Maybe we we are at a tipping point of some sorts. a power shift? All this talk of "the establishment." Perhaps we are at that point in martial arts movies where blood is dripping down over one of the fighter's eyes and he starts to get desperate and defensive. And then swing wildly. This is also before he cracks his neck with his hands and motions with both fingers to "bring it on." Before getting kicked in the head. And then the credits scroll.

Or perhaps it is just the same inane "IT vs. SCADA" conversation that has been raging for the past 5 years.

Yeah Jjakpae is an awesome Korean martial arts (taekwandoe) movie. A must see.

Using Hashes Like it is 1999

This week I picked up [what I thought would be] a quick logfile analysis task. Things started out great. I took the time to look at the logfile format and generalized 4-5 different messages (with appropriate regexes to get the data I needed) generated by the security device. Next I extended a basic "logrunner" class I wrote last month for analyzing the debug output from the Intel FreeBSD drivers (basically you do some sysctl's and it dumps some kernel messages to see counters missed, received packets--much better than netstat).

In my logrunner class, you basically can "attach" various simple regex matches and a symbol and you get a nice hash back with the values you want and it hides all the low-level details of matching or handling time stamps, etc. (HINT: If you are mucking with syslog files in Ruby and you are not using the Time API, you are a fool, but I digress).

After a few hours in I thought things were going fine, before some distractions kept me from working on it again until until the next afternoon (I overconfidently estimated this would take about 4 hours from start to finish), so I was in a rush. The initial desire to develop something be a more general purpose tool and that was designed properly was replaced with the brute force, quick hack, get-r-done approach.

I ended up iterating through the output hashes output by the logrunner tool to create more hashes some with the IP address as a key, others with a username as the key. And all of this pointed to at least another hash (or two) so I ended up with something like:

blah[blah][blah] = { 1 => { a => b, c => d }, 3 => { a=> q, d => z } }

This would have been a trivial task except there was no single session identifier (or even username or IP address) on each line that I could tie the various pieces of data together. Then I kept getting confused (and alternating between |k| and |k,v| with my Ruby blocks) it took my longer than I had hoped but I was done in about 7. I had the output I wanted. Went from a few hundred megs of logs to a nice Excel-friendly CSV file. And I thought I was done.

Until Friday afternoon, when I found it some additional data was needed. Extracting the data wasn't a problem (that was done in 5 minutes), but correlating it and getting the report format was. Should I add another hash? Redefine the hashes I'd written? Five o'clock on Friday (with restless hungry kids) is not a time for clarity of thought, but this morning I realized the Ruby I was written was as unreadable as the Perl I used to write back in the day.

Spending the afternoon driving out in the snow which turned to sleet which turned to rain finally beat some sense into me. I mapped out the data on paper (this time) and did right. Came up with 6 simple classes (2 base and 4 sub) to abstract away the hashes and ended up with less than 1/10th of the lines of code in the main loop and a 1/3 of the iterations. Nothing fancy, no Ruby foo, nothing that couldn't be done in Python. And the code is actually readable. The moral of the story? If you are using hashes 4-6 levels deep you have a problem. Stop, step away from the keyboard and come up with a cleaner design. Do it right the first time, you won't regret it. Because quick hacks have a funny way of running on systems for a long, long time.