Friday, July 06, 2007

Minor Rant on Fuzzing


Bejtlich's Pre Review triggered some painful memories. Back when I was a teacher, I ran across two types of really bad papers: those that were so bad they were funny and those are those that are so bad they made you angry, really angry -- because they were wasting your time.

Last year, I had the misfortune of serving as a technical reviewer for the new Addison Wesley Fuzzing book. The manuscripts I read clearly fell into the into the second category. It just wasn't worth the $750 (or whatever it was they were going to pay me) to provide feedback and fill out the little forms, so I eventually quit responding to emails from the editors and a deleted all the copies of manuscripts I had in my possession. Now, to be honest, it wasn't just that the manuscripts were a lost cause that I gave up the endeavor. I did have a lot on my plate: trying to get my house on the market in Austin, finish up some projects for my last job, and figure out where the hell I was going to live in Chicago--and move two kids and two dogs cross country, without losing any of them in Oklahoma. Which almost happened.

But if I thought the book had any hope of being useful I probably would have found the time. Unfortunately, from the table of contents, it doesn't look like they fixed the book's structural flaws. Not only the did conceptual sequence not make much sense to me, the audience and purpose were always a mystery. But maye that was maybe because I didn't ever see the first section. I was never sure if it existed? Was it be written last? There was no clear driving purpose linking the content. Was the book targetting professional application security teams (Software Security is an example of one of these, a very useful book) or just a quick way for 3rd rate independent researchers to find a bug or two. It appeared to be the latter, which to me was a pointless exercise. Why invest time in writing (let along reading) a book on the topic of vulnerability testing that does not go beyond what you could find by downloading tools from Packet Storm.

Lastly, I wonder if they cleaned up annoying colloquial writing style that sounded like a transcript of a bad Black Hat talk (except what I assume were Pedram's Amini's chapters, the were fairly well written and had some original content as well) but I guess I'll never know. I'd love to hear these problems were fixed, but I'm certainly not going to spend good money on finding out myself.

2 comments:

dre said...

Well, if you read Bejtlich's blog entry on the Fuzzing Book review, did you also happen to read my comment? I tried to cover the issue that you are both complaining about.

Yes, the book was difficult and unstructured - but I think there were a bunch of great takeaways. Probably more than any other book I've read lately on software testing, let alone security. I'm using the concepts that I explained and taking them to a whole new level. It will become a talk (already outlined) that I'm going to give at the next Chicago OWASP. I'm meeting with the guys that run it next week to discuss, so check the OWASP Chicago website later next week for more details.

I also just went / am going through a painful process of moving to Chicago. It would be neat to meet you, possibly at the Chicago 2600, ChiSec, or other event. Did you hear about ChicagoCon yet?

Matt Franz said...

dre,

I did read it (and am not unfamiliar with the topic, actually quite sick of it), but I don't see how that is a defense of their work :)

But I'll admit the style and structural issues probably prevented me from seeing some good content. Maybe they fixed some of the problems. The DDJ looks better than what I saw last Fall. But given the caliber of folks involved in the effort and the that this was first book on the topic I expected a whole lot more. Of course, Pedram's chapter's were miles above the stuff that Sutton wrote, which was most of what I read, was sent first, didn't find much new/interesting in, and found so objectionable.

I made it to one Chisec back when it was really cold (and got stuck on the L for over an hour) and haven't been back. Given that I work in the far North suburbs making downtown is pretty difficult.