Showing posts with label Protocols. Show all posts
Showing posts with label Protocols. Show all posts

Wednesday, January 14, 2009

Inside the Gmail Login Sequence (or, has anyone documented all the parameters and JSON response codes)

I generally don't like Wiley books, but near the end of a chapter on how Gmail works actually isn't that bad.

I'm sure there has to be more stuff like

/gmail?
ik=344af70c5d
&view=cv
&search=inbox
&th=101865c04ac2427f
&lvp=-1
&cvp=0
&zx=9m4966e44e98uu

As you can see, this the message ID of the message I clicked on.
But the others are mysterious at the moment. At this point in the
proceedings, alarms went off in my head.Why, I was thinking, is
the variable for message ID th—when that probably stands for thread.
So, I sent a few mails back and forth to create a thread, and loaded
the Inbox and the message back up

elsewhere dissecting the URL parameters, but I haven't found it apart from looking at the libgmail source, the constants file in particular. Has nobody documented this stuff or is google burying any documentation on reverse engineering Gmail?

It is sort of curious that the author is using tcpflow. Fine tool, but using an interceptor proxy like paros or something like firebug is a hell of a lot more efficient than sniffing.

Saturday, June 09, 2007

I Promise, Last JXTA Blog of the Weekend

Anyone that has ever tried to use Applet-based embedded device managers (just how many times a day does my office-mate curse Bluecoat?!!) knows that the "write once, run anywhere" credo of Java is not all its cracked up to be. But it does work sometimes. And distributing a [cross-platform] stack of apps that only depends on a JVM has its advantages on having to satisfy OS package dependencies and/or scripting language dependencies (ala GEMs or Python eggs)

To that end I've built a self-contained package for playing around with JXTA that not only includes the necessary .jar's for JXTA 2.4.1 (or at least for what I've been doing so far) but also includes the latest version (1.0.0RC) of JRuby. This a good companion to the JXTA Programmer Guide which I've been going through today.

Hopefully this makes it a little easier for folks to play around with this Open Source P2P API. As bloated and complex (and probably overkill for the mysterious new security project I have in mind) as JXTA is, it was far better documented than some of the alternatives such as or P2PS and there seem to be a lot more applications out there using JXTA.

Hello JXTA (JRuby Style)

So the original Java version was 42 lines

hello is a short wrapper shellscript to set the classpath to:

$HOME/java/jxta.jar:$HOME/java/bcprov-jdk14.jar:$HOME/java/jxtaext.jar:$HOME/java/log4j.jar

and ran jruby jxtahello.rb

which is

require 'java'

include_class "net.jxta.peergroup.PeerGroup"
include_class "net.jxta.peergroup.PeerGroupFactory"
include_class "net.jxta.exception.PeerGroupException"

npg = PeerGroupFactory.newNetPeerGroup()
puts "Hello from JXTA group " + npg.getPeerGroupName().to_s
puts " Group ID = " + npg.getPeerGroupID().to_s
puts " Peer name = " + npg.getPeerName().to_s
puts " Peer ID = " + npg.getPeerID().to_s
npg.stopApp()


And when executed prints pretty much the same:


franz-g4:~/Desktop/jxta/ruby mdfranz$ ./hello
INFO 2007-06-09 15:49:55,134 NullConfigurator:::132 JXTA_HOME = file:/Users/mdfranz/Desktop/jxta/ruby/.jxta/
INFO 2007-06-09 15:49:55,305 NullConfigurator::adjustLog4JPriority:316 Log4J [user default] requested, not adjusting logging priority
INFO 2007-06-09 15:49:56,681 NullConfigurator::adjustLog4JPriority:316 Log4J [user default] requested, not adjusting logging priority

Hello from JXTA group NetPeerGroup
Group ID = urn:jxta:jxta-NetGroup
Peer name = bobo
Peer ID = urn:jxta:uuid-59616261646162614A7874615032503399A33A2EB7BB47E5BA108F6B6D2BDF3203


This probably the shortest JXTA code shows the real power of JRuby in prototyping and learning JAVA APIs.

BTW, all the JXTA examples are up at http://www.threatmind.net/jxta

Hello JXTA


So I downloaded Netbeans 6.0 Milestone 9 this morning and after briefly playing around with it is new Ruby project capabilities (quite nice) I finally got a JXTA 2.4.1 Hello World app built and running. I'll probably document this better, but the main thing was getting the right libraries (.jar's) added to the Netbeans project:
  • jxta.jar jxtaext.jar
  • log4j.jar
  • bcprov-jdk14.jar
All of these were available (replace link here when JXTA site is back up) and the Netbeans project source is available here if you are interested.

franz-macbook:~/dist mdfranz$ java -jar JxtaSample.jar
Starting JXTA ....
INFO 2007-06-09 11:27:52,358 NullConfigurator:::132> JXTA_HOME = file:/Users/mdfranz/dist/.jxta/ WARN 2007-06-09 11:27:52,360 NullConfigurator::load:233> Platform Config not found : file:/Users/mdfranz/dist/.jxta/PlatformConfig INFO 2007-06-09 11:27:52,361 NullConfigurator::adjustLog4JPriority:308 Log4J logging preference not set, using defaults INFO 2007-06-09 11:27:52,362 AutomaticConfigurator::buildPlatformConfig:159> New PlatformConfig Advertisement INFO 2007-06-09 11:27:52,421 AutomaticConfigurator::buildPlatformConfig:223 HTTP advertisement missing, making a new one. INFO 2007-06-09 11:27:52,425 AutomaticConfigurator::buildPlatformConfig:305 TCP advertisement missing, making a new one. INFO 2007-06-09 11:27:52,428 AutomaticConfigurator::buildPlatformConfig:365 Relay Config advertisement missing, making a new one. INFO 2007-06-09 11:27:52,433 AutomaticConfigurator::buildPlatformConfig:405 Rdv Config advertisement missing, making a new one. INFO 2007-06-09 11:27:52,436 AutomaticConfigurator::buildPlatformConfig:430 Proxy config advertisement missing, making a new one. INFO 2007-06-09 11:27:52,438 AutomaticConfigurator::buildPlatformConfig:457 PSE Config advertisement missing, making a new one. INFO 2007-06-09 11:28:07,012 NullConfigurator::adjustLog4JPriority:316 Log4J [user default] requested, not adjusting logging priority INFO 2007-06-09 11:28:09,909 NullConfigurator::adjustLog4JPriority:316 Log4J [user default] requested, not adjusting logging priority

Hello from JXTA group
NetPeerGroup
Group ID = urn:jxta:jxta-NetGroup Peer name = moonface Peer ID = urn:jxta:uuid-59616261646162614A787461503250334C06A0908D664660BDEFCD8DF2E958E603 Good Bye ....

Stay tuned for more blogs on JXTA. And if you are wondering why the sudden interested in Juxta (that is how it is pronounced) I 'm strongly considering using it in a new Open Source Network Security Project I'll be launching this summer.

Monday, April 30, 2007

just rtfm ng_netflow(4)

On FreeBSD 6.2, to use the built in kernel netflow export capability, as long as the ng_ether is loaded, the example at the bottom of ng_netflow(4) should work just fine. However, I did put up a a wiki page on the topic, after googling all over. It will be interesting to see what sort of performance is possible with it

Compiling Flow-tools on OSX/gcc4

Recent (at least 4.0.x) versions of gcc don't like flow-tools but there is a patch

franz-g4:~/Desktop/flow-tools-0.68 mdfranz$ ./configure --prefix=/opt/local
checking for a BSD-compatible install... /usr/bin/install -c


SNIP

franz-g4:~/Desktop/flow-tools-0.68 mdfranz$ make
Making all in lib
make all-am
source='ftio.c' object='ftio.o' libtool=no \
depfile='.deps/ftio.Po' tmpdepfile='.deps/ftio.TPo' \
depmode=gcc3 /bin/sh ../depcomp \
gcc -I. -I./lib -I. -I. -I. -g -Wall -g -Wall -c `test -f 'ftio.c' || echo './'`ftio.c
ftio.c: In function 'readn':
ftio.c:2270: error: address of register variable 'ptr' requested
ftio.c:2270: warning: target of assignment not really an lvalue; this will be a hard error in the future
ftio.c:2270: confused by earlier errors, bailing out
make[2]: *** [ftio.o] Error 1
make[1]: *** [all] Error 2
make: *** [all-recursive] Error 1
franz-g4:~/Desktop/flow-tools-0.68 mdfranz$ patch -p1 <>

Sunday, April 29, 2007

Painless Non-Enterprise Netflow

Tonight I released another ubuntutrinux-core snaphot that include fprobe and a few tools from flow-tools. I'll spare the introduction to Netflow except to comment why this might useful for Linux/Trinux (its obvious for routers!) as opposed to other network monitoring tools.

In terms of data your are getting about what you would get with a port logger such as ippl or other port listeners that log connections from hosts

root@gx620:/tmp# flow-cat biglast | flow-print | head
srcIP dstIP prot srcPort dstPort octets packets
24.136.0.111 239.255.255.250 2 0 0 32 1
24.136.0.189 239.255.255.250 2 0 0 32 1
82.211.81.145 24.136.x.y 17 123 123 76 1
24.136.2.30 239.255.255.250 2 0 0 32 1
24.136.2.67 224.0.0.251 2 0 0 32 1
24.136.2.67 239.255.255.253 2 0 0 32 1
10.48.120.1 224.0.0.1 2 0 0 28 1
24.136.0.163 239.255.67.250 2 0 0 32 1
24.136.19.48 224.0.0.253 2 0 0 32 1

Notice this is mostly multicast cruft on RCN with the exception of NTP traffic to the Ubuntu time source. But big deal. Some of you may remember (back in the day!) a NSWC tool called SHADOW (where Northcutt and Irwin made their claim to fame) that was basically a collection of Perl scripts that managed tcpdump file capture and viewing through a web interface.

Well flow-tools allows you to a lot of the same stuff with much less overhead and all from the command-line

$ flow-cat biglast | flow-stat -f5 -S 1 | head -25

# Args: flow-stat -f5 -S 1

#
#
# port flows octets packets
#
80 2464 12632818 162315
53 1099 214515 3053
1026 414 250208 710
32768 405 131967 766
1027 314 155033 314
123 186 14136 186
443 177 333745 3020
7 175 19075 175
5222 141 36842 370
3408 120 15892 220

Basically you "flow-cat" the saved file to a number of different tools, flow-stat being the most useful for me. Not terribly surprising HTTP is at the top nor the 1026 to my firewall. Damn cable.

$ flow-cat biglast | flow-stat

#
# Fields: Total
# Symbols: Disabled
# Sorting: None
# Name: Overall Summary
#
# Args: flow-stat
#
Total Flows : 13168
Total Octets : 380416991
Total Packets : 517136
Total Time (1/1000 secs) (flows): 222894382
Duration of data (realtime) : 34320
Duration of data (1/1000 secs) : 364934
Average flow time (1/1000 secs) : 16926.9733
Average packet size (octets) : 735.6227
Average flow size (octets) : 28889.5043
Average packets per flow : 39.2722
Average flows / second (flow) : 36.1758
Average flows / second (real) : 0.3837
Average Kbits / second (flow) : 8360.8132
Average Kbits / second (real) : 88.6753

After concatenating all this data into a single file, it only took about about 800k for about 10 hours of traffic.

And how did I kick all this off?

First I ran fprobe on Trinux and made sure it was working by testing it out with EHNT
which is the quickest way (it took me a while to wander through the flow-tools manpages, and the ubuntu startup script (in /etc/init.d) for capturing flows didn't work. I'm using pcap to get this but there is a version of fprobe that can generate flow from iptables.

#fprobe -u nobody collector-ip:collector port

BTW, netflow uses UDP. You can sniff to make sure the flow updates are being sent. And then on the server (you'll want to be more restrictive on the local and remote ports, the 0's)

#flow-capture -w /raid/flows/ 0/0/4444 -S20

which creates the directory hierarchy like:

root@gx620:/raid/flows/2007# ls -alR | less

.:
total 0
drwxr-xr-x 3 root root 72 2007-04-29 12:04 .
drwxr-xr-x 3 root root 72 2007-04-29 12:04 ..
drwxr-xr-x 3 root root 80 2007-04-29 12:04 2007-04

./2007-04:
total 2
drwxr-xr-x 3 root root 80 2007-04-29 12:04 .
drwxr-xr-x 3 root root 72 2007-04-29 12:04 ..
drwxr-xr-x 2 root root 2208 2007-04-29 22:15 2007-04-29

./2007-04/2007-04-29:
total 326
drwxr-xr-x 2 root root 2208 2007-04-29 22:15 .
drwxr-xr-x 3 root root 80 2007-04-29 12:04 ..
-rw-r--r-- 1 root root 1145 2007-04-29 12:15 ft-v05.2007-04-29.121249-0500
-rw-r--r-- 1 root root 4500 2007-04-29 12:30 ft-v05.2007-04-29.121904-0500
-rw-r--r-- 1 root root 2153 2007-04-29 12:45 ft-v05.2007-04-29.123001-0500


Of course there are tons more options (the -S20 just says write a status report to syslog every 20 minutes like:

Apr 29 22:20:00 localhost flow-capture[27718]: STAT: now=1177903200 startup=1177867131 src_ip=192.168.100.1 dst_ip=192.168.169.162 d_ver=5 pkts=2918 flows=13828 lost=1 reset=0 filter_drops=0

Saturday, March 17, 2007

The Quest for JSON-RPC [In]Security

In Web 2.0-land, things really seem to be heating up with the not so interesting (or, in and of itself no more interesting than XML) JSON and the more interesting (as a lightweight replacement for XML-RPC or XMLHttpRequest used in AJAX apps) JSON-RPC to the point hat I've started tracking things on a wiki page.

To break down, here is what is sent on the wire. JSON-RPC is quite similar to XML-RPC except instead of sending and XML message via an HTTP POST you use JSON, such as:

POST http://jsolait.net/services/test.jsonrpc HTTP/1.1
Host: jsolait.net
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2
Accept: text/xml, application/xml, application/xhtml+xml, text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Proxy-Connection: keep-alive
Content-Type: text/plain
Referer: http://jsolait.net/wiki/examples/jsonrpc/tester
Content-Length: 99
Pragma: no-cache
Cache-Control: no-cache

{"id":"httpReq", "method":"args2Array", "params":["hello", {"a": 1234, "b": "five"}, [6, 7, 8, 9]]}HTTP/1.0 200 OK


And obviously the server will respond with something like

HTTP/1.0 200 OK
Date: Wed, 28 Feb 2007 22:32:17 GMT
Server: Apache/2.0.55 (Debian) DAV/2 SVN/1.3.2 mod_python/3.2.10 Python/2.4.4
Content-Type: text/plain; charset=UTF-8
Proxy-Connection: close

{"id":"httpReq", "result":["hello", {"a":1234, "b":"five"}, [6, 7, 8, 9]], "error":null}


In terms of security, so far I've only found one presentation at CCC somewhat related to JSON-RPC security and I didn't find it terribly interesting. This article by Metaeye Security Group was a bit better, but had nothing to do with RPC, only JSON. Then there is the August 2006 Working Draft we see the following as a "non-goal:"

JSON-RPC does not address security, correlation, reliability, sessions management, state management and many other features that may be found in other RPC protocols or distributed systems. Developers are instead encouraged to employ features of HTTP and related Internet technologies that can offer some of these features in an equally compelling, robust and satisfying manner. Some features may be defined as extensions by other specifications.

Now it is tempting to mock this sentiment as the normal "security never makes it into 1.0 argument" (or think TCP/IP protocols that say, hey IPSEC will handle all security -- or our web application is secure if it goes over HTTPS) but there is actually some sense to their logic.

In an ideal world, it would definitely be better to rely on the web [application] server, web framework (or whatever) to do session management, authorization, authentication, and definitely encryption. Why reinvent the wheel, badly? However in practice this normally means form based authentication plus a strong session ID (whether Java, PHP, ASP, or whatever) all over SSL. But what if I don't want that or can't have that, for example if I'm not running the client within a browser--and yes I could definitely see some use cases for non-browser based JSON-RPC (as an alternative to SOAP or XML-RPC) and given the number of implementations out there some other folks obviously do as well.

Since HTTP Digest Authentication is not a pervasive or a robust as we would like (a case in point: try to find a Python/Ruby/Java based Web Server to expose XML-RPC web services that supports RFC 2617--actually Ruby seems the best choice) you are stuck with Basic Authentication over HTTPS. What I wish is that there were a lightweight authentication and authorization mechanism comparable to what Amazon does with S3 and its REST APIs.

If you are doing "RPC stuff" over HTTP, you need more granular security mechanisms than can be provided by HTTP[S] or the appserver.

Right now the big boys in web service land (Amazon, Google, Yahoo) don't seem to be exposing their APIs with JSON-RPC yet (although they do expose data in JSON) similar to the way that SOAP and REST are currently used, but it is probably only a matter of time. The other aspect of JSON-RPC that is sort of interesting is that several of the JSON-APIs are borrowing heavily from their XML-RPC counterparts, which have had their share of security issues--at least in PHP, Python, and Ruby.


Update
I just ran across some fresh blogs (last week) on JSON (In)Security that are relevant:
Securing your JSON and JSON is not as safe as people think it is.

I will continue to add stuff like this to my Json Wiki Page as I find it.

Saturday, September 23, 2006

Pcapper: cool non-C based PCAP

Jeff Dell from Activeworx pointed out that Pcapper is a great cross-platform way to decode captures from tcpdump and friends in response to my question on my work blog.

franz-g4:~/dev/scada-nasl/dnp3 mdfranz$ pcapper.py -v dnp.1

Packet:
asctime 2006-09-19 11:53:36.711335
caplen 54
destination 00:c0:4f:0c:7b:1d
ether_type 8
pktlen 54
source 00:0c:29:cf:38:82
tv_sec 1158684816
tv_usec 711335
IP datagram:
checksum 0
destination_address 192.168.169.140
flags 2
fragment_offset 0
header_length 5
id 18544
protocol 6
source_address 192.168.169.11
tos 0
total_len 40
ttl 128
version 4
TCP datagram:
ack 1
ack_number 3660227963
checksum 54275
data_offset 5
destination_port 20000
fin 0
flags 16
psh 0
rst 0
sequence_number 2451677669
source_port 3016
syn 0
unused 320
urg 0
urgent_pointer 0
window 17469
Payload:
Dump source has length: 0


Run time 0.903 seconds
Processed 108 packets
Processed 10376 bytes
Processing speed 119 packets per second
Processing speed 11 kbytes per second

End of program

Works on OSX, haven't tried it on anything else. Next is to use the lib in a new protocol reverse engineering tool I started just today.

Stay tuned.

Monday, September 04, 2006

Fuzzing on Rails

Only 10 minutes in playing around with a new web fuzzer I wrote vs. an unnamed Rails app, I'm getting stack traces. Probably not the end of the world, but not great either. I'm wondering what Rails provides out of the box for validation and how it compares to what other frameworks such as Struts.


undefined method `include?' for nil:NilClass
./script/../config/../vendor/rails/actionpack/lib/action_controller/cgi_ext/cgi_methods.rb:49:in `parse_request_parameters'
./script/../config/../vendor/rails/actionpack/lib/action_controller/cgi_ext/cgi_methods.rb:47:in `each'
./script/../config/../vendor/rails/actionpack/lib/action_controller/cgi_ext/cgi_methods.rb:47:in `parse_request_parameters'
./script/../config/../vendor/rails/actionpack/lib/action_controller/cgi_process.rb:70:in `request_parameters'
./script/../config/../vendor/rails/actionpack/lib/action_controller/request.rb:12:in `parameters'
./script/../config/../vendor/rails/actionpack/lib/action_controller/session_management.rb:122:in `set_session_options_without_components'
./script/../config/../vendor/rails/actionpack/lib/action_controller/components.rb:178:in `set_session_options'
./script/../config/../vendor/rails/actionpack/lib/action_controller/session_management.rb:116:in `process'
./script/../config/../vendor/rails/railties/lib/dispatcher.rb:38:in `dispatch'
./script/../config/../vendor/rails/railties/lib/webrick_server.rb:115:in `handle_dispatch'
./script/../config/../vendor/rails/railties/lib/webrick_server.rb:81:in `service'
/usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service'
/usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run'
/usr/lib/ruby/1.8/webrick/server.rb:172:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:161:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:161:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:95:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `each'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:82:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:82:in `start'
./script/../config/../vendor/rails/railties/lib/webrick_server.rb:67:in `dispatch'
./script/../config/../vendor/rails/railties/lib/commands/servers/webrick.rb:59
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:21:in `require__'
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:21:in `require'
./script/../config/../vendor/rails/activesupport/lib/active_support/dependencies.rb:136:in `require'
./script/../config/../vendor/rails/railties/lib/commands/server.rb:30
script/server:3:in `require'
script/server:3

Thursday, July 27, 2006

IKE DoS Attacks are Like So Year 2000

The folks over at Liquidmatrix are in their Cisco Conspiracy Theory Mode again as a result of a Cisco IKE Advisory in VPN3k, IOS, and PIX causing me to lapse lose my normally quite-restrained blogging persona and lapse into bouts of inside jokes.

Must be a slow news week. What next? "Telnet Weak Authentication Vulnerability in IOS" leads to Internet collapse?

In hindsight, it's amazing (I guess the ask for forgiveness not for permission rule was still effect back at Cisco in 2001) that I was even able to present on this stuff (on those ugly templates no doubt!), but if you look at slides 56-72 in a CanSecWest Prezo I did back in April '01 you'll see treatment on DoS issues in IKE although these had been discussed earlier much within the IETF. And if I am talking about it, it can't be rocket science.

And of course I can't go without mentioning the IKE DoS king and the Son of IKE Dos King. Perhaps Venkat (or Earl) will be able to get the Director/VP level approval to blog on this issue. We'll see.

Don't get me wrong developing exploits for silly protocol design errors and saying the sky is falling is all good fun, but let's lose to tone of exasperation.

Wednesday, March 01, 2006

ifsh: interactive fuzzing shell

Well my replay tool is now morphing into an interactive network fuzzing shell!

mdfranz@franz-d610:~/dev/realtcpreplay$ ./clientplay.py 172.16.126.132 102 follo
w.pickle pause
0>

Sending:22 bytes

Received: 14 bytes
1>

Sending:191 bytes

Received: 147 bytes
2> help


> rb filename - read binary file and use for next message
> ra filename - read ascii file (00 ff 00 ff) for next message
> testcase directory - specify a directory of testcases (binary file to use)
> fuzzload [min,max]
> fuzzcount n - how many fuzzloads to generate
> fuzzheader n - how deep in payloads to overloay fuzzloads
> send
> continue
> setpoint
> sh[ow] exchange - dump all messages sent/received so far
> sh[how] last-sent
> sh[how] last-rec


2>

Sending:36 bytes

Received: 32 bytes
3> fuzzcount 100
FUZZCOUNT now set to 100
3> fuzzload
Generating fuzzloads: Generating fuzzloads: . . . . .
3> go

*** Reset by peer ***

Tuesday, January 31, 2006

Now I'm even sicker of Fuzzing!

Someone does a slashdot post on muSecurity (actually they call them a "cracker attack emulator"-which, I must say sounds very cool), and along down in the thread, somebody mentions my Fuzzing page and I end up getting close to 4000 hits today.

What is the world coming to?