DOWNLOAD pkviz HERE (For Mac / OS X users only):
- 64 bit: http://jackwhitsitt.com/pkviz.zip
- 32 bit: http://jackwhitsitt.com/pkviz_32.zip
- PPC: http://jackwhitsitt.com/pkviz_ppc.zip (this is untested, so someone let me know if it works?)
Sample data can be found HERE: http://jackwhitsitt.com/pkvizsampledata.cap
Free Source Code: http://sintixerr.wordpress.com/2010/02/20/pkviz-packet-visualization-animator-source-code-available/
This is “beta” quality code, so please let me know if there are issues I need to fix or features I should add. (Note: Right now, I suggest using 3000 packets or less in one capture file, but it depends on your machine. Use a smaller animation depth if youre having performance probs.)
PKVIZ: INTRODUCTION AND HELP
Pkviz is a tool for plotting and cycling through and animating a series of network packets captured by tcpdump. What makes it unique is that the packets’ structure is visualized, not any labels and not time itself. Pkviz takes each byte in a packet and plots it out end-to-end, left-to-right, from the first byte to the last. How high the dot gets plotted depends on the value of the byte: Bytes with a value of 0 are at the bottom and those which are 255 (0xff) – the maximum value of a byte – get plotted at the top.
This might not be interesting for one packet, but that changes when you start looking at thousands of packets. Pkviz can cycle through thousands of packets in the set so you can see what happened on the wire.
If the protocol or content is one you’re unfamiliar with, pkviz begins to allow you to fuzz the protocol. What are the value ranges for a given position in the packet? Is it a human-readable/ascii protocol, or only machine-readable? Is free form text common? Are there any patterns in the protocol that you were unaware of? Etc.
In addition to helping to fuzz unknown protocols, it can be used to find outliers. If you watch 300 packets of the same type go by and suddenly you see one with a byte value out of the range the other 300 had, maybe something unusual or interesting is going on?
If nothing else, though, it also looks kind of cool. Check out the screen-shot and movie below:
Minimum Use:
Pkviz is going to work from an ascii-hex formatted file from tcpdump. This means, you want to run tcpdump (live or through an existing pcap) with, at a minimum:
“tcpdump –X | somefile”
You can add any other parameters to tcpdump you need to (I usually use –s0), but the –X is needed to make sure pkviz can read it. I’ve included a sample file to test with (above). When creating your own file, edit it ahead of time to trim out any blank lines at the end.
Once you have your file, open pkviz and load it (button located under general). The button will stay a darker grey until the file has been processed. Bear with it – it takes about 30 seconds for 2000ish packets on my machine. Right now you have to load a file every time you open pkviz. The capability to save the data exists, but isn’t stable yet so I’ve disabled it.
Once loaded, click the Render button and it still start plotting.
Advanced Use: Movement
- Pause Button. This will stop the automatic flow of the packets, or restart it. Some features require the flow to be paused before they will work. When pkviz reaches the last packet, the flow is automatically paused so you can go back and look at the stream.
- Forward/Back: This will step you through the packet stream one packet at a time. Bear in mind that the animation depth will affect how this looks. If you have a depth of 1 packet, every time you click, a new packet will appear and the old go away. If you have a depth of 50 and click forward, a new packet will appear and the other 49 will fade a little bit, keeping a sliding window of 50 packets on screen at any given time – as with happens with the animation.
- Jump to Packet. While the animation is paused, you can use this to move the current packet to any packet in the stream that has already been displayed by the main animation (You cant jump forward….yet). If you want to jump between all packets, wait until the animation has finished and then….jump around”. You can always go backward, though.
Advanced Use: Animation
- Animation Depth. Pkviz plots 1-n packets on screen at any given time. Animation Depth defines “n” and defaults to 30. The sliding window has based on “current packet” and n packets –previous- to it in time. The further back in the window a packet is, the dimmer it is. Adjusting this depth window allows you to see different patterns over time, focus on rangers of activity or individual packets, etc. Play with different settings here – you see different types of activity.
- Speed. This adjusts the time delay between when pkviz plots one packet and when it plots the next. How fast or slow this is will depend on a) Your workstation speed and b) The current animation depth. If you are plotting a 50 or 60 packets window, it will be noticeably slower than 30. If you’re animating a 1-packet window, it will race through. You will see different patterns depending on how fast you go – so play with this as well.
Advance Use: Analysis
- Show Packet. Ok, so maybe you want to see the underlying data of the most current packet? Clicking “Show Packet” (I recommend pausing first, but you don’t have to) will open up a new window containing an ASCII/HEX representation of the packet. There are no scroll bars, so you will have to arrow-key down to get to the end of larger packets, but you CAN copy/paste the data to a text editor to see it.
- Expand Headers. By default, pkviz plots packets based on a linear representation of byte position. However, for the many times you’re going to be interested in the TCP, IP, or other headers, this is a problem: Those headers are all clumped together at the front and hard to see. If you check “Expand Headers”, pkviz will plot the byte positions on a logarithmic scale. This means that roughly the entire left-hand side of your graph will be headers, and the other 95ish% of the packet will be compressed visually. NOTE: This box can only be checked PRIOR to beginning the render, so load the tcpdump file, then check the box, then click render.







12 comments
Comments feed for this article
November 28, 2009 at 2:07 pm
Packet Visualizer/Animator DONE! (ish) and Posted « Jack Whitsitt: Art and Security in Washington, DC
[...] Packet Visualizer/Animator [...]
December 1, 2009 at 5:24 pm
Julien.T
It seems an interesting application but i can’t use it.
With my MacOS 10.5.8/x86, it says “Impossible to open this application with this release of MacOS X”
so do i need 10.6 ? x86 only or x86/ppc ?
thanks for your great work.
December 1, 2009 at 5:32 pm
Jack Whitsitt
Julien, did you try the 32bit version? Let me know if that works.
Either way, you pointed out an omission – I dont have a PPC version up there. I’ll try and fix that tonight and you can try it as well.
If it’s an OS X 10.5 vs 10.6 issue, that might be harder to fix, but I’ll look at that, too.
December 2, 2009 at 2:58 pm
How the Matrix Really Looks: New Data Visualization Art | Juxt Art
[...] http://sintixerr.wordpress.com/pkviz-packet-visualizer-and-animator/ [...]
December 8, 2009 at 1:19 am
mike
no work on 10.5.8, neither 32/64/ppc version, same error as reported
December 19, 2009 at 1:54 pm
Julien.T
have update to 10.6.2 and it starts ok. Sadly, after it doesn’t seem to be able to load the pcap file i try :(
Is there a recommanded limit size to avoid to load.
I try a 8MB tcpdump pcap file which makes about 34MB in ascii/text.
I let it run about 1 hour without result.
thanks
December 19, 2009 at 1:59 pm
Jack Whitsitt
Julien: try 2GB (the file size after a tcpdump -X) I know that size works on a Mac Mini.
Mike: Working on an update for pre-10.6 versions of OS X…some incompatibilities.
February 20, 2010 at 3:47 pm
Pkviz Packet Visualization Animator Source Code Available « Jack Whitsitt: Art and Security in Washington, DC
[...] Pkviz: Packet Visualizer/Animator [...]
February 22, 2010 at 5:21 pm
dfaulkner
Ever seen the Spinning Cube of Potential Doom?
http://www.nersc.gov/nusers/security/TheSpinningCube.php
February 22, 2010 at 5:58 pm
Jack Whitsitt
I’ve seen it and, though I like Bro, Im not a fan of the idea. It looks cool and provides some interesting insights, but my personal experience is that using perspective to simulate 3D with data that isn’t natively arranged like that makes analysis harder than 2D. I get a lot more out of putting two 2D scatterplots side by side with linked dimensions than anything 3D does. It also makes data more easily selectable if you’re dealing with interactive analytics tools like Advizor or Tableau.
With regard to pkviz vs the cube, the cube displays dimensions that are human-relevant (like IP, port, etc) whereas pkviz is displaying bytes within individual packets as they’re physically laid out. These are two completely different perspectives and offer different insights :)
February 24, 2010 at 2:05 pm
Robert
Any chance that this will ever load PCAP (as opposed to tcpdump -X) output? I have high-performance capture tools that our organization uses to capture data on high-performance links and doing a pcap grab followed by a tcpdump -r -X just to convert is a pain.
February 24, 2010 at 2:08 pm
Jack Whitsitt
Yes, I agree, it should do pcap format instead of the -X. The input parser was originally written to solve a particular problem I had and pkviz grew around it. Next step is to write a pcap parser. I’ll definitely post an update when that happens.
The code is also open source and the input is pretty well segmented from the visualization code-wise, so if someone else has a pcap parser they want to hack on before I get to it (I’ve not written one before), they’re more than welcome to.