2008-09-03

Over-visualization fun

I'm a very big - nay, a huge t-shirt fan. I'll admit, I even subscribe to a t-shirt blog. If I attended meetings, it'd be an illness.

Threadless is a tee site I'm particularly fond of. While browsing their seemingly bottomless vault of shirts for sale, I came across this one. It hit home for a number of reasons.

Over the past few weeks I've struggled with the problem of visualizing a massive amount of data relating to some security incidents. This has proven a worthy endeavor not only in illustrating causality that isn't apparent in the raw data itself, but also in communicating to management various parts of the "story," letting them draw their own conclusions. I'll hopefully get to writing about a couple of techniques (no data, naturally) that have been particularly helpful in the coming weeks.

In a number of cases, the approaches I've taken have failed, most due to "over-dimensionality;" trying to cram too many variables into the diagram. What resulted was cool, but required far too much explanation - much like the visualization in this picture. The data itself in this case is likely meaningless, but it's a good example of what can result when analysts are overly ambitious in attempting to communicate findings. It's easy to do. When we understand all of the data we have, thanks to many long hours of study and analysis, we feel every detail is important because we understand its contribution. But in telling the story, guiding readers to a conclusion, or illustrating causality, many times it is necessary to gloss over detail that can be spoken to or revealed if additional questions arise.

I've found that studying Tufte's literature has been a great help in improving my skills in visualization throughout the course of this calendar year, and while I appreciated this skill before, I now realize how critical it is to this profession. I'd encourage everyone in InfoSec to find a way to sharpen their skills in data visualization. It will pay dividends in your career you didn't expect.

With special thanks to my boss for initially inspiring me to investigate this topic more thoroughly.

2 comments:

Raffy said...

If you haven't looked at secviz.org you should definitely look there. People submit examples about visualization of security data there.
In addition, you might be interested in the book: Applied Security Visualization. Among many other security visualization topics, it talks about how to visualize highly-dimensional data sets by, for example, using parallel coordinates.

Michael Cloppert said...

Raffy,

Thanks for the feedback. I've been interested in your book, but haven't had the time to look into it. Honestly, I was disappointed by my last purchase, Security Data Visualization. I have hopes yours will be a bit more helpful.

I'd second your suggestion to check out secviz.org to anyone else reading this. While I only discovered it recently, I've read every entry that's shown up in my feed reader ever since adding it :-)

-Mike