Why my Twitter Feed is Hilarious

...or, the yes-huh, nut-uh of "cyberwar":

Security Academia: Stop Using Worthless Data

I have a new litmus test that I use to help me vet the many intrusion detection related academic papers that come across my desk. I call it the "relevant data test." If your approach does not study relevant data, I will not read it. You may indeed have found a new way to leverage Hidden Markov Models in some neat heuristic, layered approach. I do not care. Novel or precise as your approach may be, the applicability of it is predicated upon the relevancy of your data. You may as well have found a new way to model the spotting of a banana as it ripens, if your data has nothing to do with intrusions in 2010.

It's time to wake up, folks. A 10-year-old data set for intrusion detection is utterly worthless, as your conclusions will be if you use it. I will never again read further than "benchmark KDD '99 intrusion data set." There is no faster way to communicate to an informed audience that you just don't understand intrusions than by analyzing data that is this old. Such attacks are generations behind those that modern network defenders face today. Understand this: you are solving the problems exemplified by your data set. If your data is 11 years old, so is your problem, and your solution is only as effective as that problem is relevant. Few, if any, attacks from 1999 are relevant today.

Make no mistake about it, I understand the researcher's lament! There is no modern pre-classified data set like those relics of careers gone by. Finding a good corpus is excruciatingly difficult. But in legitimate, scientific, empirical studies, this is absolutely no excuse for using irrelevant data. In fact, without first establishing the relevancy of ANY data set, even those used in the past, one's findings fall apart.

To pick but one example, in the last two issues of IEEE Transactions on Dependable and Secure Computing, two of the three IDS-related articles based their findings on data sets that are 7 or more years old. This is emblematic of why so much research is ignored by industry, and that which isn't often falls flat in practice. If I were an editor of that periodical, which I have been reading for quite some time, I would have rejected nearly every intrusion detection paper submitted in the last 3 years outright on this basis alone.

The data commonly considered the "gold standard" by academics has not been relevant for at least half a decade. Research done in that period whose findings relied on 2001 and prior data is not in any way conclusive, in my professional opinion.

Spy Museum opens FUD exhibit

It is really bothersome to see a museum as popular and, until recently, esteemed as the Spy Museum open an exhibit pandering to fear. In the two-sentence description, a "cyber attack" is compared to Pearl Harbor, immediately discrediting anything that might be contained therein. Disturbingly, this analogy is made by Richard Clarke, someone with serious pull in matters of national policy. Such ludicrous hyperbole may make the museum some serious coin, but it sets back understanding of real-life CNA and CNE issues, the balance between them, and their practical use in modern society and warfare. The result will be misplaced priorities by decision-makers for whom these visitors vote, poorly-invested research and defense dollars, and if left unchecked, economic, military, and intelligence disadvantages on the world stage. Like the CNN-broadcast "Cyber Shockwave," the only thing missing from this exhibit is an F-35, Bruce Willis, and the "I'm a Mac" guy.

An exhibit headline, visible on the museum's website, reads "If cyber spies break America's security codes, could power lines turn into battle lines?" A better question is "who is the curator, a 16-year-old World-of-Warcraft gamer?" On second thought, even a pizza-faced teen would probably know this doesn't make one bit of sense.

Update
A description of the phear. Sadly, it's recommended as something to do. And believe.

It’s a frightening thought—and an exhibit that, for better or worse, is designed to imbue its viewers with the reality of that fear as well as educate them. This is the kind of thinking that led to an extra gift, tucked into the Spy Museum’s Field Guide to Asymmetrical Warfare and passed out at the reception: a flash drive.

(Emphasis my own)

TL;DNT: Academia and industry are both failing

(Too long, did not tweet) I think this is more applicable to my personal blog on industry and academia anyway.

On the cusp of 2010, the state of information security in our society can only be described as a mess. I've come to the conclusion that my career path will now and forever be an effort to bring more science of computing to security in practice (severely lacking now), and reality of security to academia (also severely lacking now). This is at the heart of our mess, and will also be the solution to it. Few-to-no tenure-track professors at accredited universities have real-world experience.

Academic papers are written around decade-old problems, using decade-old data sets, demonstrating a decade-old mindset and ignorance to the volatility of security in practice. There are few models - even fewer that are relevant - and little agreement on terminology as fundamental as risk, threat, and vulnerability.

Industry makes risk decisions with scant or no objective data, builds models on subjective criteria, suffers from physics envy, and is often totally incapable of performing analysis that adheres to the scientific method. In some cases, industry still fails to recognize that security is risk management, evident by the all-too-common requests for ROI to justify security spending. I've seen nearly every word in the English language prefixed by "cyber-" in the last 24 months, simply because it's a buzzword. It's so overused I cringe the few times I have to say it, and the hype risks an overcorrection in the coming years that will back-burner the issues at hand, or water them down with gimmicks and sales pitches to the point where serious concerns in need of resolution are met with the eye-rolling more appropriately reserved for notions such as "cyber Katrina" or "cyber 9/11."

The US now has a "cyber security czar," virtually ensuring failure of public policy just as we've seen with most other "czars" (how's that war on drugs going?). Policymakers don't realize that electronic espionage is just as serious if not moreso than traditional methods of espionage. No agreement has been made on how conflicts (espionage and outright aggression) escalate beyond the internet into the real world, despite having very serious real-world implications in and of themselves. We are not holding to account other countries who tacitly or explicitly permit attacks against our country's critical infrastructure, ensuring their continuity for lack of any sort of risk associated with their actions. Open dialogue is taking place, but only on the most greatly exaggerated, dated, or unlikely risks, reducing national information security strategy to the same level of effectiveness as airline security.

I normally don't like rants without solutions, so for that I apologize. Maybe I'm just in a bad mood. At the risk of reducing all these problems to one oversimplified solution, I strongly feel that bringing academia and industry closer together in how to approach information security issues is the only way to begin to fix most of these problems.

A song for the season

Enjoy. Thanks to my coworker Roger for the assist.

On the 12th day of Christmas, my CIRT did find for me...
12 users clicking
11 hackers hacking
10 sites cross-scripting
9 drives receiving
8 gigs a-taken
7 widgets stolen
6 passwords broken
5 forged emails,
4 PDFs,
3 word docs,
2 hyperlinks,
... and a hole in Adobe new-Player

Speaking at 2010 DC3 Cyber Crime Conference

I'm happy to share that a presentation of mine has once again been selected for the DC3 Cyber Crime Conference, held in St. Louis at the end of January, 2010. I'm very excited to be speaking again. You can read about my past presentations here and here. If you're planning to attend, I'd love it for you to drop by on Thursday from 1:30-3:30PM.

Intelligence-driven Response for Computer Network Defense
Abstract

Network defense against sophisticated adversaries requires a new approach than what the information security industry typically prepares its analysts for. From the overarching incident response process down to the specific questions each analyst must be able to answer, classic incident response techniques and procedures are insufficient in the face of persistent and focused intrusion attempts. A detailed understanding of one’s enemy, specifically, is an overlooked concept in industry-standard information security pedagogy and mindset which can offer strategic, actionable insight into effective response. This presentation extends some information warfare concepts to discuss how intelligence-driven analysis and response can improve the defensive posture of organizations facing advanced persistent threat actors. Examples will be given at the micro and macro level; attendees should be technically well-versed as they are able to see the “big picture” of computer network defense.

Speaking at SANS CDI

I will be participating in four separate events at SANS CDI this year. While the panels aren't yet listed on SANS's website, they should be soon, and Richard Bejtlich has a good overview on his blog. Specifically, I will be involved with:

• Commercial Security Intelligence Service Providers as a moderator
• Noncommercial Security Intelligence Service Providers as a moderator
• Unix and Windows Tools and Techniques as a panelist
• CIRTs and MSSPs as a panelist

If you have budget left for the year, you should definitely check it out. It's going to be a great few days of material, paired with the usual selection of great SANS training.