2008-10-17

Antivirus is failing; long live antivirus

From the most recent SANS Newsbites:

--Security Suite Vendors Question Secunia Study
(October 15, 2008)
Makers of antivirus products and security suites are calling into question the validity of a recent study from Secunia. The study tested a dozen security suites against "300 exploits targeting vulnerabilities in various high-end, high-profile programs" and found the highest scoring suite caught just 64 of the 300 exploits. Some of the companies whose products were tested say that just one aspect of their products was examined. Others whose products were not included called the study a publicity stunt.
http://www.darkreading.com/document.asp?doc_id=166027&f_src=drdaily
http://www.theregister.co.uk/2008/10/15/secunia_tests_backlash/
[Editor's Note (Skoudis): Designing a thorough and fair test regimen is quite difficult, and running the suite of tests against increasingly complex products is very time consuming and expensive. Matt Carpenter and I did this in 2007 for seven endpoint security products, and it consumed two months of our time. Whenever you see a test report of security products, make sure you look carefully at the description of the test methodology and testbed to determine what they measured and how. No test suite is perfect, but some better reflect operational environments than others.]


I took a look at Secunia's test methodology. They cover a broad range of exploits used by sophisticated adversaries in modern highly-targeted attacks. Their results for particular malicious files & attack types I've seen reflect my own experiences at a large enterprise CIRT, defending against highly-targeted attacks designed for the explicit purpose of compromising proprietary information. Not surprisingly, their resulting detection rate reflects my experiences as well. While the proportions used by Secunia may not have fairly reflected the universe of malware that's "in the wild" today, I don't care. There's no point in comparing detection rates for Blaster, Slammer, and other previously-solved problems. What I care about are the serious threats; the Malware that's being used against carefully-selected targets, that's working. The malware that only has to change by less than 5% (as measured by fuzzy hashing ala ssdeep) to evade detection by leading vendors. That's where the adversaries' foci is today, it's where we need anti-virus the most, and it's where anti-virus is failing us. Naturally, their conclusion is spot-on:

These results clearly show that the major security vendors do not focus on vulnerabilities. Instead, they have a much more traditional approach, which leaves their customers exposed to new malware exploiting vulnerabilities.

Kudos to Secunia for standing up to the industry.

Most of the anti-virus vendors are fighting hard to maintain a status quo which no longer reflects reality. If you'll recall, they lashed out against Skoudis and Carpenter when their tests led to similar conclusions about the state of the AV industry almost exactly a year ago. They're better off putting their resources into product engineering to address 21st century threats, than marketing and PR.

No comments: