TOP OF THE NEWS
-- Overhaul AntiVirus Product Testing Now
(October 10, 2007)
Momentum is growing for an overhaul of the way anti-virus products are tested. Presently, tests focus on signature-based malware detection and have changed little over the last 10 years. However, anti-virus technology is changing to meet the demands created by new malware, and the organizations believe that tests need to be reformulated to reflect that change. The tests don't examine the effectiveness of behavioral anomaly malware detection, a technique that is proving valuable in catching malware that spreads quickly. Proposals for new testing methods and procedures will be presented in November at the Association of AntiVirus Asia Researchers 2007 Conference in Seoul, Korea.
[Editor's Note (Skoudis);: Given the change in threat, with new malware introduced every hour, this is a good development. My colleagues and I have been pushing for something like this for about 2 years now. I made a call very much like this at a talk during the Anti-Spyware Coalition meeting in February 2006. My colleague Tom Liston and I released a tool called Spycar a year and a half ago to test the behavior-based claims of anti-malware vendors, and discovered that most were either not doing any behavior-based detection at all or had severely broken behavior-based functionality. Many of the anti-malware vendors publicly scoffed at our testing, while a few said that our results were interesting. Just last month, my colleague Matt Carpenter and I tested a bunch of products again, and found that the behavior-based detection capabilities of all the major vendors were severely lacking. I'm hopeful that this new initiative will help improve the state of
(Liston): Saying we were "skoffed at" is Ed's way of being polite. Essentially, they told us that we were amateurs who didn't know what we talking about, even _after_ we had discovered and disclosed serious flaws in the behavior-based detection offered by several major vendors. With increasing numbers of malware being released and the rise of targeted malcode attacks, behavior-based detection is going to become
anti-malware's front line defense. It's nice to finally see this reality being addressed by AV vendors' testing methods, and, of course, it's also nice to be vindicated.
(Northcutt): The anti-virus industry has served us well. Without them, computing might well have failed, but they are a tad stuck in their ways and malware grows ever more complex. I hope that projects like Spycar, which hold vendors responsible for something beyond signature detection, continue to prosper.]