Hackers Looking Hard For Anti-virus Software Vulnerabilities
"Our goal is pretty simple," said Neel Mehta, the team lead for Atlanta-based Internet Security Systems' (ISS) X-Force research group, and one of the two who will talk at Black Hat Wednesday about their anti-virus vulnerability research. "We want to state the severity of the threat, and point out that just like every other part of the network, security software is not immune from attack."
Mehta will join Alex Wheeler, an independent security consultant whose name was splashed about last winter when he discovered critical vulnerabilities in major anti-virus vendors' products, including those from F-Secure, Symantec, and Trend Micro.
The pair used binary auditing tools and techniques, said Mehta, to examine anti-virus software and find the vulnerabilities, which ISS announced starting in February.
"When you don't have access to source code, you can either try and black box test [the software] and see what happens, or look under the hood [using binary auditing]," said Mehta.
Although he and Wheeler found numerous vulnerabilities, Mehta was quick to characterize anti-virus software as "better than average" when it comes to security.
"AV vendors are, of course, fairly aware of security, and have a security focus. But the sheer complexity of the operations their software's doing -- decompressing many different file formats, for instance -- means they're bound to make mistakes.
"It comes down to 'secure' versus 'perfect,'" he added. "I think AV products are secure, but they're not perfect."
By some other counts, security software isn't even that secure. According to a recent study by the Yankee Group, vulnerabilities affecting security software are up substantially, in large part because they're virgin territory compared to other vectors, such as operating systems, which have become more secure.
ISS' Mehta was quick to agree. "Hackers are looking to security software as the core operating systems become more secure. They're looking for more targets elsewhere."
Other reasons why security software in general and anti-virus software specifically are being targeted include their wide deployment and, of course, the implications of a breach.
"AV is an important security mechanism," said Mehta, "but it's a potential weak point."
Signature-based anti-virus defenses, Mehta argued, are inherently vulnerable, because they rely on a compromise between performance and security. "There'll always be something a hacker can take advantage of, whether it's a limit on file size for scanning or the recursion level in an archived file."
Mehta mentioned one technique that attackers can use to fool signature-based file scanning AV products. "They can make a file look like two different formats. The AV software will try and recognize the file as a specific type to scan it, but the file actually looks like something else, something benign, for example."
Mehta has a vested interest in pushing non-signature-based defenses, since ISS' security products rely instead on behavior-based examinations of potential threats.
Ironically, however, ISS was the victim of the one major worm outbreak traced to a vulnerability in security software. In early 2004, the Witty worm snuck through ISS firewalls, and reportedly infected tens of thousands of PCs worldwide.
But ISS learned its lesson, an analyst said. “Not coincidentally, ISS tightened up its security processes and decreased its share of vulnerabilities last year relative to 2003,” said Andrew Jaquith, a senior analyst at the Yankee Group, in an earlier interview.
"We haven't seen a virus or worm attack anti-virus software," Mehta said, "but all it takes is one time."
In 2004, in fact, ISS found, but didn't widely publicize, a major vulnerability in McAfee's anti-virus management console that "could have compromised an entire enterprise," said Mehta. "The attacker could force an update to the management software, which would then update all the clients.
"The fact is that security software is under fire," Mehta concluded.