Linux Vendors Question Forrester Security Report
Last week, Forrester senior analyst Laura Koetzle released her year-long study of published security vulnerabilities and their fixes during the time span from June 1, 2002 to May 31, 2003.
Using metrics she and her colleagues devised, they measured the number of days customers of Windows and Linux were at risk from vulnerabilities, the percentage of security problems fixed, and how each operating system ranked in the severity of its uncovered flaws. Koetzle's report compared Windows with four distributions of Linux: those from Red Hat, SuSE, Debian, and Mandrakesoft.
Microsoft was the only vendor of the five studied which released a fix for every disclosed vulnerability, although none of the Linux makers were far behind. Red Hat, for instance, fixed 99.6 percent, while even last-place Debian patched 96.2 percent of its severe vulnerabilities.
The four Linux companies took issue with those conclusions.
In a joint statement, the four said: "Despite the report's claim to incorporate a qualitative assessment of vendor reactions to serious vulnerabilities, it treats all vulnerabilities as equal, regardless of their risk to users. As a result, the conclusions drawn by Forrester have extremely limited real-world value for customers assessing the practical issue of how quickly serious vulnerabilities get fixed."
Koetzle defended the survey Friday, saying that she did rank the vulnerabilities by separating them into severe, medium, and low based on the same criteria applied by the U.S. government's National Institutes for Standards and Technology's (NIST) ICAT project.
"Essentially, the vendors don't like the [vulnerability] severity ranking that I used," she said. "You can argue about [which severity ranking system to use] all year and not get anywhere. Rather than come up my own, I used the ICAT project's. I didn't want to get into the business of deciding which vendor's rating system was more accurate."
The Linux vendors also knocked the Forrester report for what it saw as a slant toward Microsoft Windows. "The claim that one software vendor had fixed 100 percent of their flaws during the period of the report should be incentive for a closer investigation of the conclusions the report presents," the four said in the joint statement.
Koetzle reiterated her study's accuracy, claiming that well before the report went public, she checked every discrepancy brought up by any of the vendors. On the Microsoft issue, "I checked out every single thing that [the vendors] brought up, and couldn't find any instance of a recorded vulnerability in Windows that was never fixed."
Although some might read into the Linux vendors' comment that they're implying some impropriety on the part of Forrester, Koetzle doesn't see it that way.
"If they wanted to accuse me of kowtowing to Microsoft, they would have said so straight out," she noted. Her report, she said, was totally independent, and didn't rely on more than cooperation from any of the vendors. "We didn't take a dime for this from anyone."
While the four vendors said that open-source shouldn't be treated the same as Windows -- "We believe the report does not treat vendors of Free Software and the single closed source vendor in the same way," the quartet said in their statement -- that's exactly what Koetzle tried to do, she said.
"We wanted to provide data so enterprises could make rational decisions, not ones based on pre-conceived notions."
And she's happy with result. "Nobody disagrees with the facts in the report," she noted.
This story courtesy of TechWeb.