Windows Vista was hit by significantly fewer publicly disclosed security flaws in its first year than Windows XP and open source rivals in their first years, according to a report from Microsoft.
The report, written by Jeff Jones, a security strategy director in Microsoft's Trustworthy Computing group, is part of Microsoft's effort to show that its work on redesigning the security architecture and adding new security features to Vista have paid off.
Jones also found that changes to the way Microsoft handles patching has resulted in less work for system administrators on Vista compared to Windows XP.
The report comes on the heels of figures from Secunia, which reported fewer vulnerabilities for Windows in 2007 compared to open source operating systems in the same time period. However, Microsoft's report compares the way each OS fared in its first full year of supported distribution.
Comparisons between different types of operating systems on the basis of numbers of public bug reports are often downplayed by security experts, who say they are only part of the picture. For instance, Linux-based OSs are composed mainly of third-party components whose bug reports are all known publicly, whereas third-party components play a small part in Windows and many bugs may be uncovered but not made public.
However, Microsoft's main interest with the new report is in convincing users that Vista — which has received heavy criticism over bugs and usability issues — is more secure and more easily managed than XP.
"The results of the analysis show that Windows Vista has an improved security vulnerability profile over its predecessor," said Jones in the report. "Analysis of security updates also shows that Microsoft improvements to the security update process and development process have reduced the impact of security updates to Windows administrators significantly compared to its predecessor."
But numbers don't tell the whole story, countered a pair of security professionals.
"As a lot of people have said, Gartner included, [corporate] switching to Vista has been a slow process," says John Pescatore, Gartner's security guru. "Microsoft has said all along that Vista would be a big leap in security, and it's been on that drum beat. Everybody does that. Oracle, Sun, Red Hat, they all say that they have fewer patches than the other.
"But Vista is definitely a major [security] improvement over XP."
Andrew Storms, research director at security vendor nCircle, agrees. "The year-one for Vista and year-one for XP clearly show that Vista has had fewer vulnerabilities than XP. That's a good sign for Microsoft and the enterprise — and consumers."
Even so, both Pescatore and Storms question the worth of bug counts, as well as some of Microsoft's measurements. "More important than the number of vulnerabilities is what attacks are targeting and how much pain there is in entering patches," said Pescatore. "That should be the real measurement."
In other words, a vulnerability in Vista or XP should not be treated as the equal of one in say, a Linux distribution. The former, because of the widespread use of Windows and much greater interest on the part of attackers to exploit the OS's weaknesses, must be patched without delay. The latter? "For most companies that use Linux on the desktop, they can patch in a much more leisurely fashion," says Pescatore. That's because they know the likelihood of an attack is slim.
Storms came to a similar conclusion, but for a different reason. "Outside of the Vista-to-XP, comparisons are a moot point," says Storms, referring to the section of Jones' report where Vista's vulnerability count was compared against Mac OS X, Red Hat Enterprise Linux and Ubuntu Linux. "You just can't compare Vista to Linux," says Storms. "There are simply too many variables. I'd argue the same for Mac OS."
Nor, says Pescatore, should all of Microsoft's conclusions be taken as gospel. "Patch events," which both Jones and Wilson cited, is a good example.
According to Microsoft, patch events is the number of times a company has to activate its patch management process because a vendor has issued a security update. Jones, for example, contrasted Vista's 9 patch events in its first 12 months with XP's 26, although as he acknowledged, XP's events were spread across more days because Microsoft had not yet moved to a monthly patch schedule.
"Patch events don't take into account the days spent making sure an enterprise's applications will work once a patch is deployed," counters Pescatore.
And just as a vulnerability on one OS shouldn't be equated with one on a different OS, patch events aren't comparable, either. "It's a fact that when 'Patch Tuesday' comes around and there are critical Windows patches, you have to start calling overtime. With other products, [those patches] can wait until the end of the month," Pescatore says.
But while he questions some aspects of Jones' report too, nCircle's Storms gave Microsoft an "A" for effort. "It's worth Microsoft's time to do this, and talk about Vista like this, on a marketing level and on a public relations level. They've come leaps and bounds in communicating [about security] with the public.
"They're actually putting together numbers and releasing them," Pescatore says. "That's hard to find among OS vendors."
In its first year Microsoft released 17 security bulletins and patches affecting Vista, compared to 30 for XP in its first year, Jones wrote.
Microsoft fixed 36 bugs in Vista compared to 65 in XP, and there remained 30 unpatched bugs in Vista, compared to 54 for XP in their first years.
The number of vulnerabilities fixed in Mac OS X and in Linux-based operating systems was higher in their first years, Jones said: 360 in Red Hat Enterprise Linux 4 Workstation, 224 in Ubuntu 6.06 LTS and 116 in Mac OS X 10.4.
The figures for Red Hat and Ubuntu apply to a reduced set of components, which Jones used in order to make the figures more comparable to those of Windows.
"It is a common objection to any Windows and Linux comparison that counting the 'optional' applications against the Linux distribution is unfair, so I've completed an extra level of analysis to exclude component vulnerabilities that do not have comparable functionality shipping with a Windows OS," Jones wrote.
Jones compared the number of "patch events" during the year for each operating system, indicating the number of days out of the year that administrators needed to deal with patches for each OS.
"My analysis found that administrators were required to mobilise much less often for Windows Vista than any other product examined," he wrote.
Vista had nine patch events, XP had 26, Red Hat had 64, Ubuntu had 65 and Mac OS X had 17, Jones found.
Jones admitted that the figures do not indicate which operating system is "more secure" than the others, saying any such analysis would need to look at software quality, administrative controls, physical controls and other issues.
However, he said the figures were nonetheless important within their context. "This report is a vulnerability analysis, which may provide some elements that could be part of a broader security analysis," he wrote.