The security world according to vendor surveys

One survey describes USB devices as the leading security threat facing IT managers these days. Another one talks about zero-day attacks emerging as the biggest driver of security budgets. A third survey says a majority of 1600 consumers polled want their banks to implement risk-based authentication mechanisms.

What's the common thread tying together these and a myriad of similar surveys released on a regular basis these days? They are all conducted by vendors, selling products that address the very issues being highlighted as major challenges or issues in their survey results.

But that doesn't make the surveys untrustworthy, or any less valuable as communication tools, insist vendors.

Red flags, info-spam or useful data?

"The fact that a survey is from a vendor is absolutely a red flag," admits Sebastian Holst, senior vice president of sales and marketing at PreEmptive Solutions Inc. in Mayfield Village, Ohio. But don't let that persuade you to completely discount its findings either, he says: Most good vendor surveys are very transparent about the methodology, the demographics of those surveyed, the questions asked and the filters used to vet inconsistent responses. "If the quality of the research is sound, these surveys can be very effective" in communicating key issues and threats to the target market, says Holst.

Not really, says Matt Kesner, chief technology officer at Fenwick & West LLP, a Mountain View, Calif.-based law-firm, who argues that most vendor surveys are just more "info-spam." "It would be a huge surprise if vendors published a survey that was less than glowing," he says. "As the amount of this fake or near-fake news release information [increases] we become more desensitized to it. We just ignore it," says Kesner.

At issue is the growing number of surveys being released by security vendors. Most are conducted by the vendors themselves, though some are commissioned projects undertaken by third parties. Often the sample sizes are in the hundreds; in some cases the number exceeds a thousand. The surveys purport to highlight some key security challenge or issue confronting IT managers -- one that typically can be resolved using the vendor's products, as a few examples show:

- A survey by Centennial Security Ltd. of over 370 individuals at a security trade show showed a majority saying they considered USB memory sticks and similar portable storage devices to be their "number one" security concern. Centennial sells a product called DeviceWall, designed to help companies lockdown unauthorized file transfers from corporate systems to portable storage devices.

- About 91 percent of respondents in a survey of more than 1,600 adults from eight countries said they wanted strong authentication methods that went beyond the usual username and passwords for online financial transactions. Around 82 percent wanted their banks to monitor online and telephone banking sessions for signs of fraud. That survey was done by RSA, the security division of EMC Corp. The company sells strong authentication and fraud detection products.

- Two-thirds of IT professionals (67 percent) in a PatchLink Corp. survey of more than 200 CIOs anticipated an increase in zero-day (0day) threats this year, while 29 percent said that the zero-day threat is the primary issue driving their security budgets. PatchLink's products include a vulnerability management suite that is designed to help companies deal with zero-day threats.

- An application risk assessment survey conducted by PreEmptive and involving responses from over 600 companies showed more than two-thirds not having controls in place to protect against illegal reverse engineering of their applications by others. About 27 percent of companies in industries most at risk of such infringement, such as computer software and banking, had no controls in place for preventing their products from being reverse engineered, and just 36 percent had both the tools and the policies to combat it, said Preemptive -- which sells so-called data obfuscation products for source code protection.

PR's not the only point

Such vendor surveys can evoke occasional skepticism, said Bill Piwonka, vice president of product management at Centennial. "We live in a media savvy world. So there's going to be some skepticism," with vendor surveys, he said. But according to Piwonka, the reality is that such surveys can play an important part in helping shape product strategy for the vendor and in helping educate the customer on important issues.

"One of my deliverables to management is to ensure that our product portfolio is going to deliver value to customers and to prospects," he said. Surveys provide a way to test and validate assumptions about the changes that might be needed to a product or roadmap to deliver that value, he said.

Marc Gaffan, director of product marketing at RSA, said there are some key drivers for doing such surveys. Very few of them have to do with PR.

"We want to make sure that we understand the market correctly and understand our business goals," he said. "We want to be able to see in what direction we need to move from a product perspective. PR is just a byproduct," he said. According to Gaffan, the very fact that vendor surveys can sometimes be perceived as self-serving by consumers is enough reason to be careful about how the information is publicized.

Don Leatham, director of solutions and strategies at PatchLink, "would be stunned" if customers were not skeptical of vendor surveys. "Our customers are very well versed in vendor speak and vendor spin," he said. But in the same manner that these surveys can help vendors validate assumptions about the market, he says, they can help customers validate assumptions of their own.

"Our customers are looking for guidance and information from all sorts of sources," Leatham said, "but they kind of like the validation that people who are in their same position can give" in vendor surveys. Leatham added that surveys help their customers and other readers understand what sort of challenges and issues their peers are facing.

Consider the source -- and the reader

David Jordan, the chief information security officer for Virginia's Arlington County, said that the value derived from such surveys depends on who's reading them. For someone relatively new to security or inexperienced in the field, vendor surveys can be a pretty valuable source of information, he said. This is particularly so, he said, because there are relatively few independent and reliable sources of information on vendor technologies.

Jordan also comments that it's hard to imagine than smart vendors would risk alienating their customers by publishing surveys that are blatantly self-serving without having solid research backing their claims. "I can't imagine even one CEO at a major company putting their products in jeopardy for a 30-day promotion," he said,

It's also not surprising that vendor surveys highlight the importance of their own products. "That's just American business. Everyone does it," he said.

A lot depends on the care that is taken in conducting the survey, said Pete Lindstrom, an analyst with Midvale Utah-based Burton Group. "You need to see the survey instrument to see if it is biased in some way," he said. He adds that there is often a lack of clarity around definitions and the analysis may not completely reflect the numbers.

"In a general sense I am a fan of surveys, but the devil is in the details," he said. "The non-discriminating person can be easily swayed by these survey results. That is often the reason vendors use them. But they can provide good information if the methodology is appropriate and the survey instrument is available," he said.

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Show Comments