Security execs push for broader use of metrics

The difficulty of obtaining metrics doesn't mean they're not important, conference attendees say

Measuring IT security risk and the effectiveness of corporate defences can be a difficult and somewhat imprecise task. But that shouldn’t be an excuse for not trying to gather such metrics, IT managers said at the annual RSA conference in San Jose, California.

Security professionals have long advocated that companies use both quantitative and qualitative metrics to get a more granular view of IT risks and the controls needed to mitigate against them. At the conference many attendees said the topic is taking on increased importance because of regulatory requirements that are pressuring corporate executives to demonstrate due diligence on protecting data assets.

“Start using metrics to make security decisions, and don’t get too hung up on the quality of the data, and don’t get too hung up on complicated methodologies,” advises John Meakin, group head of information security at London-based Standard Chartered Bank. “Just start doing it.”

Meakin says that using metrics to prioritise security threats and vulnerabilities according to the risks they pose to IT assets can help security managers target their resources more effectively — and determine whether they are putting the appropriate amount of money into security efforts.

For instance, Standard Chartered has been moving to a risk-based approach to vulnerability management over the past three years. As part of this effort, Standard Chartered has classified all of its core information systems on a value scale of high, medium and low based importance to its business operations and the disruptions or losses that would result from security failures on them, Meakin says. He added that the bank has developed similar measurements for threats and vulnerabilities and the likelihood that they will be exploited on each of its systems.

The approach has given Standard Chartered a much clearer picture of IT risks enterprise-wide, Meakin says, adding that it has also helped the bank to better marshal its security resources. As an example, he says that about three years ago the bank was considering encrypting all confidential traffic moving over one of its WANs because of security concerns. But a metrics-based risk assessment showed that such encryption was overkill.

“I would say one of the worst things I could do is to spend too much money on security,” Meakin says.

Moving on from ‘gut feel’

Zions Bancorporation in Salt Lake City, in the US, started using metrics as part of its IT security efforts about four years ago. The goal was to move away from relying on “a subjective gut feel of risk” to get a more accurate view of threats, vulnerabilities and available security controls, says Preston Wood, the bank’s chief information security officer. “It’s very much about making sure you spend just enough [on security] — not more, not less,” Wood says.

The metrics that the bank’s security staff put in place have given officials at Zions a much clearer picture of the effectiveness of both its tactical and strategic security efforts, according to Wood. He says the metrics have also been useful in getting the bank’s business units to understand the nature of the IT security risks they face.

Meakin acknowledged that setting both quantitative and qualitative security metrics can be a big challenge given the dynamic nature of threats and the difficulty involved in attaching a definite value to information assets.

“But there’s no excuse not to start doing it,” says Dan Geer, chief scientist at Verdasys, a security software vendor. “This is an idea whose time has come.”

Despite the challenges, it is possible to begin gathering and using metrics, Geer says. He adds that the goal shouldn’t be so much about arriving at specific numbers for measuring security risks but about getting a feel for what’s important.

“I’m fairly certain that A is better than B and that B is better than C,” Geer says. “I’m not sure if I can say A is 3.2 [times better than] B and that B is 6.9 [times better than] C.” But that isn’t even necessary, he says.

The key is not to make the whole process overly complicated, agreed Pete Lindstrom, an analyst at Spire Security. Security-risk metrics are “simply a probability, based on legitimate experience of your network,” Lindstrom says.



Used to assess the value of IT assets, the vulnerabilities faced by those assets, what measures are available to protect them and at what cost.

They also let users look at the level of exposure to security threats and the probable damage that would result if one were realised.


Includes baseline measurements, such as categories and numbers of IT assets, threats and vulnerabilities in important business areas.

Also used to measure figures such as the percentage of assets within a category under an acceptable level of control.

Source: Enterprise Management Associates

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags metricsSecurity ID

More about Enterprise Management AssociatesRSASpireSpireZions Bancorporation

Show Comments