Preparation key to beating internet problems

Keep monitoring and be ready, says Frank Hayes

You can beat internet sabotage. Martha Stewart Living Omnimedia did. Several weeks back, I mentioned a 2006 incident in which Con Edison Communications "accidentally hijacked internet connections to investment houses, a bank, Martha Stewart's publishing empire and the New York Daily News." I implied that Martha Stewart was knocked offline. I was wrong.

Here's what actually happened: Just after midnight on January 22, 2006, Con Edison began telling the internet that it was Martha Stewart. That is, Con Edison erroneously began sending out routing information to redirect internet traffic intended for Martha Stewart Living Omnimedia to its own servers.

The publishing company was a Con Edison customer. So were other businesses and internet providers whose routing information Con Edison hijacked at the same time. The result was a mess that wasn't completely cleared up for 18 hours — and some businesses were offline for most of that time.

But not Martha Stewart, whose CIO, Mike Plonski, wrote to me to clarify what happened at his company.

Plonski's secret sauce? No big secret — just network monitoring and redundancy.

Plonski said: "While one of the internet connections at our corporate offices was impacted by the ConEd issue you describe, we, as a company, are smart enough to have employed redundancy, both by location and carrier, for our network operations. As a result, during this time frame, we simply flipped all of our internet traffic to run over our secondary line until ConEd resolved their issue."

OK, it was a little more complicated than that. Plonski says his staff spotted the problem through routine network monitoring. There was clearly something wrong with network traffic coming to the corporate office over the Con Edison line. Thanks to the monitoring, the company knew about the problem about 30 minutes after it started.

Because of the type of outage, an IT staffer had to connect and manually switch over to a redundant line. That took another hour.

Total time for the outage: about 90 minutes in the wee hours of a Sunday morning. Total impact: minimal.

An outage? Yes. A knockout? No way.

And handling the problem didn't require rocket science — just monitoring, redundancy and sharp IT staff work.

That's important, because today your business runs on the internet to at least some degree. With outsourcing, increasingly automated supply chains and software as a service, your business operations will soon be depending on the internet more and more.

But while you've sold the internet to your management as a great platform for business, in reality, it's ... well, shaky. What you want -- and need -- is stability. What you've got is a global network in which digging equipment and boat anchors can tear up physical connections at any moment — and hackers, spammers and censorship-happy politicians can sabotage it just as often.

You can't fix the internet. You can't prevent that damage and sabotage. But you can use monitoring to spot minor bottlenecks and major attacks. You can use redundancy to guarantee there's a path to the internet even when your usual route is cut off. You can plan and react to reduce the impact of an outage to a fraction of what it would otherwise be.

You can do this. Like Martha Stewart's IT shop, you really can beat internet sabotage.

And — as someone once said — that's a good thing.

Join the newsletter!

Error: Please check your email address.

Tags managementperparationinternetmonitoring

Show Comments

Market Place

[]