The effects of an increasing number of outages of web-based service providers were felt in New Zealand recently when hosted software provider Xero was forced offline for 45 minutes after datacentres at RackSpace, which hosts Xero's data, failed on June 30, causing a 45-minute outage for Xero customers. The Rackspace incident is far from the only case of a service provider going down in recent months. Based on news reports, the number of such incidents appears to have been increasing lately, analysts say. They cited shutdowns of the Google Apps software hosted by Google, a major outage at PayPal earlier this month and a distributed denial-of-service attack on Twitter, as well as the Rackspace datacentre incident.
Observers pointed to several possible reasons for the apparent uptick in online outages, including IT budget and personnel cutbacks, increasing corporate dependence on hosted applications — and bad luck.
Kurt Roemer, chief security strategist at Citrix Systems, says he wonders whether a two-hour shutdown of Cisco Systems' website this month "would [have] happened a few years ago... when they had multiple people checking every single change." Cisco blamed the outage on human error.
IT staff cuts spurred by the economy are likely to continue throughout the remainder of the year. According to a survey of 300 IT centre managers last year by the US Association for Computer Operations Management, half of all datacentres were planning to cut 2009 budgets by an average of 15 percent. Respondents at 14 percent of those companies said the cuts would include layoffs of IT staffers.
Ken Brill, executive director of Uptime Institute, a datacentre engineering and consulting firm, says such budget and personnel cutbacks can prove disastrous to IT. "We're not doing the maintenance we should be doing, and when you don't do maintenance, you increase the probability of catastrophic failure," he says.
Brill adds that energy-efficiency efforts may be prompting datacentres to cut back on redundant equipment and run their systems harder, exposing equipment flaws.
Leslie Daigle, chief technology officer of the Internet Society standards group, says the online service business will probably survive despite periodic outages. "The internet is in a state of constant evolution, and that really does provide its overall resilience," she says.
Some IT managers say that recent outages aren't changing their minds about using hosted services.
For example, Ted Maulucci, CIO at real estate developer Tridel, defends the durability of hosted datacentre systems, despite the fact that his provider had suffered an outage. "Five years ago, it was not uncommon to experience the odd interruption, whereas today it has been pretty rock-solid, other than the major failure that happened," he says.
Neal Puff, CIO for Yuma County, Arizona, agrees, even though the county moved its ERP system from a cloud-based provider back in-house last year. Puff says he still believes that "a well-run hosted solution with reliable connections will perform well and be as reliable as any well-run in-house system".
Paul Williams, Xero's general manager of finance, says Xero was able to communicate with customers throughout the June 30 disruption through its blog and on Twitter.
The disruption came around the financial year end in Australia, where Xero has a growing customer base, but that wasn't much of an issue, he told Computerworld at the time.
"We were lucky it wasn't on GST day," he said.
He remains supportive of Rackspace, despite the incident.
"From a Xero point of view, Rackspace is one of our long-term partners. We are disappointed about what happened last week, but the ability of a top tier provider like Rackspace is what we need," Williams says.
He says Xero pays a premium over other offerings to get that level of service.
- Additonal reporting by David Watson