- 12 November, 2003 01:24
FRAMINGHAM (11/11/2003) - On Aug. 14, 2003, there was a blackout in NYC. But Craig Sisler, CIO of Capital Printing Systems in New York's East Side, had a plan.
CSO: What was it like for people when the blackout occurred?
Craig Sisler: Surreal is the first word that comes to mind. The initial feeling was a general sense of alarm. The power went down completely, lights went out, workstations went dark. Because it was daylight, we couldn't tell that it was a citywide blackout. We thought it was a building emergency or a fire. The first thing I did was to tell everyone to remain calm. I went into the server room and covered the servers with duct tape and plastic in case the sprinklers went on.
So duct tape is good for something after all?
It's low-tech, but it worked. There were no signs of fire, and we were able to do an orderly shutdown. We don't have a generator, but we have enough uninterruptible power supply to shut down safely and avoid systems crashing.
Did you have a disaster recovery plan in place?
Last year, we created a disaster recovery site in New Jersey. It's basically a microcosm of our system offsite. Our postmortem of the blackout taught me that had it happened in the middle of the night, things wouldn't have gone as smoothly. There was a core group of key personnel who knew what to do. But had this happened at 3 a.m., things might have been different. We don't have a road map of a disaster recovery plan that someone could pick up and execute. I now know that we need to create one.
What about the people plan?
One of the themes that came back from employees was that communication was our biggest problem. We knew there were certain people we needed to contact, but we didn't have a phone chain in place. We're looking into that. Without that, we left a lot of people literally and figuratively in the dark. Another thing that came up was the comfort factor. We have refrigerators with water and snacks. But what if this happened in the middle of January? We'd need real food and blankets.
After the blackout, you solicited feedback from the entire corporate staff as to what could have been done better. Why?
From an IT perspective, the end user often has more useful feedback than a technical person. I wanted to solicit opinions from people who wouldn't ordinarily volunteer them. The most useful suggestions came from nonexecutives--bright people who care about the company but who aren't often involved in the business decisions.
What were some of their comments?
One user reminded us that we should make sure the disaster recovery plan is communicated to everyone. Others suggested that we should have food available. It may sound crazy, but I contacted an organization that distributes MREs (meals ready to eat) to the military. MREs have a long shelf life, and they don't take up much space.
What did you do with the feedback you received?
I compiled all the comments that I got and created a response review task list, which I broke into sections: Things that we can just go ahead and do, things to buy now, and finally a list for things that need more review and budget approval.
How should CSOs go about asking for feedback after an event?
That depends on the culture of the company. My feeling is that the way we did it was the way to go. Send an e-mail out to everybody so that everyone feels like their views are important. You want people to think from the gut.