It’s a good idea when thinking about disaster recovery to look at contingency planning, especially for those organisations that would need to operate during a disaster. This is a different issue to transferring operations to a remote site. There are various issues in operating in a disaster environment, usually starting with electrical power but also extending to employee concerns and the need for resupply.
When demand gets beyond the capacity of uninterruptible power-supply batteries, even with those as big as the 48-hour supplies of carrier-grade telephone central offices, there will be a need to generate power. Diesel generators are the most common, in part because diesel fuel is much safer than petrol. The engines may also need other resources besides fuel.
Diesel fuel has a definite storage life, and diesel engines powering generators should be run on a regular basis. Regular test-runs help recognise pending or actual problems, keep the engines lubricated and help with turnover of the fuel.
Diesel fuel tanks need maintenance, and possibly periodic testing of the fuel. Water can collect in the tanks from condensation of air.
Tank, generator and battery placement
Lessons about locations for fuel tanks came from both September 11, 2001, and the hurricanes in the US. Without touching the various conspiracy theories touring the internet, it is clear that having the fuel tanks for the emergency generators on the roof of World Trade Centre building 7, which housed New York’s Emergency Operations Centre, did not contribute to continuing operations when oil cascaded down and added to fires.
Diesel generators and their tanks can be a significant distance apart, as long as fuel can be pumped by the available pump motors and there is a proper method for getting air or exhaust into the tanks to avoid suction locks.
Large battery banks, with liquid acid electrolyte, often can’t be placed above a certain level of the building because of the local fire code. The concern is acid raining on to fire fighters.
After 9/11, surviving carriers, ISPs and financial institutions continued to operate on their diesel. Typically, such installations had a week of fuel and a plan, coordinated with fire and police officials, to have resupplies driven to them. Many operators were shocked, however, to find the engines working poorly, and then stopping, after 24 to 48 hours.
They stopped because the massive dust contamination clogged their inexpensive air filters and they had no replacements. At subsequent network operators’ meetings it was found that 9/11 was not unique in putting significant dust into the air. This had happened after the Mount Saint Helens explosion, but also after large forest fires in western Canada.
• Test diesel generators about once a week. Periodically, test the automatic power fail-over and have the fuel tested for contamination.
• Fuel tanks don’t need to be next to the generator engines. There are potential problems if they are too high or too low. Tanks too low can be designed to work underwater, but it won’t be easy to refill them. Middle levels of parking garages can be good locations.
• Have contracts in place for fuel refills. Store spare fuel and air filters. Keep a starting-battery inside and charged.
• Be sure your facility, especially if it is windowless, dedicates some power to necessary heating, ventilating and air conditioning.
• If your water supplier loses electrical power you may not have running water. Generous supplies of bottled water will make life easier, along with alcohol sanitisers and personal wipes. Rainwater or even floodwater can be used. If you use floodwater put bleach in it simply to make it safer to handle.