Chris Finucane recently received an unsolicited email from a user who wanted to know why his files were suddenly flying over the WAN. The question — which came from someone who often complained about network sluggishness — surprised Finucane. “If users notice the difference without you asking, you know you have something good,” says Finucane, who is CTO for the Office of Inspector General for the US Department of Health.
In this case that something is WAN optimisation, an advanced technology for improving application performance over the wide area. A year ago, the department began deploying Riverbed Technology’s Steelhead appliances in its 85 field and ten regional offices, including its Washington headquarters. In the process, it has opened the floodgates for wide-area data transfers and positioned the Inspector General’s Office to handle bigger and more information-intensive projects.
In search of better WAN performance
Finucane hadn’t planned on exploring using this type of technology as he had already allocated his annual IT budget to other projects. In addition, John Rogosky, the office’s network manager, had just finished a network upgrade, swapping 56kbit/s frame relay links for T-1s between the offices and MCI’s vBNS mesh network. But the bandwidth increase didn’t deliver the performance upgrade Finucane expected.
“People should have been cheering. But not a lot of notice came out of that,” he says.
The Office of Inspector General (OIG) is a watchdog organisation for the department’s 300-plus programmes, which include Medicare and Medicaid. Auditors make sure programme funds are distributed and used properly; analysts evaluate programmes for efficiency and effectiveness; investigators look into possible instances of fraud or abuse and lawyers provide legal services. The OIG can assess monetary penalties for violations such as false billing, as well as exclude people from participating in federal healthcare programmes.
With its investigative bent, the OIG is a data- and research-heavy outfit. While remote staffers had some access to centralised systems, they had grown accustomed to less-than-ideal email performance and file server access over the WAN. With the bandwidth upgrade doing little to improve conditions, Finucane realised he needed to do more, especially as requirements for sharing data among far-flung users were increasing.
“We had a couple of different initiatives that were pushing us to find a way to streamline our network,” he says.
For example, leaders wanted field staff to be able to query a central mainframe database, then produce and transfer customised test reports or spreadsheets. “With the existing infrastructure it didn’t look like we were going to be able to support it with an acceptable response time for our users,” Finucane says.
In addition, the OIG was mulling over long-range plans to consolidate its IT infrastructure. With its existing set-up, it maintains servers in each of its ten regional offices. This means carrying more user licences than it would need if the applications ran on a central server. The OIG could, potentially, cut back on licensing by centralising more servers and applications, but not unless Finucane could find a way to offset the performance hit users in remote offices would suffer if they lost their local server power.
With so many network-intensive demands looming, Finucane and Dave Agsten, the infrastructure architect, started looking into ways to improve throughput. As part of that effort, the OIG tested WAN optimisation devices from three vendors. The offerings were comparably priced, but Riverbed’s Steelhead appliances delivered the greatest performance improvements, Finucane says.
Redundant transfers be gone
In tests, and in production, Finucane found some of the most impressive gains came from Riverbed’s ability to eliminate repetitive traffic from WAN links — a feature the vendor calls scalable data referencing.
Riverbed’s devices work by intercepting TCP requests sent across a WAN link, then segmenting and indexing the data. Once the data has been indexed it is compared with data on the disk. To speed transmissions, the appliance doesn’t send data segments that have already been transferred across the WAN; a reference is sent instead. Even if a file is emailed first, and later posted to a file server, the appliance can recognise the content and avoid sending it across the WAN.
OIG tested the scalable data-referencing technology by sending 150MB files repeatedly. Data files previously processed by the appliance could be transferred in seconds, not minutes, even if the file-name was different or some data elements had been changed, says Finucane.
The appliance also reduces the number of TCP packets required to transfer data. It intercepts TCP requests and repacks the TCP payload with references to data on the other end of the WAN, reducing the number of round trips generated by TCP. The Riverbed technology minimises latency by reducing unnecessary chatter in applications such as Microsoft Exchange, he says.
Built-in management features let the OIG monitor system conditions, as well as track traffic reductions. It sees the greatest reduction in Microsoft Windows Common Internet File Sharing transfers, but HTTP, Messaging API and SMTP traffic are also reduced. Overall, the data traversing the network is reduced to about one-tenth of what it would be without WAN optimisation, he says.
Looking ahead, the OIG is keeping tabs on Riverbed’s plans to develop client-emulation software, to let mobile staff achieve some of the same throughput improvements from laptops. The software would increase throughput and make even a dialup line efficient for people on the road performing investigations and audits, says Finucane.
Finucane is glad he didn’t go the add-more-bandwidth route. In retrospect, he says, OIG might not have experienced any improvement bumping up line speeds, and the expense would have been higher. “Doubling the T-1s in our field offices and adding more T-1s in our regional offices would probably have been the same as the cost of the first-year investment in the Steelheads,” Finucane says. Because OIG can avoid paying the cost of increased bandwidth in the future that’s money saved each year.