Three weeks on and the Ministry of Health is still beset by problems caused by the Downadup worm, otherwise known as Conficker.
The Ministry is, according to Microsoft, the only large organisation in New Zealand that has been affected by the worm, which has reportedly hit up to 15 million PCs worldwide.
"There have been a couple of minor issues in New Zealand," says Microsoft national technology officer Brett Roberts.
"The ministry doesn't have a support contract with us. They contacted us and we spent 20 to 30 hours with them on a goodwill basis.
"A patch was made available [on October 26], which highlights the importance of having a robust process."
Computerworld has been contacted, both anonymously and directly by people connected to the ministry, all of whom claim that the problems are largely due to a failure to provide adequate maintenance and patching.
One detailed email, received anonymously, asked why the systems at the ministry were so "back level" and why basic maintenance, such as patching, was not performed. It told us that promises that services would be restored were repeatedly broken.
It also asked why systems storing sensitive and private information, including pharmaceutical prescribing, child health check results, mental health records, and index information to match the data to individual people, were not secured to basic levels.
As a result of that correspondence, Computerworld posed questions to the ministry about the age of its operating systems and its patching and security policies. We also asked if the source of the infection had been identified and how the ministry fought the infection.
We asked what the ministry had learned from the exercise and what, if any, processes would be changed as a result.
We asked who was supplying maintenance and support, given the ministry didn't have a support contract with Microsoft. We asked whether this arrangement was up to date and whether Health would be taking any action against its supplier.
The response received last week from the ministry's public relations unit was that they were good questions but wouldn't be answered for another week when the ministry proposed to analyse what had happened.
"The Ministry of Health is not in a position to answer your questions this week as most of the questions will be included in a post-incident review which is scheduled to take place next week," the ministry wrote.
"Alan Hesketh is happy to answer your questions and discuss the ministry's experiences and learnings following the review process.
"At the moment, the internet is being progressively opened to a number of staff. We believe we are now at the stage of 'cleaning the tail end of the work'; this involves double-checking that all computers and laptops are cleaned and updated. When we are 100% sure that all devices have remained clean for a set period of time, internet access will be opened up for all staff."
The Ministry's 2,000 PCs were affected. Alan Hesketh, deputy director general of the health information directorate, said in an earlier interview with Computerworld that the final 80 of these should be confirmed as clean on Thursday January 15.
"We've been using a number of different anti-virus vendors," Hesketh says. "No one particular vendor has given one answer. It's a combination."
"It's particularly smart malware with a number of different vectors." "It's likely we were infected during December and that this version was activated in January," he says. "We detected it when we returned to work."
He says the worm communicates with the internet and generates a series of domain names each day. "It's likely to bring down a payload from those websites and bring down new variants."
All 12 Ministry locations across the country were affected, but Hesketh says the worm didn't get out to other health-related sites.
"We cut off the internet last Thursday [January 8] so it couldn't get out. It's not spread via email and our external email was up the whole time.
It seems more than significant that Health was, according to Microsoft, the only organisation of any size in New Zealand affected by the worm.
An internal document, issued Friday week by Health's IT service desk, shows the services that were taken out due to the internet not being available. These include remote laptop replication, remote connection to the ministry's network, access to the internet from the ministry or any ministry device, including the use of a Vodafone or Telecom card, internal access to the ministry websites, and any other applications that are available via the internet to the ministry, web mail, use of portable devices (USB sticks, CD, iPods, MP3 players and any other media-type player) to download a file without being scanned first, synchronisation with PDAs, Quicker and Webtrends.
Staff were told they were barred from dialling in and connecting to the ministry's network.
"You will not be able to receive or send data to any ministry server while working outside the ministry's offices. You can work on your local drive," it said.
They were also advised to check their home PCs and to ensure that personal devices had up-to-date anti-virus and firewall software and had been recently scanned.
"Staff in the office, up to their ears, and looking a little stressed," the advisory said.
Meanwhile, another anonymous correspondent from within Health wrote saying we had to get to the bottom of some of the "ambiguity" in the ministry's statements to date. One Computerworld article said the Proclaim payments system had not been disrupted.
"This obviously wasn't the case and it seemed likely that someone was taking the position that as long as payments are made within service level agreements, then all is well. Which is true, since the SLAs aren't particularly demanding, but doesn't reveal the true extent of Health's recent IT failures," our correspondent wrote.
"There is some speculation that a USB memory stick introduced the worm in to Health. No one will ever know for sure, but this is mainly a Remote Procedure Call (RPC) worm where a PC is infected via an open port with a vulnerable (unpatched... in this case patch MS08-087 was the one required) service running. I'd put money on it being introduced via an infected laptop that someone had taken home over the break and plugged into their ADSL connection. Once reconnected to the Health network it would have infected everyone within minutes."
The source says protection is usually achieved by applying several layers of security technology, technology such as patching, anti-virus or a desktop intrusion detection product, a personal firewall, and network access control.
"Health's problems were caused by its inability to do even the simplest and most essential of these... using a supported operating system and keeping it patched. I think that an organisation handling some of the most sensitive private information in NZ should have higher standards."
Old-fashioned gateway security is no longer enough to manage the myriad threats and attacks now in the wild, our correspondent said. It has been made obsolete and replaced by a model where every device protects itself, no matter where it is or what it is connected to.
We await the ministry's post-incident review.