New Zealand is home to what Scott Houston, former CTO at Weta Digital, claims is the largest assembly of processors offered for “on demand” outside use in the world — the Weta Digital array, now over 1,000 blade processors strong, each clocking 2.8GHz.
Weta Digital creates digital graphics, particularly special effects for movies, and was instrumental in the Oscar-rated success of the Lord of the Rings trilogy.
University of Auckland researchers are using 48 of the processors to simulate the spread of HIV and its progression to Aids among large populations. Research is also in progress on the Weta array on the genetics of crustaceans and the identification of the species — "and even the individual pod" — of whalemeat samples, Houston says.
Descending from the heights of academic research, the first major project using the array was the design and simulated walkthrough of a $25 million super-yacht, he says.
The processors are being applied to land-based architecture through a partnership with Auckland multimedia and design company Urban Voyage. Architects are even busier than film-makers, Houston says, and it could hold up a job to have to send everyone home at the weekend in order to have anough processing cycles for a big rendering task.
Houston sees such partnerships marrying powerful hardware with specialist knowledge and software as a growing part of the supercomputer market.
Supercomputer-based research is potentially handicapped by the need for secure fast communications, particularly where reference data resides overseas, says Houston. There is probably a case for concluding agreements to import “the basic data” in any domain, such as the sequence of the human genome for biotechnology, so it could be kept available locally.
That the supercomputer is based in New Zealand could have great appeal to members of non-aligned nations, he says. Research establishments and private companies that may have used US-based supercomputer services in the past could well be nervous of surveillance of their traffic into the US as part of the Homeland Security effort, he says.
Grid computing is an emerging potential use of supercomputers and arrays of less powerful machines, with Sun boss Scott McNealy predicting it as the “fourth big wave” of success for the company, most recently in an address to media and analysts on Sun’s California campus in February.
Some problems, such as weather simulation, cannot be split into discrete parts for grid processing and will still require massive parallel processing on a single site, Houston says; but many problems are amenable to grid processing.
For the links of the grid, many organisations simply used the internet, he says, but Supercomputing Centre partner Gen-i and its owner, Telecom, stand ready with virtual private network technology in cases of greater security need.
Federated use of many computers had its early trials on massed ranks of PCs in such projects as SETI, the search for extra-terrestrial intelligent life. “Given sufficient security, there is no reason why [a supercomputing grid] couldn’t use your processors,” he says.