Cloud computing and flash-based storage are pushing each other forward as users of internet-based services such as social networks demand near-real-time access to the growing mountain of data those services access.
Executives at web-based companies such as MySpace and Facebook say flash storage technologies are vital to the future of businesses like theirs, which must deliver data to thousands of users simultaneously.
"In the last 20 years, spinning disk really hasn't gone any faster, and right now we're really on the cusp of a change with flash technologies," said Richard Buckingham, MySpace's vice president of technical operations, at The GigaOM Network's Structure 09 conference in San Francisco earlier this year.
At the same conference, Jonathan Heiliger, vice president of technical operations at Facebook, predicted that "flash is going to have a very, very significant effect on not just storage, but infrastructure as a whole. I think it's going to have at least as significant an impact as going from single-core to multicore CPUs," he said.
Flash storage is faster than hard disk drives because it doesn't need to spin a disk to get to a particular bit of data. With flash technologies such as solid-state disk drives (SSD) and PCI Express flash cards, it's possible to read data anywhere in a storage device in less than a millisecond, compared with several milliseconds on a traditional hard disk drive. SSD and flash storage systems also take up less space and use less power than spinning disks.
The datacentres of cloud-based companies are so big that all of those benefits really matter, says Andrew Reichman, an analyst at Forrester Research.
Although corporate IT shops could also take advantage of the performance gains, the high cost of flash technology will likely continue to blunt its progress in enterprise IT, according to some analysts. IDC estimates that SSD storage costs as much as 25 times more than spinning disks on a per-gigabyte basis, says Jeff Janukowicz, an analyst at the firm.
Nonetheless, the growing needs of large-scale web-based companies for better performance, improved capacity utilisation and lower power consumption will help drive up enterprise SSD sales by an average of 165% annually until 2013, IDC predicts. Gartner also expects a surge in flash use, projecting 2009 sales of 281,000 SSDs, up from 59,000 last year.
In addition to fast access to data stored in the cloud, Facebook says flash storage can provide "tremendous" gains in reliability while using significantly less power than other storage systems, Heiliger said at the Structure 09 conference.
MySpace expects that flash storage will save space in datacentres while still supporting the fast page loads that users demand, according to Buckingham. By replacing disk drives with flash technology, MySpace can use 1U servers instead of taller 2U models, which would save a lot of space in a company whose datacentres collectively occupy 60,000 square feet.
MySpace also hopes to one day use flash technology to cache frequently used data and to maintain indexes for searches, Buckingham said at the conference.
However, the social networking company won't rely on flash for persistent data, such as the pictures users post on their pages. Only about one-twentieth of the company's data would ever be stored on flash. "I'm never going to write something to an SSD and hope it lasts forever," Buckingham said.
MySpace is working with flash vendors to establish baselines for performance and reliability, he added.
Hoping to take advantage of the growing demand, several top storage vendors have unveiled flash technologies in the past year.
EMC added SSD technology to several of its storage offerings, while IBM executives have said that flash should be available for all of the company's enterprise storage platforms by the end of this year. Meanwhile, Hewlett-Packard offers SSD technology for its high-end XP storage arrays and mid-range Enterprise Virtual Arrays, as well as flash cards made by Fusion-io that fit into HP servers.