Panasas’s 10x saving bumps NetApp from HPC storage deployment

Australia-based University of Wollongong’s Molecular Horizons Institute, a world leader in cryogenic electron microscopy, has deployed 1.2PB of Panasas scale-out NAS to support high-performance computing (HPC).

The five-year partnership will see Panasas storage hardware replace NetApp equipment that had reached end of life. But Panasas was also chosen for its support and expertise in life sciences as well as cost per TB.

Molecular Horizons’s cryogenic electron microscopy (Cryo-EM) expertise sees it work with partners worldwide. It is based around capabilities that let researchers study high-resolution imagery of frozen specimens of viruses, cellular structures and proteins.

Raw data sets can range from around 5TB to 40TB from imaging equipment that runs for 80% of the year, operating 24/7 and taking between 5,000 and 10,000 images of the same sample to provide visualisations to researchers across the world.

That amounts to large data holdings that currently run to several hundred TB, although partners migrate their own data away after six months on Molecular Horizons' storage.

Molecular Horizons's bulk storage infrastructure had been provided by NetApp. It had reached end of life and the institute was looking elsewhere, said James Bouwer, director of Cryo-EM at Molecular Horizons.

Panasas had existing life sciences expertise, said Bouwer, whereas NetApp didn’t, and had a massive cost-per-TB differential, with Panasas offering 1.2PB for the same price as NetApp offered 100TB.

“For the price we paid – we looked at NetApp as well – we probably could have got 100TB from what we spent,” said Bouwer. “We got 1.2PB and a partner. I think the math there is fairly clear.”

“We faced the prospect of not much support from NetApp. They said, ‘We’ll deliver the system, then it’s yours, take care of it’. Panasas have expertise in life sciences and we’ve worked with them closely,” he said.

“Panasas shipped all the gear, we worked with them, and we built it ourselves because of Covid, but [we’re] supported remotely by them,” he added.

Molecular Horizons has deployed 1.2PB of Panasas ActiveStor scale-out storage, which runs the PanFS parallel file system. It provides multiple tiers of storage based on NVMe solid state and spinning disk capacity.

That deployment is currently in a testing phase, with Panasas providing a great deal of help in architecting what Bouwer describes as a “very complex set of workflows”.

What other benefits does Molecular Horizons expect from moving to Panasas? “We’re going to get really fast access to files, to the active layer,” said Bouwer.

Panasas’s Dynamic Data Acceleration added tiering based on file size to its PanFS scale-out NAS last year. It is a move aimed at providing customers with storage to suit a wide variety of HPC and artificial intelligence/machine learning (AI/ML) workloads by exploiting the speed of SSD for small files and the massed throughput of HDD for large files.

Dynamic Data Acceleration tiers data to different media within the storage system, but not by usage characteristics. Its tiering by file size claims a 2x performance advantage in GBps terms over file system rivals BeeGFS, Lustre and IBM’s GPFS/Spectrum Scale.