Versity targets Exabyte volumes in data archiving
Versity builds archiving for Exabyte data volumes on tape and in object storage. It has built its own S3 and transitioned to a GPL licence for supercomputing archives
Versity’s archiving products only start at dozens of petabytes of minimum storage volumes, while its installed storage volume has reached several exabytes with no sign of slowing down.
“We actually manage two projects of 1EB, with another in its first phase of five and already at 1.5EB,” said Versity co-founder and president Bruce Gilpin, during a recent IT Press Tour event in the US, attended by Computer Weekly’s French sister publication, LeMagIT.
Versity was founded in 2011, and works with major centres of high-performance computing such as Pawsey Supercomputing Research Centre in Australia. Super-computing supplier Cray has also been a reseller since 2013.
The company reached profitability in 2015 without having to take money from investors. The majority of its funding has come from directors and employees, and that has allowed the company to be independent. In 2021, Dell began distribution of Versity. It also collaborates with Spectra Logic and achieves 50% of turnover in the US, with the remainder in the rest of the world. Growth is organic, without acquisition.
Versity’s efforts focus on three key areas: namespaces that can handle hundreds of billions of copies; parallelism, of data, of metadata and code optimised for tape storage; and a modular architecture that’s easy to install and administer.
The technological underpinnings of Versity’s offerings are built on Scalout Archive Manager (ScoutAM) and open source file system ScoutFS, which can support hundreds of billions of files and objects. ScoutAM automatically directs data to the most appropriate media according to the admin’s preferences in performance or cost terms. The ScoutAM architecture is all-software without the need for specific hardware.
The roots of this archiving system go back to Storage Archive Manager-Quick File System (SAM-QFS), which was launched by Sun Microsystems in 2008 and has since been available on an open source basis. Compared with its previous product, Versity Storage Manager (VSM), also based on SAM-QFS, Versity has rewritten ScoutFS into Golang, which is suited to large-scale distributed environments.
A GPL licence solution
Versity has also developed its own version of S3 for ScoutFS so that it can easily store and retrieve data on a system that’s compatible with its object storage.
Rewriting S3 was never one of Versity’s initial objectives. It was using MinIO, but that supplier changed licensing for its Kubernetes storage system from Apache 2.0 to AGPL v3, with the implication that meant customers who modify code or make additions to suit their needs must present changes to the wider community.
“VSM was using MinIO and we didn’t find a replacement, so we built one,” said Gilpin. “AGPL is a poisoned pill because all the work done also becomes part of AGPL. All the customers that build customised solutions on our product haven’t authorised that.”
The only way for customers that personalise ScoutFS to avoid that is to pay AGPL for a private licence. In fact, ScoutFS is available under a simple GPL licence that means any added functionality doesn’t have to be distributed.
Read more on archiving
- Cloud archiving: A perfect use case, but beware costs and egress issues. The advantages of cloud archiving are ease of use and being well-suited to potential latency issues, but IT teams need to be aware of costs and issues around moving data from and between clouds.
- Key storage choices: Cloud vs tape for archive storage? Tape still has benefits, such as an ‘air gap’ that can insulate archives from threats to data integrity. But what are the opportunities for cloud in tape’s traditional use cases?
