By: Michael Feldman

In 1999, Peter Braam introduced Lustre, an open-source parallel file system which went on to become one the most popular software packages for supercomputing. Braam’s success with Lustre was just the start of a career in which he founded five companies and guided technology development in numerous others -- Sun Microsystem, Red Hat, and Xyratex, among them. He’s currently working with Cambridge University on the Square Kilometer Array project and its Science Data Processor effort.

Braam’s newest venture is Campaign Storage, a startup he founded in 2016, along with Spectra Logic CEO Nathan Thompson. The company has come up with a novel tiered  storage management platform for HPC and other environments where archival storage performance and scalability are paramount. Its first product was unveiled at SC16 last November.

Recently, we got the opportunity to ask Braam about the underlying technology and what kinds of problems Campaign Storage purports to solve. We also asked him to offer some perspective on Lustre and where that technology might be headed.

TOP500 News: Let’s start off with the Lustre file system, a technology you obviously have a lot of familiarity with. What do you think about the state of Lustre today? Has it reached some sort of stasis? And what do you think of its future role in terms of upcoming developments like exascale computing?

Peter Braam: The creativity and push for Lustre by virtually all major storage vendors, customers and developers is far beyond my wildest dreams. It's doing extremely well. Many important changes are being made to Lustre, for example, delicate improvements for data management, quality of service and new performance changes.  

For the product, there will be new transitions, and I'm sure that they can be handled as well as previous challenges. I think I gave Lustre a good start, with amazing help and a little drama. But many, many others have made all contributions for a very long time now.

Exascale requires dramatically new technology, particularly to deal with the fastest I/O. However, tiers are inevitably part of the storage infrastructure now, and Lustre is likely to continue to play a key role in one of the tiers. Campaign Storage is an example of this.

TOP500 News: Do you think Lustre has grown beyond its traditional high performance computing roots?

Braam: One of the major internet payment providers told me they run real-time fraud checks for payments relying on Lustre. ISPs reported distributing video content on a truly massive scale with a Lustre-based CDN. Companies sell Lustre for mission-critical environments with a 15-minute support response window. There are hundreds of users like this. I think that likely Lustre has grown beyond its HPC roots, and perhaps a centralized collection of success stories could help all vendors. 

TOP500 News: So, let’s turn to your new venture, Campaign Storage. To start with, what’s the origin of the technology and how did the “Campaign” nomenclature arise?

Braam: Campaign Storage has space for everything and is a storage system for tiers. Los Alamos National Laboratory (LANL) invented the technology, and the name "campaign" refers to a period of time during which data is involved in processing. Data movement between the fastest tiers to a middle tier is required. This middle tier also turns out to be a much better place to do data management. 

TOP500 News: You refer to Campaign Storage as “a new tier.” Where does it fit into the storage hierarchy, and what problem is it designed to solve?

Braam: The cost of bandwidth, that is dollar per GB/sec, increases from the fastest tiers to the slower tiers by three to four orders of magnitude. The cost of storage capacity increases in the opposite direction. New tiers have increased bandwidth by more than 10x already and this will continue with technologies like 3D XPoint. Campaign Storage has much more affordable capacity than a fast tier and much higher bandwidth than an archive. As a result, it can absorb much data from the faster tiers and hold so much that analytics data is more comprehensive.

TOP500 News: Can you briefly describe how the technology works?

Braam: It's surprisingly simple. A simple filter file system manages file metadata in multiple metadata repositories built on existing file systems, for example, ZFS with Lustre. Without customization, it just uses the namespace and extended attributes. The file data can reside in commodity object stores, like S3, tape object stores like Spectra Logic Black Pearl or CEPH or Swift, or an offering from a plethora of vendors. Using mature backends allowed LANL to take it into production remarkably quickly.

It works together only because the use cases focus on migration in and out, and not, for example, on parallel I/O for computing. We are adding search and indexing technology that works with the ZFS backend and Lustre, and we are adding new data movers.

 

 

TOP500 News: How do containers fit into this?

Braam: Containers are the natural capacitor – a must-have – to allow fast tiers to do fine-grained I/O, while not burdening slower tiers with the same I/O workload. The costs make this a necessity. Similarly, the analytics we create can be computed at an appropriate granularity, in fast containers, and can migrate without re-computation to Campaign Storage.

Much new container technology will be needed for exascale and DAOS, and commercial products from DDN and Cray are the leading candidates the moment. We focus on the next tier. Campaign Storage is the right place to manage this container data, or file collections, longer term.

TOP500 News: As far as the company goes, what kinds of customers are you looking to attract?

Braam: We try to help those who need more capacity at a lower cost or a simpler, powerful archiving infrastructure. Many companies have shown interest. We target not only HPC sites, but also mainstream data centers who face cost challenges, sometimes with premium NAS in their fastest tiers. Spectra Logic CEO Nathan Thompson and I were guided by this feedback about companies' needs as we defined the goal of the company. 

TOP500 News: So what’s the next step for Campaign Storage – both the technology and the company?

Braam: The product has been under development since Supercomputing 2016. That's not so long, and a quite a bit remains to be done. That is the top priority for me and our killer team. Like Lustre, Campaign Storage is a software solution and many system vendors can become partners. Developing such relationships and working with customers to define the technology is the business focus for now. That network will lead to success.