By: Michael Feldman

Niagara, Canada’s newest multi-petaflop supercomputer tasked for academic use, is now available to researchers across the country. 

 

Niagara supercomputer. Source: University of Toronto, screen capture

 

The system, which is installed at a facility run by the University of Toronto and is the third in a trio of new multi-petaflop supercomputers that have been deployed in Canada over the past year. According to the announcement, Niagara will deliver over three petaflops of performance, putting it on par with its two academic brethren: Cedar a 3.7-petaflop supercomputer installed at Simon Fraser University, and Graham, a 2.6-petaflop system deployed at University of Waterloo.

All three systems were developed and deployed in collaboration with Compute Canada, a federated consortium of regional academic supercomputing networks: WestGrid, Compute Ontario, Calcul Québec, and ACENET. The new supercomputers, along with handful of smaller systems, were designed to bring the country’s aging supercomputing infrastructure into the petascale realm, providing its academic researchers with an aggregate capacity of at least 12 petaflops. With the University of Toronto system now up and running, that mission is all but complete.

Niagara is comprised of 1,500 of Lenovo’s ThinkSystem SD530 nodes, four of which can inhabit a 4U ThinkSystem D2 Enclosure. An SD530 node houses two of Intel’s Xeon Scalable Processors – in the case of Niagara, a pair of 20-core chips, which presumably means either Xeon Gold 6138 or 6148 processors. The 1,500 nodes are glued together with InfiniBand and are backed by 12 petabytes of external storage. In this instance, the storage is based on Lenovo’s Distributed Storage Solution (DSS-G), the computer-maker’s high performance solution for IBM’s Spectrum Scale file system, formerly known as GPFS.

The ThinkSystem SD530 is the older sibling to Lenovo’s newly announced SD650, a warm-water cooled system will power the upcoming SuperMUC-NG at the Leibniz Supercomputing Center. The SD650 has the advantage of being able to run the fastest and hottest Xeon Scalable Processors, such as the 205-watt Xeon Platinum 8180, at full tilt. The air-cooled SD530 is limited to chips of 165 watts or less. In general, those cooler-running processors offer proportionally less performance, but much better price-performance.

Niagara will support a range of modeling and analytics codes, according to Daniel Gruner, chief technology officer at SciNet, in the University of Toronto. “We support research in diverse fields – from climate science to humanities, astrophysics to life sciences, social sciences to engineering, and physics to chemistry,” said Gruner, in a prepared statement. “We run traditional data processing, simulations, analytics, and increasingly, machine learning as well. The size of the new system, coupled with the ability to easily scale out to the full size of the cluster is crucial for supporting our broad community of scientists.”

One of the Niagara’s first big challenges will be running a research code to discover how the world’s oceans function. The work, which is being led by climate expert Richard Peltier at the University of Toronto, relies on real-time data streamed from sensors on the bottom of the ocean. By using the entire system to run the code, Peltier thinks they may be able to shed light in new areas of ocean science.

“It’s never been done before,” said Peltier. “It could be days or even a week depending on the spatial resolution we decide to work at…. We hope the results will warrant publication and be a major coup for Niagara.”

The $18 million system was funded by the Canada Foundation for Innovation, the Government of Ontario and the University of Toronto.