By: Michael Feldman

The new Green500 list of the most energy-efficient supercomputers demonstrates some significant progress from last year. Thanks to the new manycore processors from Intel and NVIDIA that are starting to penetrate the top systems, performance per watt numbers are on the rise.

The top two systems, DGX SATURNV and Piz Daint, both use NVIDIA’s P100 Tesla GPU, which exhibits excellent energy efficiency. The DGX SATURNV, NVIDIA’s first petascale in-house supercomputer, earned the top spot on the Green500 list with a rating of 9.46 gigaflops per watt. That’s more than a 40 percent jump from the number one system just six months ago on the June 2016 list. It owes its superior efficiency to the server upon which it is based: NVIDIA’s DGX-1, which houses eight P100 GPUs.

Piz Daint, a Cray XC50 system, took second place with a mark of 7.45 gigaflops per watt. That system raised its stature on the list due to an upgrade from its original K20x graphics processors to the new P100 GPUs. Although it’s based on the same processor technology as the DGX SATURNV, it uses a lower ratio of GPUs to CPUs. The former number one Green500 system, the Shoubu supercomputer, equipped with the PEZY-SCnp manycore accelerator, slipped into third place with an energy efficiency of 6.67 gigaflops per watt.

 

 

With the exception of the XStream system at number eight, which uses the NVIDIA K80 accelerator, the remainder of the top 10 systems relied on homogeneous manycore processors in a standalone configuration. Five of those machines are powered by Intel’s new “Knights Landing” Xeon Phi processors, which exhibited energy efficiencies of between 3.84 and 5.61 gigaflops per watt. The outlier here is the Sunway TaihuLight supercomputer, the number one system on the TOP500 list, which earned the number four spot on the Green500 with a value of 6.05 gigaflops per watt. Taihulight is powered by the SW26010, a 260-core ShenWei processor, designed and manufactured in China.

Overall, heterogeneous systems continue to demonstrate better energy efficiency than homogeneous systems, but that gap appears to be narrowing with the appearance of standalone manycore processors like Xeon Phi and ShenWei. One interesting trend to watch will be how these manycore architectures continue to evolve, and if the self-hosted ones will be able to catch or overtake coprocessors with regard to energy efficiency.

There has been progress in other areas as well. For the first time in the 10-year history of the Green500, the mean power consumption of all systems on the list actually went down. In the past, due to the growth in the size of the top supercomputers, power consumption rose even as the energy efficiency of those machines dropped. It remains to be seen if the reversal is a one-time event or will start a new trend.

It’s not expected that the topmost systems on the list will make such large gains in energy efficiency over the next six to twelve months as they did during this cycle. The next big jump is apt to occur when the next generation of GPUs, Xeon Phi processors, and perhaps the Chinese manycore processors are available to supercomputer makers in the late 2017/early 2018 timeframe.

 

If we use the most efficient system on the current list (the DGX SATURNV) as a baseline, an exascale system with that level of energy efficiency would draw 105.7 MW. That is about five times the desired goal of 20 MW, but only a little more than three times the 30 MW upper limit of the initial systems targeted under the US exascale effort.

The good news is that we can expect two or three more generations of manycore processors before the first exascale systems are built, so the wattage for the initial systems should be considerably less than 100 MW. Green500 founder Wuchun Feng is optimistic that the numbers are starting to line up. “While reaching a 20 MW exascale is highly unlikely by 2020, China has unofficially indicated that their target power for an exascale supercomputer in 2020 to be 40 MW,” Feng told TOP500 News. “This 40 MW projection, I think, is attainable.”

Regardless of the exascale milestone, HPC systems continue to become more energy efficient with each passing year, thanks in part to Moore’s Law, but also due to a concerted effort on the part of component makers and system engineers, who are squeezing the maximum amount of performance from a given power envelop.

The next Green500 list will be announced in June 2017 in conjunction with the ISC High Performance conference, which will take place in Frankfurt, Germany.