New Flatiron Institute Supercomputer the Most Power-Efficient Ever Built

A new supercomputer operated by the Flatiron Institute in New York City tops the latest Green500 List of the most power-efficient supercomputers in the world.

The revised list, which ranks supercomputers based on the number of floating-point operations per second (or ‘flops’) per watt of power, clocks the new supercomputer at 65.091 billion flops per watt. That edges out the previous record holder, which managed 62.684 billion flops per watt. At 2.038 million billion flops, the new Flatiron Institute supercomputer is also the 405th most powerful computer in the world.

“This supercomputer opens up opportunities for doing new kinds of science,” says Ian Fisk, co-director of the Flatiron Institute’s Scientific Computing Core. “This is a workhorse machine, and we’re going to let our researchers try new things and drive discoveries.”

Researchers at the Flatiron Institute will tap the new supercomputer’s power to tackle thorny problems in computational astrophysics, biology, mathematics, neuroscience and quantum physics. The system uses the NVIDIA accelerated computing platform, which is well-suited for machine learning applications, such as multibody simulations of the universe’s evolution, predicting how proteins fold and function, and finding correlations in genomic studies. Accelerated computing platforms also excel at linear-algebra calculations that simulate how electrons behave at the quantum scale. That’s because such platforms use graphical processing units (GPUs) to run many more calculations in parallel than traditional central processing units (CPUs) can.

Housed in a data center in New Jersey, the system itself was built by Lenovo and leverages the efficiency capabilities of the company’s ThinkSystem SR670 V2, a server designed to install easily into traditional data centers and to be accessible to more researchers. Its efficient design effectively keeps things cool even in a high-heat, multi-GPU environment. In its operating configuration, the machine boasts 20 nodes connected with the NVIDIA Quantum InfiniBand networking platform, with each node containing four NVIDIA H100 Tensor Core GPUs. (During the efficiency benchmarking, the machine used 10 nodes, each with eight GPUs.)

The transistors on the NVIDIA chipsets measure just 4 nanometers wide, which means that more computational muscle can be packed on the same size chip. That smaller transistor size and many other optimizations have helped reduce the new supercomputer’s energy consumption and maximized its performance efficiency.

Lowering a supercomputer’s power usage without sacrificing performance comes with many benefits. Electricity is a significant component of a machine’s operating costs and environmental impact. Every watt consumed is converted into heat that must be removed from the system, requiring even more power. Moreover, more energy-efficient systems can slot into existing data centers without needing costly electrical upgrades.

Fisk says the number one power-efficiency ranking wasn’t the primary goal. “This computer will allow us to do more science with smarter technology that uses less electricity and contributes to a more sustainable future,” he says. “That’s what’s important to us.”

The new Flatiron Institute supercomputer supplants Frontier Test & Development System (TDS), located at Oak Ridge National Laboratory in Tennessee, on the power efficiency list. Frontier TDS set an extremely high bar on the June 2022 list, at 62.684 billion flops per watt, while the main Frontier system — which broke the exaFLOP (1 billion billion flops) barrier and remains the fastest supercomputer in the world — checks in at 52.227 billion flops per watt. Unseating Frontier presented a daunting challenge, Fisk says.

While both are record setters, the new Flatiron Institute supercomputer is a markedly different beast from Frontier, which was laboriously designed and built to reach 1 exaFLOP. Conversely, the new Flatiron Institute system utilizes commonly available systems, GPUs and networking, making it “very high performance and very efficient without being particularly exotic,” Fisk says. “It only took a couple of people to load the system in. This kind of efficiency is now accessible to a lot more groups, rather than just the largest supercomputing centers.”

The obtainability of the technology behind the new supercomputer represents a shift in the industry.  “Lenovo is committed to making the same exascale-ready technologies available for users of all sizes and across all industries,” says Kirk Skaugen, executive vice president of Lenovo and president of the company’s Infrastructure Solutions Group. “Our collaboration with Flatiron and NVIDIA demonstrates a pivotal milestone in enabling supercomputing technology at nearly any scale and supporting smarter technology for all.”