Supercomputer Made in Jülich Setting New Benchmarks

Forschungszentrum Jülich and its partners have built the most powerful supercomputer in Europe with unparalleled flexibility and energy efficiency.

Jülich, 16 November 2020 – Forschungszentrum Jülich’s upgraded supercomputer JUWELS is now capable of 85 petaflops. This is equivalent to 85 quadrillion computing operations per second or the computing power of more than 300,000 modern PCs. Thanks to its new booster module, JUWELS is able to massively expand the application limits of simulations and also offers the strongest platform in Europe for the use of artificial intelligence (AI). Developed by Forschungszentrum Jülich, Atos, a global leader in digital transformation headquartered in France, the Munich-based supercomputing specialist ParTec, and the accelerated computing platform company NVIDIA, JUWELS is the fastest computer in Europe.

It currently ranks number 7 on the TOP500 list of the world's fastest computers published today. The Jülich supercomputer financed by the national Gauss Centre for Supercomputing takes third place in the current Green500 and is the world’s most energy-efficient supercomputer in the highest performance class.

Video: Flexible supercomputer made in Jülich (Duration: 4:01 min)

“We see supercomputing not only as the subject of our research, but above all as a powerful tool that helps us to address complex research topics together with our partners from science and industry,” said Prof. Wolfgang Marquardt, Chairman of the Board of Directors of Forschungszentrum Jülich.

“With its fully expanded JUWELS system, Forschungszentrum Jülich provides scientists from a wide range of institutions and scientific disciplines with access to supercomputing capacities of the highest level. At the same time, however, the system also demonstrates our responsible action with respect to the ever increasing energy demand for the provision of computing power.”

Bridging the gap between simulations and reality

“A very recent example in the current COVID-19 crisis is supporting simulations for drug development," said Prof. Thomas Lippert, head of the Jülich Supercomputing Centre (JSC). "Only the computing power of the booster enables our researchers to simulate the processes before, during and after a potential drug meets a receptor or protein realistically enough."

Wirkstoffforschung
Wirkstoffforschung auf biologisch relevanten Zeitskalen
Forschungszentrum Jülich, HHU Düsseldorf / C. Pfleger, H. Gohlke

Another current project is the detailed simulation of surface, earth, and groundwater movements. With the new JUWELS booster module, researchers are for the first time able to perform simulations with the required fine resolution, depicting details such as individual slopes or river corridors.

 
 

Early Access Applications

Computing time on the JUWELS booster is precious. Already during the first test phase in late summer and autumn of this year, the first applications ran on the system. The so-called "Early Access Applications" provide useful information for code optimization and first scientific results.

Read more...

Intelligent task sharing – highest energy efficiency

JUWELS is based on a highly flexible modular architecture developed by Forschungszentrum Jülich together with European and international partners. “With its powerful, highly efficient graphics processors, the booster module is designed for extremely computationally intensive applications that can be easily processed in parallel on a large number of computing cores,” said Dr. Dorian Krause, who is responsible for setting up and operating the extraordinarily complex system at Jülich. “JUWELS is also the leader in terms of energy efficiency among the top 10 fastest computers in the world.”

NVIDIA A100 Tensor Core GPU
Forschungszentrum Jülich / TRICKLABOR

JUWELS is one of the first supercomputers worldwide using NVIDIA A100 Tensor Core GPUs, based on the NVIDIA Ampere architecture. The booster unites around 12 million CUDA cores (FP64) across its more than 3,700 graphics processors, connected via an NVIDIA Mellanox HDR 200Gb/s InfiniBand high-performance network. The booster alone reaches a peak performance of 73 petaflops. With nearly 2.5 exaflops of peak AI performance, i.e. 2.5 trillion computing operations per second, it offers the strongest platform in Europe for the use of artificial intelligence (AI).

“The key highlight of JUWELS is that both modules – the previous “cluster module”, which works with fast processors (CPUs), and the booster module with its GPUs – are very tightly interconnected,” says Bernhard Frohwitter, CEO of the Munich-based supercomputing specialist ParTec. The interaction of the modules controls ParTec's modular software system ParaStation Modulo, a world-leading development from Germany. “With ParaStation Modulo, JUWELS can dynamically access CPUs and GPUs within a code at will and thus optimize the calculation.”

Wasserkühlung im JUWELS-Superrechner
Forschungszentrum Jülich / TRICKLABOR

“Both modules come from Atos, based on our BullSequana X infrastructure, whose 100% highly-efficient water-cooled patented DLC (Direct Liquid Cooling) solution contributes significantly to the low energy consumption of the system.” explains Agnès Boudot, Senior Vice President, Head of HPC & Quantum at Atos. “Atos’ design ensures that the fullest computational power issued from CPU and GPU blades is translated into users’ applications.”

Prepared for future technologies

Prof. Thomas Lippert sees the JUWELS system also as a milestone on the path towards the European exascale computer, which is set to be launched in 2023. The construction and operation of such a supercomputer is regarded throughout the world as the next major step in supercomputing. With a computing power of at least one exaflops, i.e. 1 trillion double precision floating point operations per second, it would be at least 12 times faster than the JUWELS booster.

“JUWELS' modular architecture, the design of its computing nodes, its network, infrastructure, and cooling system as well as its software architecture can be transferred to an exascale computer in the next years, while keeping costs and energy consumption at a reasonable level,” said Thomas Lippert. As far as the future of supercomputing is concerned, thanks to its modular design, JUWELS is perfectly prepared to integrate future technologies being researched at Forschungszentrum Jülich — for example quantum computer modules or neuromorphic modules that work on the model of the human brain.

JUWELS-Supercomputer in der Rechnerhalle des Jülich Supercomputing Centre
Forschungszentrum Jülich / Wilhelm-Peter Schneider

Funded by federal and state governments

The acquisition of the booster is being funded by the federal government and the state of North Rhine-Westphalia. JSC operates JUWELS as a member of the Gauss Centre for Supercomputing (GCS), the association of Germany’s national supercomputing centers: three computing centers of Forschungszentrum Jülich (JSC), the Bavarian Academy of Sciences (LRZ) and the University of Stuttgart (HLRS).

Computing time on JUWELS is allocated on a national and European level after application and scientific evaluation. GCS and Forschungszentrum Jülich are supported by the Federal Ministry of Education and Research (BMBF) and the Ministry of Culture and Science of the State of North Rhine-Westphalia as well as the Ministry of Science, Research and the Arts of Baden-Württemberg and the Bavarian State Ministry of Education, Science and the Arts.

 
 

Jülich concept

The modular concept realized at the Jülich Supercomputing Centre (JSC) was developed as part of a long-term cooperation with the Munich software company ParTec. It provides a platform for a supercomputer consisting of several specialized modules that can be dynamically combined as required using uniform software. Since 2011, European partners from industry and research have been developing and testing the first modular design systems under Jülich leadership, and the concept has been continuously expanded in the EU-funded Deep research projects.

The booster is the result of a collaboration between the experts at JSC and the supercomputer manufacturer Atos (France), the supercomputing specialist ParTec (Germany), and the accelerated computing and networking platform company, NVIDIA (USA).

Specifications:

 

JUWELS-Cluster

JUWELS-Booster

Performance

12 petaflops (12 quadrillion computing operations per second)

12 petaflops (12 quadrillion computing operations per second)

Compute nodes

2511 CPU nodes + 56 GPU nodes

936 GPU nodes

Processors

total of 5134 CPUs (Intel Xeon Skylake) + total of 224 GPUs (NVIDIA V100)

total of 1872 CPUs (AMD EPYC Rome) + total of 3744 GPUs (NVIDIA A100)

Cores

122,768 CPU cores + 71.680 FP64 CUDA cores (GPUs in total)

44,928 CPU cores + 12,939,264 FP64 CUDA cores (GPUs in total)

Main Memory

total of 275 TB

total of 479 TB + total of 150 TB High Bandwidth Memory

Network

100 Gb/s (Mellanox InfiniBand EDR)

200 Gb/s (NVIDIA Mellanox HDR InfiniBand)

Supercomputer for Highly Complex Calculations and AI

JUWELS made huge gains in computing performance thanks to a new booster module. Researchers from all over Europe have been adapting and developing programs over recent months so that they run massively parallel on the more than 3,700 graphics processors (GPUs) in the booster.

Read more...

Further information:

Download press release (pdf)

Press release dated November 14, 2019, A turbocharger for the supercomputer JUWELS

Jülich Supercomputing Centre

TOP500 list of the fastest computers in the world

Images:

Images may be used for editorial purposes provided credit is given.

Forschungszentrum Jülich / Wilhelm-Peter Schneider
Forschungszentrum Jülich / Wilhelm-Peter Schneider

Forschungszentrum Jülich / TRICKLABOR
Forschungszentrum Jülich / TRICKLABOR

Contacts:

Prof. Dr. Dr. Thomas Lippert
Director of the Jülich Supercomputing Centre
Tel: +49 2461 61-6402
Email: th.lippert@fz-juelich.de

Dr. Dorian Krause
Jülich Supercomputing Centre
Tel: 02461 61-3631
Email: d.krause@fz-juelich.de

Press contact:

Tobias Schloesser
Corporate communications
Phone: +49 2461 61-4771
Email: t.schloesser@fz-juelich.de

Last Modified: 10.11.2022