Multi-GPU Programming Bootcamp

This bootcamp covers step-by-step ways to improve application performance using cues from profilers and provides an understanding of the underlying technologies and communication topology needed to leverage powerful NVIDIA libraries to extract more performance from the system.

Scaling applications to multiple GPUs across multiple nodes requires one to be adept at programming models and optimization techniques, and proficient at performing root-cause analysis using in-depth profiling to identify and minimize bottlenecks. This bootcamp covers step-by-step ways to improve application performance using cues from profilers and provides an understanding of the underlying technologies and communication topology needed to leverage powerful NVIDIA libraries to extract more performance from the system.

Please ensure you meet all prerequisites / eligibility before you apply.

Agenda & Content

All times are Central European Summer Time (CEST).

0th day – 28 September 2026

12:00–13:00   Cluster Dry Run Session (optional; please attend in case you need help with login)

1st day – 29 September 2026

08:45–09:00   Join in

09:00–09:15    Welcome to the Multi-GPU Programming Bootcamp (Moderator)

09:15–09:40    Introduction to Nsight Systems and GPU Computing (Lecture and Read-Only Lab)

09:40–10:30    Overview of a Single GPU Code

10:30–10:45    Break

10:45–12:30 Parallelizing a Single GPU Code using CUDA memcpy, Streams, and an Introduction to Single Node Multi-GPU (P2P) (Lecture and Lab)

12:30–13:00    Lunch

13:00–15:00    Introduction to Multi-Node Multi-GPU (GPU Direct) programming (Lecture and Lab)

2nd day – 30 September 2026

08:45–09:00    Join in

09:00–10:55    Introduction to NVIDIA Collectives Communications Library (NCCL) (Lecture and Lab)

10:55–11:10    Break

11:10–13:00    Introduction to NVSHMEM (Lecture and Lab)

13:00–13:30    Demo of the Nsight Systems Profiler Report for a Multi-Node Application

Content level

Entry level

Intermediate

Prerequisites

Experience with C/C++, NVIDIA CUDA, MPI

Target audience

Course for academia, industry, and public administration.

Course format

This course will be delivered as a LIVE ONLINE COURSE (using Zoom).

All communication during the bootcamp will be done through Zoom and Slack.

Hands-on labs

Attendees will be given access to an A100 GPU on one of the supercomputers of the organizers.

Lecturers

Event Moderator:  Claudia Blaas-Schenner (ASC Research Center, TU Wien)

Instructors: Paul Graham (NVIDIA)

Teaching assistants from the participating HPC centres

Language

English

Prices and eligibility

This course is partly funded by the EuroCC 3 project. Therefore, the course is open and free of charge for participants from academia, industry, and public administration from EU or EuroHPC JU member states or from countries associated to the Digital Europe Programme.

Registration

Registration for this training event is handled by Austrian Scientific Computing (ASC).

Please register with your official email address to prove your affiliation.

You will get an automatic confirmation via email (subject starting with "[Indico] Registration"), please check your Spam/Junk folders.

Following your successful registration, you will receive further information approximately 1 week before the bootcamp.

Organizers

This bootcamp is co-organized by the National Competence Centres for Supercomputing in Austria (ACA), Germany (SIDE), and Poland (NCC-PL) together with their supercomputing centres Austrian Scientific Computing (ASC), High-Performance Computing Center Stuttgart (HLRS), Jülich Supercomputing Centre (JSC), Leibniz Supercomputing Centre (LRZ), Academic Computer Centre Cyfronet AGH (Cyfronet), and OpenACC organization and NVIDIA.

Acknowledgements

This course is partially funded by the EuroCC 3 project.

EuroCC 3 has received funding from the European High-Performance Computing Joint Undertaking (JU) under Grant Agreement No. 101306701. The JU receives support from the European Union‘s Digital Europe Programme and Germany, Albania, Austria, Belgium, Bosnia and Herzegovina, Bulgaria, Croatia, Cyprus, Czechia, Denmark, Estonia, Finland, France, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Montenegro, the Netherlands, North Macedonia, Norway, Poland, Portugal, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Türkiye, and Kosovo.