Deep Learning and GPU programming using OpenACC @ HLRS 2023
Overview
Learn how to accelerate your applications with OpenACC, how to train and deploy a neural network to solve real-world problems, and how to effectively parallelize training of deep neural networks on Multi-GPUs.
The workshop combines an introduction to Deep Learning and Deep Learning for Multi-GPUs with a lecture on Accelerated Computing with OpenACC.
The lectures are interleaved with many hands-on sessions using Jupyter Notebooks. The exercises will be done on the AI partition of HLRS's cluster Hawk.
This course is organised by HLRS in cooperation with LRZ. All instructors are NVIDIA certified University Ambassadors.
1st day Accelerated Computing with OpenACC
Lecturer: Dr. Volker Weinberg (LRZ) / Dr. Momme Allalen (LRZ)
AGENDA (all times in CEST)
09:00-09:30 Welcome & Intro
09:30-11:00 Profiling
11:00-11:15 Coffee Break
11:15-12:15 Introduction into NVIDIA® Nsight™ Systems
12:15-12:45 Lab 1
12:45-13:15 OpenACC Directives
13:15-14:30 Lunch
14:30-14:45 Lab 2
14:45-15:15 GPU Programming
15:15-15:45 Lab 3
15:45-16:00 Coffee Break
16:00-16:30 Data Management
16:30-16:45 Lab 4
16:45-17:00 Q&A, Final Remarks
SLIDES
- dli-intro-hlrs-2023-v1.pdf
- Intro_Nsight Systems.pdf
- Module1.pdf
- Module2.pdf
- Module3.pdf
- Module4.pdf
- Module5.pdf
- Module6.pdf (for self-study)
HANDS-ON MATERIALS
DOCUMENTATION
2nd day: Introduction to Deep Learning
Lecturer: PD Dr. Juan Durillo Barrionuevo (LRZ)
AGENDA (all times in CEST)
09:00-09:20 Welcome and Intro
09:20-11:00 Introduction to Deep Learning
11:00-11:15 Coffee Break
11:00-12:00 How a Neural Network Trains
12:00-13:15 Convolutional Neural Networks
13:15-14:30 Lunch Break
14:30-15:45 Data Augmentation, Deployment and Pre-Trained Models
15:45-16:00 Coffee Break
16:00-16:45 Advanced Architectures
16:45-17:00 Q&A
SLIDES
HANDS-ON MATERIALS
- Jupyter Notebooks: https://github.com/JuanJDurillo/deep_learning_notebooks using especially (install via pip)
tensorflowtensorflow_datasetsnumpypandasscipymatplotlib
DOCUMENTATION
3rd day: Introduction to Deep Learning for Multi-GPUs
Lecturer: PD Dr. Juan Durillo Barrionuevo (LRZ)
AGENDA (all times in CEST)
09:00-09:15 Intro
09:15-11:00 Stochastic Gradient Descent, a focus session on how training works
11:00-11:15 Coffe Break
11:15-12:00 Hands On Session on Gradient Descent
12:00-13:15 Introduction to Distributed Training
13:15-14:30 Lunch Break
14:30-15:00 Hands On Session on Distributed Training
15:00-15:45 Algorithmic Challenges of Distributed SGD
15:45-16:00 Coffee Break
16:00-16:45 Hands on session on Algorithmic Challenges of Distributed SGD
16:45-17:00 Q&A
SLIDES
HANDS-ON MATERIALS
- Jupyter Notebooks: https://github.com/JuanJDurillo/deep_learning_notebooks (see Day 2)