PRACE Course: Deep Learning and GPU Programming Workshop @ LRZ 2020

GROUP PICTURE


OVERVIEW

Learn how to train and deploy a neural network to solve real-world problems, how to generate effective descriptions of content within images and video clips, how to effectively parallelize training of deep neural networks on Multi-GPUs and how to accelerate your applications with CUDA C/C++ and OpenACC.

This 4-days workshop  combines lectures about fundamentals of Deep Learning for Multiple Data Types and Multi-GPUs with lectures about Accelerated Computing with CUDA C/C++ and OpenACC.

The lectures are interleaved with many hands-on sessions using Jupyter Notebooks. The exercises will be done on a fully configured GPU-accelerated workstation in the cloud.

The workshop is co-organized by LRZ, IT4Innovations and NVIDIA Deep Learning Institute (DLI) for the Partnership for Advanced Computing in Europe (PRACE). LRZ as part of GCS and IT4Innovations are both PRACE Training Centres which serve as European hubs and key drivers of advanced high-quality training for researchers working in the computational sciences.

NVIDIA DLI offers hands-on training for developers, data scientists, and researchers looking to solve challenging problems with deep learning.

 All instructors are NVIDIA certified University Ambassadors.


NVIDIA Deep Learning Institute

The NVIDIA Deep Learning Institute delivers hands-on training for developers, data scientists, and engineers. The program is designed to help you get started with training, optimizing, and deploying neural networks to solve real-world problems across diverse industries such as self-driving cars, healthcare, online services, and robotics.


TRAINING SETUP

To get started, follow these steps:

  1. Create an NVIDIA Developer account at http://courses.nvidia.com/join Select "Log in with my NVIDIA Account" and then '"Create Account".
  2.  Make sure that WebSockets works for you:
    • Test your Laptop at http://websocketstest.com
    • Under ENVIRONMENT, confirm that '"WebSockets" is checked yes.
    • Under WEBSOCKETS (PORT 80]. confirm that "Data Receive", "Send", and "Echo Test" are checked yes.
  3. lf there are issues with WebSockets, try updating your browser.
    We recommend Chrome, Firefox, or Safari for an optimal performance.
  4. Visit http://courses.nvidia.com/dli-event and enter the event code provided by the instructor.
  5. You're ready to get started. Please complete the survey at the end of the course to share your feedback.


1st day: Fundamentals of Deep Learning for Multiple Data Types

Lecturer: PD Dr. Durillo Barrionuevo (LRZ)

AGENDA

10:00-10:20  Welcome and Intro

10:20-12:00  Introduction to CNNs and Object Segmentation

12:00-13:00  Lunch Break

13:00-14:20  Word Generation with RNNs

14:20-14:30  Coffee Break

14:30  Group Picture

14:30-15:45  Image Captioning by Combining RNNs and CNNs

15:45-16:00  Q&A

SLIDES

dli-intro-2020-v3.pdf

dli-mdt-2020.pdf

DOCUMENTATION

Python 2.7


2nd: Fundamentals of Accelerated Computing with OpenACC

Lecturer: Dr. Volker Weinberg (LRZ)

AGENDA

10:00-10:15  Intro

10:15-12:00  Profiling

12:00-13:00  Lunch Break

13:00-14:20  OpenACC Directives

14:20-14:30  Coffee Break

14:30-15:45  GPU Programming

15:45-16:00  Q&A, Final Remarks

SLIDES

DOCUMENTATION


3rd day: Fundamentals of Accelerated Computing with CUDA C/C++

Lecturer: Dr. Momme Allalen (LRZ)

AGENDA

10:00-10:15  Introduction CUDA C/C++

10:15-12:00  Accelerating Applications with CUDA C/C++

12:00-13:00  Lunch Break

13:00-14:20  Managing Accelerated Application Memory with CUDA unified memory and nsys

14:20-14:30  Coffee Break 

14:30-15:45  Asynchronous Streaming and Visual Profiling for Accelerated  Applications with CUDA C/C++

15:45-16:00  Q&A

SLIDES

CUDA-Intro.pdf

DOCUMENTATION

CUDA TOOLKIT

PRACE Best Practice Guide - GPGPU

NVIDIA Nsight System


4th day: Fundamentals of Deep Learning for Multi-GPUs

Lecturer: Georg Zitzlsberger (IT4Innovations)

AGENDA

10:00-10:15  Intro

10:15-12:00  Stochastic Gradient Descent

12:00-13:00  Lunch Break

13:00-14:20  Introduction to Distributed Training

14:20-14:30  Coffee Break

14:30-15:45  Algorithmic Challenges of Distributed SGD

15:45-16:00  Q&A

SLIDES

-

PRACE Survey

  • Please fill out the PRACE online survey under https://tinyurl.com/dli-survey
  • This helps us and PRACE to
    • increase the quality of the courses,
    • design the future training programme at LRZ and in Europe according to your needs and wishes,
    • get future funding for training events,
    • shape the future system architecture at LRZ.


Information on ZOOM


NEXT STEPS

Visit the NVIDIA Deep Learning lnstitute's website at http://www.nvidia.co.uk/dli to access more training and resources.

  • Start online, self-paced training in deep learning and accelerated computing (using the account you created today).
  • View upcoming workshops around the world and request an onsite workshop at your company or organization.
  • Learn about the University Ambassador Program.

Ready to kick off a deep learning project or already working on one? Choose the best software and hardware solutions at
http://www.nvidia.co.uk/deep-learning-ai/developer/


Screen Shot 2017-12-13 at 12.24.46       PRACE