OpenMP Labs June 2026

General Setup

We use the same code as for the OpenACC Lab. The exercises can be found in the file Jacobi.tgz. This file contains both C and Fortran versions of the code.

Open a Terminal in JupyterLab and copy the exercises to your home directory in the JupyterLab via

In the labs you will implement 6 different version of the code for multicore CPUs and GPUs. Note the execution time for each of those versions in the results sheet available online under https://doku.lrz.de/openmp-labs-results-june-2026-2763063802.html

Lab 1: Profiling with Nsight Systems

Check the configuration of the GPUs via the following commands which are provided with the NVIDIA HPC Software Development Kit (SDK) on the compute node:

We are using the NVIDIA HPC SDK C and Fortran compilers for the hands-on.

Compile the code using the following commands:

For C:

For Fortran:

To profile the code run the Nsight Systems command line interface on the compute node via:

This will produce the trace files laplace.nsys-rep and laplace.sqlite. Use "nsys --help" and "nsys --help profile" to get more information about the available profiler options.

For performance reasons, visualisation of the trace files should be always done on the local PCs.

Copy the trace file *.nsys-rep to your local PC by  right-clicking on the file in the left pane of the JuptyerLab and selecting "Download".

Run Nsight Systems on the local PC and open the trace file via File → Open. Zoom in the timeline and also test different views like "Top-Down view". Identify the 2 most time consuming routines of the code.

Further information is available under https://developer.nvidia.com/nsight-systems

Using NVIDIA Tools Extension (NVTX)

For C:

Add #include "nvtx3/nvToolsExt.h" in a copy jacobi-nvtx.c of the source code jacobi.c and wrap parts of the code which you want to capture events with calls to the NVTX API functions. For example, try adding nvtxRangePush("calc") before calling calcNext() and nvtxRangePop() just after calling it. We recommend to wrap calls to the functions initialize(), calcNext() and swap() and also wrap the while-loop in which the last 2 functions are called.

For C compile the NVTX annotated code as before:

For Fortran:

Add use nvtx in a copy jacobi-nvtx.f90 of the source code jacobi.f90 and wrap parts of the code which you want to capture events with calls to the NVTX API functions. For example, try adding call nvtxStartRange("calc") before calling calcNext() and call nvtxEndRange() just after calling it. We recommend to wrap calls to the functions initialize(), calcNext() and swap() and also wrap the while-loop in which the last 2 functions are called.

For Fortran compile the NVTX annotated code and link the library nvhpcwrapnvtx as follows:

Profile again on the compute node using

and visualise the profile data on the local PC.

Further information can be found in the CUDA Profiler User’s Guide https://docs.nvidia.com/cuda/profiler-users-guide/index.html

You can use the NVTX annotated code also for profiling in the next Labs if you want.

Lab 2: OpenMP Directives

Using OpenMP for multicore CPUs with the parallel for directive

For C:

Make a copy laplace2d-multicore-parallel.c of the file laplace2d.c  and consider the following directives at the right place to parallelise the calcNext() and swap() functions:

For Fortran:

Make a copy laplace2d-multicore-parallel.f90 of the file laplace2d.f90  and consider the following directives at the right place to parallelise the calcNext() and swap() functions:

Compile the code for multicore CPUs using the following commands:

For C:

For Fortran:

Run the code on the compute node. You can set the number of threads on the CPU via 

Profiling OpenMP code

Profile the code on the compute node via

OpenMP Support is limited in nsys, however you can consider to compile your code via

and profile it using

Compare with the previous profiling results.

Lab 3: GPU Programming using OpenMP

For C:

Make a copy laplace2d-gpu-parallel.c of laplace2d-multicore-parallel.c.

Compile the code using the following commands

For Fortran:

Make a copy laplace2d-gpu-parallel.f90 of laplace2d-multicore-parallel.f90.

Compile the code using the following commands

Check if the code compiles and runs correctly on the GPU.

Consider the following directives at the right place to parallelise the calcNext() and swap() functions on the GPU:

For C:

For Fortran:

Check if the code compiles and runs correctly on the GPU.

Modify the code to specify the sizes of the arrays to be copied from/to the GPU correctly. The sizes of the arrays can be specified with the  map clauses of the target directive, e.g.:

In C:

In Fortran:

For 2D arrays in Fortran, array sections must be specified as  array(start_index1:end_index1, start_index2:end_index2). If the whole array is used in OpenMP data mapping, it can be written simply as e.g.  map(to: array) without specifying bounds in Fortran.

Run and profile both versions of the code on the compute node and compare with the previous profiling results.

Consider using managed memory with OpenMP by compiling the same code with

For profiling GPU code use

Lab 4: Data Management

OpenMP Structured Data Directive

For C:

Make a copy jacobi-gpu-data-structured.c of the file jacobi.c  and include the following directive at the right place with the right map clauses to create a data region:

#pragma omp target data <data clauses>
{
...
}

Compile the code using the following commands

Run and profile the code on the compute node and compare with the previous profiling results.

For Fortran:

Make a copy jacobi-gpu-data-structured.f90 of the file jacobi.f90  and include the following directive at the right place with the right map clauses to create a data region:

!$omp target data <data clauses>
...
!$omp end target data

Compile the code using the following commands

Run and profile the code on the compute node and compare with the previous profiling results.

OpenMP Unstructured Data Directives

For C:

Make a copy laplace2d-gpu-data-unstructured-parallel.c of laplace2d-gpu-parallel.c.

Include the following directive at the end of the  initialize() function with the right data clauses 

#pragma omp target enter data <data clauses>

Include the following directive at the beginning of the  deallocate() function with the right data clauses

#pragma omp target exit data <data clauses>

Compile the code using the following commands

If you get error messages, try to understand the problem and adapt the other files used during compilation accordingly.

Run and profile  the code on the compute node and compare with the previous profiling results.

For Fortran:

Make a copy laplace2d-gpu-data-unstructured-parallel.f90 of laplace2d-gpu-parallel.f90.

Include the following directive at the end of the  initialize() function with the right data clauses:

!$omp target enter data <data clauses>

Include the following directive at the beginning of the  deallocate() function with the right data clauses:

!$omp target exit data <data clauses>

Compile the code using the following commands

If you get error messages, try to understand the problem and adapt the other files used during compilation accordingly.

Run and profile  the code on the compute node and compare with the previous profiling results.

Solutions

The solutions can be found in the file OpenMP-solutions.tgz

Copy the solutions to your own home directory via