4.1 Enroot - Introduction

The LRZ AI Systems use the Enroot container framework provided by NVIDIA, to run jobs in isolated environments.
Enroot allows users to define custom software stacks and operates entirely in user space.

No root privileges are needed to build or run containers. This approach ensures flexibility while maintaining system security and performance.
Enroot supports various container image formats, including Docker images.

This means you can use any standard Docker image, such as those from Docker Hub or the NVIDIA NGC Cloud.
Using NVIDIA NGC containers requires registration. For details, see section 4.2 Enroot - Images from Nvidia NGC (backup), and refer to the official Enroot import documentation.

Please note:

  1. Enroot is not available on the SSH login nodes of the LRZ AI Systems.
    To use Enroot on the compute nodes, please start an interactive session, as described in 5.1 Slurm Interactive Jobs.

  2. Tools like module, conda, and pip are not officially supported on the LRZ AI Systems.
    The only supported method for defining custom software environments is through the use of Enroot containers.

The Enroot Workflow

Running containerized applications with the Enroot framework follows a slightly different workflow compared to other container technologies.
In most common use cases, you’ll only need three commands: enroot import, enroot create, and enroot start.
However, we recommend reviewing the Enroot documentation for a deeper understanding.

Since the Enroot container runtime is currently not available on the SSH login nodes of the LRZ AI Systems,
please start an interactive session, as described in 5.1 Slurm Interactive Jobs, before proceeding with any of the steps below.

Importing an Enroot Container Image

To use a container image from a registry (e.g., Docker or NVIDIA NGC), start by importing it with the enroot import command.
This converts the image into an Enroot-compatible .sqsh file, which can be reused to launch multiple containers.

enroot import docker://ubuntu

Creating an Enroot Container

Once you have an Enroot image, use the enroot create command to expand it into a local container filesystem. Each container requires a separate run of this command. To assign custom names to containers, see the official documentation: https://github.com/NVIDIA/enroot/blob/master/doc/usage.md

The command syntax is:

enroot create --name <name-to-give-the-container> <path-to-an-Enroot-image>


If the --name option is not specified, the container will inherit the image name (without the .sqsh extension).
For example, the following command creates a container named ubuntu from the image file ubuntu.sqsh, which must be present in the current directory:

enroot create ubuntu.sqsh

Running Software inside an Enroot Container

Once you have an Enroot container, you can run applications within its defined software environment using the enroot start command.
You can either execute a predefined hook script inside the container (refer to the container image documentation for its location)
or directly specify the executable as a command-line argument.

enroot start ubuntu


If you need to run a command as root inside the container, you can use the --root option. Note that this grants root privileges only within the container environment, not on the host system.

enroot start --root ubuntu

Importing Containers from Nvidia NGC

The catalogue of available Nvidia NGC containers can be consulted here: https://ngc.nvidia.com/catalog/containers. To import (pull if using docker terminology) these containers you need an API key, which is associated to your Nvidia NGC account. You can generate your API key here: https://ngc.nvidia.com/setup/api-key. For the rest of this section, let us refer to your generated API key as <API_KEY>. 

To configure Enroot for using your API key, create the file enroot/.credentials within your $HOME and append the following line to it:

machine nvcr.io login $oauthtoken password <API_KEY>

where <API_KEY> is the key generated as described above. 

After doing this, you can import containers from Nvidia NGC. For example, the latest tensorflow container can be imported as indicated below. 

 enroot import docker://nvcr.io#nvidia/tensorflow:20.12-tf1-py3