How to setup a Jupyter Server in the Cloud
In the following the necessary steps are described how to setup a Jupyter Server on a Compute Cloud Instance that can be used for a course.
- All accounts are temporary and only have access to this special instance.
- The course instructor can set the password for the duration of the course and later on deactivate the accounts again.
- The server is accessible from every client on the internet and the course instructor therefore should monitor the server for suspicious activity or change the password for access in case an account got compromized. However, the server is isolated from all other resources at LRZ like Linux Cluster or SuperMUC so that the amount of damage is small that can be performed by an attacker.
The course instructor should have some experience in setting up a linux OS on an empty machine and some basic understanding how linux works.
Basic Steps
Getting the Hardware
- Get a Linux Cluster Project (see: Access and Login to the Linux-Cluster)
- Get Compute Cloud Access for that project (see: Compute Cloud of SuperMUC-NG)
Setting up the Server
- Start a new instance on the LRZ Compute Cloud where the number of cores should be the number of participants of the course. In general 20 cores should be fine.
- Choose Ubuntu for the OS.
- You need a private/public key for login from your terminal/desktop
- Add the Port 8000 in a Security Group of the VM
- Generate a floating IP for the respective provider network your VM is connected to (either MWN or internet) and attach it to the VM.
- You have to use this floating IP when you later on connect to the jupyter server. http://<floating-ip>:8000
- Be aware that the system is accessible from the whole internet and can be an aim for hackers. Monitor the machine periodically and shelve it when it is not in use!
- Also see: Security Basics from the jupyterhub documentation
Installation of the software
- Do not change the root password. By doing that root cannot be attacked from the outside
- Use sudo for installation of the software
bash history
# login to the VM as ubuntu user # now become root and install the software $ sudo su $ apt update $ apt install jupyter $ apt install python3-pip $ python3 -m pip install jupyterhub $ apt install npm $ npm install -g configurable-http-proxy $ useradd -m testuser $ passwd testuser $ jupyterhub
Additional Information for the Installation
- Jupyter Server using conda see: https://jupyter.org/install
- Jupyter Hub
- for general installation see: https://jupyterhub.readthedocs.io/en/stable/quickstart.html
- as systemd service see: https://github.com/jupyterhub/jupyterhub/wiki/Run-jupyterhub-as-a-system-service
- Kernels for the software you want to use (R-Kernel, tensorflow, etc...) see: https://github.com/IRkernel/IRkernel
User administration
Provide user accounts as many as are needed for the course. Do not give them a login shell. By doing that, users can only use the jupyter hub interface to login.
bash history
# add participants of the course $ useradd -m participant1 $ passwd participant1
add as many users you want. They can access jupyterhub with the username: participant1 and password. Change the names as you wish.
Tipps and tricks
These commands come in handy for setting user passwords and cleaning up user directories after the course:
bash history
$ python >>> import os # create 20 users (user0, user1, .., user19) >>> [os.system(f"useradd -m user{i}") for i in range(20)] >>> [os.system(f"echo user{i}:NEW_SECRET_PASSWORD | chpasswd") for i in range(20)] # create an admin0 user >>> os.system(f"useradd -m admin0") >>> os.system(f"echo admin0:NEW_SECRET_PASSWORD | chpasswd") # create a folder for course material for each user which is owned by admin0 >>> os.system("mkdir /home/admin0/course_material") >>> [os.system(f"ln -s /home/admin0/course_material /home/user{i}/course_material1") for i in range(101)]