FAQ: Visual Studio Code on SuperMUC-NG - Phase 1 & Phase 2 (with internet access)


Prerequisites:

  • Basic understanding of SSH and bash shell commands;
  • A proxy software for HTTP connections installed on your local machine; you can easily install the python3 one with:
    $ pip3 install --upgrade --user proxy.py
  • VS Code with RemoteSSH extension installed on you local machine;
  • In settings for RemoteSSH, Remote.SSH: Local Server Download is set to always.

We will identify commands to be run locally with the dollar sign $ , and the ones to be run on the remote machine (Phase 1/Phase 2) with the greater-than symbol > .

Steps

Edit SSH and Bash/Zsh config file

  1. Open your ~/.ssh/config file using a text editor of your choice;
  2. Add the host with the following details:
    Host phase2 phase2.*
	User your_user_name
	Hostname pvc.supermuc.lrz.de
	# other configs, such as IdentityFile and Port...

Host *.rf
	RemoteForward REMOTE_PORT localhost:1234 	# you could use some port other than 1234 as well.

    The REMOTE_PORT must be an arbitrary integer value between 1024 and 65535; choose a value and paste it there.

  3. Save and close the file.


Now you can connect to SuperMUC-NG Phase 2:

  • with remote forward:
    $ ssh phase2.rf
  • without remote forward:
    $ ssh phase2


Find available Remote Port Number

The REMOTE_PORT  number is arbitrary, but it must be not already used by some other process in the remote host (i.e. Phase 2). To solve this problem:

  1. Connect to SuperMUC-NG
  2. Run the following command to find an available port number:
    $ ss -tulpn
  3. Look for ports that are currently being used and choose an available port number (one that is not listed) in the range 1024->65535;
  4. Use this in the ~/.ssh/config in your local machine mentioned in the previous step.


Start Proxy on the local machine

On your local machine, start the proxy using the port number you entered in the ~/.ssh/config (1234 in our case):

$ ~/.local/bin/proxy --port 1234    # here paste the local port value you set locally
Creating proxy port forward on port 1234...
2024-05-13 15:32:07,282 - pid:49535 [I] plugins.load:85 - Loaded plugin proxy.http.proxy.HttpProxyPlugin
2024-05-13 15:32:07,283 - pid:49535 [I] tcp.listen:80 - Listening on 127.0.0.1:1234
2024-05-13 15:32:07,300 - pid:49535 [I] pool.setup:105 - Started 10 acceptors in threadless (local) mode

To check that works as expected, run in another terminal session/tab:

$ curl -x localhost:1234 google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>


Set HTTP Proxy Variables and test internet connection

Open another terminal tab and connect to remote with:

$ ssh phase2.rf

In line of principle, you just need to export the following 4 variables:

> export http_proxy="localhost:REMOTE_PORT"                         
> export https_proxy="localhost:REMOTE_PORT"                        
> export HTTP_PROXY="localhost:REMOTE_PORT"                          # for conda, "http://" may need to be prepended 
> export HTTPS_PROXY="localhost:REMOTE_PORT"                         # for conda, "https://" may need to be prepended

where REMOTE_PORT is the SAME PORT VALUE you set in the ~/.ssh/config. However, this procedure can be very tedious, especially if you have to run it many times due to small errors. 

For this (and other) reasons, we provide here a shell script that set this env vars automatically. 
Use a terminal editor (such as nano or vim) to create the script export_http_users.sh  with the following content:

export_http_users.sh 
echo ""
echo "Loading ~/.bash_profile..."
source $HOME/.bash_profile
echo "Loaded ~/.bash_profile."
echo ""
 
_core_export_http_users(){
    show_usage(){
        echo "Usage:                 _core_export_http_users [value] "
        echo ""
        echo "Options:"
        echo "   -h                 Show this help message"
        echo "   -s                 Prepend 'http://' and 'https://' to HTTP_PROXY and HTTPS_PROXY values"
        echo "Values:"
        echo "   [value]            Value to export the http env vars to"
        echo ""
        echo "Export the 4 http environment variables with the given value; i.e. runs:"
        echo "     $ export http_proxy=<p>"
        echo "     $ export https_proxy=<p>"
        echo "     $ export HTTP_PROXY=<p>             # 'http://<p>' with option '-s'"
        echo "     $ export HTTPS_PROXY=<p>            # 'https://<p>' with option '-s'"
        echo "where <p> is the input value."
        echo "NOTE: YOU SHOUDN'T USE THIS FUNCTION, BUT ITS INTERFACE export_http_users"
    }
 
    local prepend_scheme=false
    local OPTIND
    while getopts "hs" opt; do
        case $opt in
            h) show_usage; return 1 ;;
            s) prepend_scheme=true ;;
        esac
    done
 
    shift "$((OPTIND-1))"
 
    # Check the number of arguments
    if [ "$#" -ne 1 ]; then
        show_usage
        echo $'\n'"Error: Function requires 1 argument, not $#."
        return 1
    fi
 
 
    lp=$1
    lps="$lp"
    lpss="$lp"
    if $prepend_scheme; then
        lps="http://$lps"
        lpss="https://$lpss"
    fi
 
    echo ""
    echo "Exporting the 4 http env vars to $lp..."
    export http_proxy="$lp"
    export https_proxy="$lp"
    export HTTP_PROXY="$lps"
    export HTTPS_PROXY="$lpss"
    echo "  http_proxy=$http_proxy"
    echo "  https_proxy=$https_proxy"
    echo "  HTTP_PROXY=$HTTP_PROXY"
    echo "  HTTPS_PROXY=$HTTPS_PROXY"
    echo "Export done."
    echo ""
}
 
 
export_http_users(){
    show_usage(){
        echo "Usage:                export_http_users [port] "
        echo ""
        echo "Options:"
        echo "   -h                 Show this help message"
        echo "   -s                 Prepend 'http://' and 'https://' to HTTP_PROXY and HTTPS_PROXY values"
        echo "Values:"
        echo "   [port]             Port to export the http env vars to"
        echo ""
        echo "Export the 4 http environment variables with the given port value; i.e. runs:"
        echo "     $ export http_proxy=localhost:<p>"
        echo "     $ export https_proxy=localhost:<p>"
        echo "     $ export HTTP_PROXY=localhost:<p>             # 'http://localhost:<p>' with option '-s'"
        echo "     $ export HTTPS_PROXY=localhost:<p>            # 'https://localhost:<p>' with option '-s'"
        echo "where <p> is the input port value."
    }
 
    local prepend_scheme=false
    local OPTIND
    while getopts "hs" opt; do
        case $opt in
            h) show_usage; return 1 ;;
            s) prepend_scheme=true ;;
        esac
    done
 
    shift "$((OPTIND-1))"
 
    if [ "$#" -gt 1 ]; then
        show_usage
        echo $'\n'"Error: Function requires 0 or 1 argument, not $#."
        return 1
    fi
 
 
    dp=13445;ll=1024;hl=65536
 
    if [ -z $1 ]; then
        read -p  "Which port do you want to use for export_http_users: " number
    else
        number=$1
    fi
 
    if [ -z $number ]; then
        echo "No port number entered; no export will be done."
 
    elif [[ $number =~ ^[0-9]+$ ]]; then
        if [[ $number -gt $ll && $number -lt $hl ]]; then
            echo "Exporting input port $number..."
 
            if $prepend_scheme; then
                _core_export_http_users -s "localhost:$number"
            else
                _core_export_http_users "localhost:$number";
            fi
 
        else
            echo "You entered $number, which is not in the range $ll < p < $hl; no export will be done."
        fi
    else
        echo "You entered '$number', which is not a valid port number; no export will be done."
    fi
}
 
export_http_users

Then, save and close the file, and source it:

> source ~/export_http_users.sh

Loading ~/.bash_profile...
Loaded ~/.bash_profile.
Which port do you want to use for export_http_users: 15000		# here paste the REMOTE_PORT value you set locally
Exporting input port 15000...

Exporting the 4 http env vars to localhost:15000...
  http_proxy=localhost:15000
  https_proxy=localhost:15000
  HTTP_PROXY=localhost:15000
  HTTPS_PROXY=localhost:15000
Export done.

>

This script source your bashrc config, define the function export_http_users and run it with the input port number provided.

You can check independently if it run correctly with $ env | grep -i proxy :

> env | grep -i proxy
HTTP_PROXY=http://localhost:15000
https_proxy=localhost:15000
http_proxy=localhost:15000
HTTPS_PROXY=https://localhost:15000
>


Finally, check to have internet connection:

> curl google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
>

If you don't, you likely:

  • made a mismatch in the remote port configuration;
  • made a mismatch in the local port configuration;
  • chose a port already in use;
  • not started locally the proxy.

If you do, you can close the ssh connection.


NOTE: conda may require the prefixes http://  and https://  to the env vars HTTP_PROXY  and HTTPS_PROXY ; to add them, simply run the function export_http_users with the -s  flag:

> export_http_users -s 12314		# here paste the REMOTE_PORT value you set locally 
Exporting input port 12314...

Exporting the 4 http env vars to localhost:12314...	
  http_proxy=localhost:12314
  https_proxy=localhost:12314
  HTTP_PROXY=http://localhost:12314
  HTTPS_PROXY=https://localhost:12314
Export done.



Connect to remote with VSCode

Now that we are sure the internet connection works, we will connect with VSCode.
Open VSCode in a new, empty window, then:

  1. Either
    1. click on the Open a Remote Window icon in the bottom-left corner, and then select Connect Current Window to Host...
    2. open the Command Palette and run > Remote-SSH: Connect Current Window to Host...;
  2. Enter the SSH host name you defined in your ~/.ssh/config file with remote forward enabled; in our case, phase2.rf ;
  3. Enter the MFA;
  4. Once logged in (it may take 5 mins the first time), source the ~/export_http_users.sh file defined above (or export the env vars manually, if you like to suffer):
    > source ~/export_http_users.sh ... # same as above

NOTE: if you modify the port value in your config script locally, you need to close and re-open VSCode! 

NOTE: if you open a new directory in VSCode on the Phase 2 system, you will have to re-source the ~/export_http_users.sh file.


Once VSCode starts connecting, a small window like this pops up on the bottom right corner. In case the connection takes too long to be established, you can click here on details and take a look at the logs:

Install VSCode extensions

You can upload remotely VSCode extensions that you already have installed locally:

  • Open the Extensions section in the left menu;
  • In the part below SSH: phase2.rf - INSTALLED  click on the small Cloud-with-downarrow symbol;


  • put a tick on the locally-installed extensions you would like to upload remotely;
  • wait until the process is completed.


Reference: https://stackoverflow.com/questions/56718453/using-remote-ssh-in-vscode-on-a-target-machine-that-only-allows-inbound-ssh-co