Merge pull request #391 from SouthernMethodistUniversity/update_vscode_instructions

Update vscode instructions

Author: jrlagrone

docs/applications/vscode.md

@@ -3,11 +3,23 @@
 Visual Studio Code is a popular code editor with extensions for coding
 in different languages. We recommend using Visual Studio Code through the
 [HPC OnDemand Web Portal](https://hpc.m3.smu.edu/). See instructions
-for the using the [portal version](/applications/vscode.md#portal) below.
+for the using the [portal version (preferred)](/applications/vscode.md#portal) below.
+You can also use [Remote Tunnels](/applications/vscode.md#tunnel) submitted via batch jobs.
+
+```{important}
+We understand that research and code development often requires interactive
+use. However, interactive use is inherently inefficient and often results
+in substantial HPC resources sitting idle. We highly recommend submitting
+scripted and batched jobs whenever possible. If you would like assistance
+converting an interactive workflow, please [contact us.](about:contact).
+```
 
 ## Portal
 
-1. Select "Visual Studio Code" from the "Interactive Apps" drop-down menu.
+Using the [HPC OnDemand Web Portal](https://hpc.m3.smu.edu/) portal is the
+preferred and easiest way to use Visual Studio Code on M3.
+
+1.  Select "Visual Studio Code" from the "Interactive Apps" drop-down menu.
 2.  Select options required for your Visual Studio Code instance. These
     options are the same as those requested via a standard Slurm script.
     Any modules that are needed at runtime should be loaded using the
@@ -30,14 +42,273 @@ run modules and code with the Slurm scheduler from an SSH session.
     Interactive Sessions" tab in your browser and select "Delete" and
     "Confirm", when prompted, to cancel the job.
 
+## Tunnel
+
+Visual Studio Code can also be used on M3 or the SuperPOD by starting
+a SSH tunnel in a job. The following is the recommended way of starting
+a session.
+
+### Setup on your computer
+
+In order to use the remote tunnel, you will need the following:
+
+- A recent version of [VSCode](https://code.visualstudio.com/download).
+- Install the following extension (these extensions are all published by Microsoft)
+  - [Remote Explorer](https://marketplace.visualstudio.com/items?itemName=ms-vscode.remote-explorer)
+  - [Remote - SSH](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh)
+  - [Remote - Tunnels](https://marketplace.visualstudio.com/items?itemName=ms-vscode.remote-server)
+
+### HPC Job Script
+
+On either M3 or the SuperPOD, copy the following job script and make any necessary modifications.
+You will need to change the Slurm account to an active allocation in [ColdFront](https://hpcaccess.smu.edu)
+that you have access to -- the one in this script is an example and will not run.
+You may need to adjust the requested resources to match your needs, for instance, if you need
+more cores or memory.
+
+We'll call this file `vscode_tunnel.sbatch`, but you can name it something else if you like. 
+
+```bash
+#!/bin/bash -e
+#SBATCH -J code_tunnel        # job name (optional)
+#SBATCH -p dev                # partition
+#SBATCH --time=0-01:00:00     # time in Days-HH:MM:SS
+#SBATCH -c 1                  # number of cores
+#SBATCH --mem=4G              # memory in GB
+#SBATCH -A peruna_0001        # slurm account to use (MUST CHANGE!)
+#SBATCH --signal=B:USR1@15    # send interrupt signal 15s before end
+
+# Set a name for the tunnel. We recommend using something
+# that identifies the system, lik m3tunnel on M3 and
+# superpodtunnel on the SuperPOD
+TUNNEL_NAME="m3tunnel"
+
+#########################################################
+# You shouldn't need to change anything below here.     #
+#########################################################
+
+# Function executed when signal USR1 is sent
+# before the job ends to clean up running processes
+function sig_handler()
+{
+        echo "function sig_handler called"
+        echo "Signal trapped -  `date`"
+
+        # stop any running commands
+        # Check if the AUTH PID exists
+        if [[ -v AUTH_PID ]]; then
+          # Check if the auth process is running
+          if ps -p "$AUTH_PID" >/dev/null; then
+            # kill it, the -15 tries to kill "nicely"
+            kill -15 $AUTH_PID
+          fi
+        fi
+
+        # check if the TUNNEL PID exists
+        if [[ -v TUNNEL_PID ]]; then
+          # Check if the auth process is running
+          if ps -p "$TUNNEL_PID" >/dev/null; then
+            # kill it, the -15 tries to kill "nicely"
+            kill -15 $TUNNEL_PID
+          fi
+        fi
+
+        # delete the temporary folder if it exists
+        if [[ -v JOBTEMP ]]; then
+          if [ -d "$JOBTEMPR" ]; then
+            rm -rf $JOBTEMP
+          fi
+        fi
+
+        exit 0
+}
+
+# "trap" or wait for the signal USR1 and call the sig_handler 
+# helper function to clean up temporary files and running
+# processes
+trap 'sig_handler' SIGUSR1
+
+# create a temp directory in $SCRATCH to download VSCode
+# cli tools and run out of
+# Reminder $SCRATCH has a 60 purge policy -- files
+# older than 60 days may be permanently deleted without warning
+# This will only be used for temporary files and this script
+# will try to delete them before the job ends
+JOBTEMP=$(TMPDIR=$SCRATCH mktemp -d)
+
+# Download the latest stable version of VSCode CLI tools
+# and extract it to the JOBTEMP folder
+echo "Downloading VSCode CLI to: ${JOBTEMP}"
+curl -Lk 'https://code.visualstudio.com/sha/download?build=stable&os=cli-alpine-x64' | tar -C $JOBTEMP -xzf -
+
+# Generate a login code for the tunnel using Microsoft as the authenticator
+echo "Generating authentication code"
+VSCODE_CLI_DISABLE_KEYCHAIN_ENCRYPT=1 $JOBTEMP/code tunnel user login --provider microsoft &
+
+# get PID of login code process
+AUTH_PID=$!
+
+# wait for authentication to succeed
+wait ${AUTH_PID}
+
+# Start the tunnel
+# By running this, you are accepting VSCode's license. We highly recommend
+# reviewing the license first: https://code.visualstudio.com/license
+echo "Starting code tunnel"
+$JOBTEMP/code tunnel --accept-server-license-terms --name ${TUNNEL_NAME} &
+
+# Get PID of code tunnel
+TUNNEL_PID=$!
+
+# Keep running until we get the interrupt signal or the code server
+# session ends
+echo "Running"
+wait ${TUNNEL_PID}
+```
+
+### Submitting the job script
+
+To submit the job script, call
+
+```bash
+sbatch vscode_tunnel.sbatch
+```
+
+from a login node. You can use the [web portal terminal](https://hpc.m3.smu.edu/pun/sys/shell/ssh/129.119.70.4)
+to submit to M3 or you can use SSH session (the SuperPOD is only available via SSH).
+
+Once you submit the job script, you should see something like:
+
+```bash
+Submitted batch job 22880401
+```
+
+and you can monitor when the job starts or it's state in the queue with
+
+```bash
+squeue --me
+```
+
+In the `NODELIST(REASON)` column, the reasons `Priority`, `BeginTime`, or `Resources` are
+normal and mean your job is waiting for its turn to start. If you see other reasons it may
+indicate a issue with the job. The codes are described in the
+[Slurm documentation](https://slurm.schedmd.com/job_reason_codes.html) or you can
+[contact us](about:contact) with questions about the status.
+
+The amount of time a job takes to start is dependent on how busy the system currently is.
+Estimated start times are sometimes available with
+
+```bash
+squeue --me --start
+```
+
+Estimated start times are based on job requests currently running or waiting in the queue.
+If a start time is given, jobs typically (but not always) start before that time.
+If the estimate is NA, the start time was not estimated -- the job scheduler does only
+computes estimates for a limited number of jobs at a time, so this is not unusual.
+
+Once the job starts, the output of `squeue --me` will look like:
+
+```bash
+22880401       dev code_tun peruna  R       0:19      1 c031
+```
+
+In this case, this tells us the job has been running for 19 seconds on node `c031`.
+If you would like the system to email you when the job starts, you can include the
+following at the top of the job script before you submit (email will only be sent
+to SMU email addresses.)
+
+```bash
+#SBATCH --mail-type=BEGIN
+#SBATCH --mail-user=peruna@smu.edu
+```
+
+### Connecting credentials
+
+Once your job starts, it should generate an output file with a name like
+`slurm-22880401.out`. The number will change each time you run and is the
+job id that was printed out when you submitted the job script.
+
+From a terminal, you can print out the file with `cat slurm-22880401.out` and
+it should look something like (it may need to run for a minute or so before
+this is available):
+
+```
+Downloading VSCode CLI to: /lustre/scratch/client/users/peruna/tmp.0El3JhwWWT
+  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
+                                 Dload  Upload   Total   Spent    Left  Speed
+100   162  100   162    0     0    507      0 --:--:-- --:--:-- --:--:--   507
+100 9816k  100 9816k    0     0  5866k      0  0:00:01  0:00:01 --:--:-- 7580k
+Generating authentication code
+To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code FE71YRZUVM to authenticate.
+```
+
+In your web browser, go navigate to [https://microsoft.com/devicelogin](https://microsoft.com/devicelogin) and
+enter the code in the log file (in this example `FE71YRZUVM`). You should treat this code like a password.
+Sharing it is a violation of [SMU policy](https://www.smu.edu/policy/8-information-technology/8-1-acceptable-use).
+
+![add code to Microsoft allow access page](../images/vscode/add_auth_code.png)
+
+After clicking next on the page, you will be asked to login to a Microsoft account.
+We recommend using your SMU account.
+
+After the access code has been added to a Microsoft account, the job script will progress and should
+look similar to:
+
+```
+Downloading VSCode CLI to: /lustre/scratch/client/users/peruna/tmp.0El3JhwWWT
+  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
+                                 Dload  Upload   Total   Spent    Left  Speed
+100   162  100   162    0     0    507      0 --:--:-- --:--:-- --:--:--   507
+100 9816k  100 9816k    0     0  5866k      0  0:00:01  0:00:01 --:--:-- 7580k
+Generating authentication code
+To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code FE71YRZUVM to authenticate.
+Starting code tunnel
+Running
+*
+* Visual Studio Code Server
+*
+* By using the software, you agree to
+* the Visual Studio Code Server License Terms (https://aka.ms/vscode-server-license) and
+* the Microsoft Privacy Statement (https://privacy.microsoft.com/en-US/privacystatement).
+*
+
+Open this link in your browser https://vscode.dev/tunnel/m3tunnel
+```
+
+### Connect VSCode to the Tunnel
+
+To connect your VSCode client, click on `Open Remote Window` in the bottom left corner
+of VSCode (it is an icon that looks similar to `><`.) That will open up options:
+
+![options presented by VSCode after clicking open remote window.](../images/vscode/options.png)
+
+Select the `Connect to Tunnel` option. Then select the `Microsoft Account` option:
+
+![options to select the type of account used for authorization. Typically this shows Microsoft and Github.](../images/vscode/auth_option.png)
+
+After selecting the `Microsoft Account` type, you may be prompted to login to your Microsoft account.
+Then you should be able to select the tunnel you created.
+
+![Shows m3tunnel running on c031 is Online as an option to connect to.](../images/vscode/tunnel.png)
+
+The tunnel name should match the name in the job script (`m3tunnel` in this example), the node it is running on
+(`c031` in this example) and it should show as `Online` (sometimes it takes a while for this status to update, so
+it is ok if the job is running and it says `Offline`.)
+
+The first time you connect to a tunnel, VSCode may install some components that allows it to connect to
+M3 or the SuperPOD. This can can 5-10 minutes, but should only be necessary the first time you connect or
+when VSCode has updates.
+
 ## SSH on Login Nodes
 
 ```{warning}
 As a reminder, running code on login nodes is prohibited. This method
 is intended only for code editing. If you want to execute code from
-Visual Studio Code, please use the [portal version](/applications/vscode.md#portal).
+Visual Studio Code, please use the [portal version](/applications/vscode.md#portal)
+or [Remote Tunnels](/applications/vscode.md#tunnel).
 ```
 
-To connect via SSH to a login node, follow the 
+To connect via SSH to a login node, follow the
 [Remove Development using SSH](https://code.visualstudio.com/docs/remote/ssh) instructions
-for using the Remote-SSH feature.
+for using the Remote-SSH feature.