Merge pull request #14 from hdpriest-ui/develop

Develop

Author: hdpriest-ui

Deploy.md

@@ -2,9 +2,24 @@
 
 ## get the most recent changes moved to campus cluster
 
+#### ONCE
+Move or re-create the central repo located at `/sw/icrn/jupyter/icrn_ncsa_resources/Kernels/` to 
+`/sw/icrn/dev/kernels/`
+
+copy tools to path loc (dev paths):
+```sh
+cp ./icrn_manager /sw/icrn/dev/bin/
+cp ./update_r_libs.sh /sw/icrn/dev/bin/
+chmod +x /sw/icrn/dev/bin/icrn_manager
+chmod +x /sw/icrn/dev/bin/update_r_libs.sh
+```
+
+Change Kernel Central Repo Path in icrn_manager to reflect environment
+-- what is the signal in icrn that we are in prod/dev? --
 
-## Changes needed to ICRN containers in dev
 
+## Changes needed to ICRN containers in dev
+JQ installed on container
 
 
 ## running the kernel index process
@@ -18,6 +33,15 @@ Tested on campus cluster via apptainer, this indexes the central repository of k
 (base) [hdpriest@cc-login2 icrn_manager]$ apptainer pull docker://hdpriest0uiuc/icrn-kernel-indexer
 
 (base) [hdpriest@cc-login2 icrn_manager]$ apptainer run --bind /sw/icrn/jupyter/icrn_ncsa_resources/Kernels:/sw/icrn/jupyter/icrn_ncsa_resources/Kernels icrn-kernel-indexer_latest.sif
+```
+
+For dev and prod environments, the kernel index job needs to be run using a custom kernel root env variable (see directly below)
+
+this will obviously be different based on the container runtime
+```sh
+#### DEV
+(base) [hdpriest@cc-login2 icrn_manager]$ 
+apptainer run --env "KERNEL_ROOT=/sw/icrn/dev/kernels" --bind /sw/icrn/dev/kernels:/sw/icrn/dev/kernels icrn-kernel-indexer_latest.sif
 ## ... output
 Collated manifest written to: /sw/icrn/jupyter/icrn_ncsa_resources/Kernels/collated_manifests.json
 ## ... output
@@ -36,8 +60,7 @@ docker build -t icrn-web -f web/Dockerfile web/
 Assuming you have a built container (obtained from dockerhub, or built locally), the container expects bind mounts to the location where the manifests/index are kept, and so is flexible for its back-end storage as long as it has access to the same disk mount as the index job:
 ```bash
 docker run -d -p 8080:80 --name icrn-web \
-  -v /sw/icrn/jupyter/icrn_ncsa_resources/Kernels/collated_manifests.json:/app/data/collated_manifests.json \
-  -v /sw/icrn/jupyter/icrn_ncsa_resources/Kernels/package_index.json:/app/data/package_index.json \
+  -v /sw/icrn/dev/kernels:/app/data:ro \
   icrn-web
 ```
 

icrn_manager

@@ -13,7 +13,10 @@ fi
 
 icrn_base=".icrn"
 icrn_kernels="icrn_kernels"
-central_catalog_default="/sw/icrn/jupyter/icrn_ncsa_resources/Kernels"
+APP_FOLDER=$(dirname $0)
+KERNEL_FOLDER=${KERNEL_FOLDER:-${APP_FOLDER}/../kernels}
+if [ ! -d ${KERNEL_FOLDER} ]; then echo "Could not determine location of kernel respository - contact administrator"; exit -1 ; fi
+central_catalog_default=${KERNEL_FOLDER}
 
 ICRN_USER_BASE=${ICRN_USER_BASE:-${HOME}/${icrn_base}}
 ICRN_MANAGER_CONFIG=${ICRN_MANAGER_CONFIG:-${ICRN_USER_BASE}/manager_config.json}

kernel-indexer/README.md

@@ -30,16 +30,16 @@ docker build -t icrn-kernel-indexer:latest .
 
 ```bash
 docker run --rm \
-  -v /sw/icrn/jupyter/icrn_ncsa_resources/Kernels:/sw/icrn/jupyter/icrn_ncsa_resources/Kernels \
+  -v /sw/icrn/dev/kernels:/sw/icrn/dev/kernels \
   icrn-kernel-indexer:latest
 ```
 
 ### With Custom Configuration
 
 ```bash
 docker run --rm \
-  -v /sw/icrn/jupyter/icrn_ncsa_resources/Kernels:/sw/icrn/jupyter/icrn_ncsa_resources/Kernels \
-  -e KERNEL_ROOT=/sw/icrn/jupyter/icrn_ncsa_resources/Kernels \
+  -v /sw/icrn/dev/kernels:/sw/icrn/dev/kernels \
+  -e KERNEL_ROOT=/sw/icrn/dev/kernels \
   -e LANGUAGE_FILTER=Python \
   -e LOG_LEVEL=DEBUG \
   icrn-kernel-indexer:latest