diff --git a/Jupyterhub/sentinel2_workflow-testUpdates.ipynb b/Jupyterhub/sentinel2_workflow-testUpdates.ipynb
new file mode 100644
index 0000000..eeeb719
--- /dev/null
+++ b/Jupyterhub/sentinel2_workflow-testUpdates.ipynb
@@ -0,0 +1,6472 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "---\n",
+ "title: Sentinel-2 Processing Workflow\n",
+ "description: This notebook demonstrates the complete workflow for downloading and processing Sentinel-2 satellite imagery using the `disasters-product-algorithms` package. \n",
+ "author: \n",
+ " - Aaron Serre (Editor, UAH)\n",
+ " - Kyle Lesinger (Editor, UAH)\n",
+ "date: January 12, 2025\n",
+ "execute:\n",
+ " freeze: true\n",
+ "---"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Run This Notebook"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "Disclaimer: it is highly recommended to run a tutorial within NASA VEDA JupyterHub, which already includes functions for processing and visualizing data specific to VEDA stories. Running the tutorial outside of the VEDA JupyterHub may lead to errors, specifically related to EarthData authentication. Additionally, it is recommended to use the Pangeo workspace within the VEDA JupyterHub, since certain packages relevant to this tutorial are already installed.
\n",
+ "\n",
+ " If you do not have a VEDA Jupyterhub Account you can launch this notebook on your local environment using MyBinder by clicking the icon below.
\n",
+ "
\n",
+ "\n",
+ "![\"Binder\"](\"https://binder.openveda.cloud/badge_logo.svg\" "\\"A")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Table of Contents\n",
+ "- [Sentinel-2 Processing Workflow](#sentinel-2-processing-workflow)\n",
+ "- [Environment Setup](#environment-setup)\n",
+ "- [Download Sentinel-2 Data](#download-sentinel-2-data)\n",
+ "- [Process Sentinel-2 Data](#process-sentinel-2-data)\n",
+ "- [View Results](#view-results)\n",
+ "- [Output File Naming Convention](#output-file-naming-convention)\n",
+ "- [Next Steps](#next-steps)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Sentinel-2 Processing Workflow #\n",
+ "\n",
+ "This notebook demonstrates the complete workflow for downloading and processing Sentinel-2 satellite imagery using the `disasters-product-algorithms` package.\n",
+ "\n",
+ "## Workflow Steps\n",
+ "1. **Configure Environment Variables** - Set processing parameters\n",
+ "2. **Download Sentinel-2 Data** - Download imagery from Copernicus\n",
+ "3. **Process Sentinel-2 Data** - Generate products with COG conversion and event naming\n",
+ "4. **View Results** - Examine the generated outputs\n",
+ "\n",
+ "## Features Demonstrated\n",
+ "- Cloud Optimized GeoTIFF (COG) conversion\n",
+ "- Event-based file naming for disaster response\n",
+ "- Multiple product generation (true color, NDVI, NDWI, MNDWI, NBR, water extent)\n",
+ "- Cloud masking (L2A only)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Environment Setup #\n",
+ "\n",
+ "Configure all processing parameters as environment variables for easy modification."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Configuration Updated:\n",
+ " Output Directory: /home/jovyan/shared-readwrite/process_sentinel2/202601_WinterWx_US_s2\n",
+ " Search Range: 20251230 to 20251230\n",
+ " Processing Level: L2A\n",
+ " Products: true, swir\n",
+ " Event Name: 202601_WinterWx_US\n"
+ ]
+ }
+ ],
+ "source": [
+ "import os\n",
+ "import subprocess\n",
+ "from pathlib import Path\n",
+ "\n",
+ "# ==================== CONFIGURATION ====================\n",
+ "# Modify these variables according to your requirements\n",
+ "\n",
+ "# --- UPDATED: Credentials for Copernicus (Avoids Subprocess Hang) ---\n",
+ "COP_USER = \"kdl0040@uah.edu\" # Update with your new account email\n",
+ "COP_PASS = \"^d8#KhU_SDj~V5_\" # Update with your account password\n",
+ "\n",
+ "# --- UPDATED: Download parameters (Using a range to find data) ---\n",
+ "#TILE_ID = \"T17RLN\" # Sentinel-2 tile ID\n",
+ "POLYGON = os.path.expanduser(f\"~/shared-readwrite/process_sentinel2/AOI/Jan2026_WinterStorm_LS_S2_AOI.shp\")\n",
+ "# Using a window to catch the Nov 13th overpass\n",
+ "DOWNLOAD_DATE = [\"20251230\"]#, \"20260101\", \"20260118\"] \n",
+ "PROCESSING_LEVEL = \"2\" # 1 = L1C, 2 = L2A\n",
+ "\n",
+ "# Processing parameters\n",
+ "PRODUCTS = [\"true\", \"swir\"]\n",
+ "EVENT_NAME = \"202601_WinterWx_US\"\n",
+ "# The satellite pass actually happened on the 12th\n",
+ "PROCESS_DATE = \"20251230\" \n",
+ "\n",
+ "# COG options\n",
+ "COMPRESSION = \"ZSTD\"\n",
+ "COMPRESSION_LEVEL = 22\n",
+ "NODATA = 0\n",
+ "\n",
+ "# Processing flags\n",
+ "ENABLE_MERGE = True\n",
+ "ENABLE_MASK = False\n",
+ "FORCE_OVERWRITE = True\n",
+ "DST_CRS = 'EPSG:4326'\n",
+ "\n",
+ "# Output directory for downloaded and processed data\n",
+ "OUTPUT_DIR = os.path.expanduser(f\"~/shared-readwrite/process_sentinel2/{EVENT_NAME}_s2\")\n",
+ "\n",
+ "# Create output directory\n",
+ "os.makedirs(OUTPUT_DIR, exist_ok=True)\n",
+ "\n",
+ "print(\"Configuration Updated:\")\n",
+ "print(f\" Output Directory: {OUTPUT_DIR}\")\n",
+ "#print(f\" Tile ID: {TILE_ID}\")\n",
+ "print(f\" Search Range: {DOWNLOAD_DATE[0]} to {DOWNLOAD_DATE[-1]}\") #Change to read first and last of array)\n",
+ "print(f\" Processing Level: L{PROCESSING_LEVEL}A\")\n",
+ "print(f\" Products: {', '.join(PRODUCTS)}\")\n",
+ "print(f\" Event Name: {EVENT_NAME}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Download Sentinel-2 Data #\n",
+ "\n",
+ "Download Sentinel-2 imagery from the Copernicus Data Space Ecosystem.\n",
+ "\n",
+ "**Note:** You will be prompted for your Copernicus credentials.\n",
+ "- Register at: https://dataspace.copernicus.eu/\n",
+ "\n",
+ "- Figure out how to add coper. email and password into the script as an environment variable\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {
+ "collapsed": true,
+ "jupyter": {
+ "outputs_hidden": true
+ }
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "--- DOWNLOADING DATA ---\n",
+ "Credentials received via arguments.\n",
+ "\n",
+ "Output directory: /home/jovyan/shared-readwrite/process_sentinel2/202601_WinterWx_US_s2\n",
+ "Level: L2A\n",
+ "Date: 2025-12-30\n",
+ "Polygon: Jan2026_WinterStorm_LS_S2_AOI.shp\n",
+ "\n",
+ "Number of products to download: 5\n",
+ "Confirmation skipped (-y provided).\n",
+ "\n",
+ "\n",
+ "\n",
+ "Downloading files: 0%| | 0/5 [00:00"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "File Details:\n",
+ "==================================================\n",
+ "\n",
+ "shortwaveInfrared:\n",
+ " - S2A_MSIL2A_shortwaveInfrared_20251230_165741_T15SXS.tif (86.3 MB)\n",
+ " - S2B_MSIL2A_shortwaveInfrared_20251230_164619_T15SWS.tif (86.3 MB)\n",
+ " - S2B_MSIL2A_shortwaveInfrared_20251230_164619_T15SXS.tif (86.3 MB)\n",
+ " - S2B_MSIL2A_shortwaveInfrared_20251230_164619_T15SYR.tif (86.3 MB)\n",
+ " - S2B_MSIL2A_shortwaveInfrared_20251230_164619_T15SYS.tif (86.3 MB)\n",
+ " - S2B_MSIL2A_shortwaveInfrared_20251230_164619_T16SCC.tif (86.3 MB)\n",
+ " - S2B_MSIL2A_shortwaveInfrared_20251230_164619_T16SCD.tif (86.3 MB)\n",
+ " - S2B_MSIL2A_shortwaveInfrared_20251230_164619_T16SDE.tif (86.3 MB)\n",
+ " - S2B_MSIL2A_shortwaveInfrared_20251230_164619_T16SDF.tif (86.3 MB)\n",
+ "\n",
+ "trueColor:\n",
+ " - S2A_MSIL2A_trueColor_20251230_165741_T15SXS.tif (384.2 MB)\n",
+ " - S2A_trueColor_20251230_merged.tif (10006.3 MB)\n",
+ " - S2B_MSIL2A_trueColor_20251230_164619_T15SWS.tif (384.2 MB)\n",
+ " - S2B_MSIL2A_trueColor_20251230_164619_T15SXS.tif (384.2 MB)\n",
+ " - S2B_MSIL2A_trueColor_20251230_164619_T15SYR.tif (383.3 MB)\n",
+ " - S2B_MSIL2A_trueColor_20251230_164619_T15SYS.tif (384.2 MB)\n",
+ " - S2B_MSIL2A_trueColor_20251230_164619_T16SCB.tif (345.0 MB)\n",
+ " - S2B_MSIL2A_trueColor_20251230_164619_T16SCC.tif (345.0 MB)\n",
+ " - S2B_MSIL2A_trueColor_20251230_164619_T16SCD.tif (345.0 MB)\n",
+ " - S2B_MSIL2A_trueColor_20251230_164619_T16SDE.tif (345.0 MB)\n",
+ " - S2B_MSIL2A_trueColor_20251230_164619_T16SDF.tif (345.0 MB)\n",
+ " - tmp6cgf6udj.tif (2347.0 MB)\n"
+ ]
+ }
+ ],
+ "source": [
+ "import glob\n",
+ "import os\n",
+ "import rasterio\n",
+ "from rasterio.plot import show\n",
+ "import matplotlib.pyplot as plt\n",
+ "import numpy as np\n",
+ "\n",
+ "# We'll check both \"output\" and the EVENT_NAME folder to be safe\n",
+ "potential_paths = [\n",
+ " os.path.join(OUTPUT_DIR, \"output\"),\n",
+ " os.path.join(OUTPUT_DIR, EVENT_NAME)\n",
+ "]\n",
+ "\n",
+ "output_dir = None\n",
+ "for path in potential_paths:\n",
+ " if os.path.exists(path):\n",
+ " output_dir = path\n",
+ " break\n",
+ "\n",
+ "if output_dir:\n",
+ " print(f\"✓ Found output directory: {output_dir}\\n\")\n",
+ " \n",
+ " # Find all finished GeoTIFF files (ignoring temp files)\n",
+ " tif_files = sorted([f for f in glob.glob(os.path.join(output_dir, \"**/*.tif\"), recursive=True) \n",
+ " if not f.endswith('.tmp.tif')])\n",
+ " \n",
+ " if tif_files:\n",
+ " product_types = {}\n",
+ " for tif_file in tif_files:\n",
+ " product_type = os.path.basename(os.path.dirname(tif_file))\n",
+ " if product_type not in product_types:\n",
+ " product_types[product_type] = []\n",
+ " product_types[product_type].append(tif_file)\n",
+ "\n",
+ " # --- PLOTTING SECTION ---\n",
+ " print(f\"Displaying {len(product_types)} product types...\")\n",
+ " \n",
+ " # Create a grid for plotting (1 row per product type)\n",
+ " fig, axes = plt.subplots(len(product_types), 1, figsize=(12, 8 * len(product_types)))\n",
+ " if len(product_types) == 1: axes = [axes] # Handle single product case\n",
+ "\n",
+ " for i, (p_type, files) in enumerate(sorted(product_types.items())):\n",
+ " ax = axes[i]\n",
+ " sample_file = files[0] # Plot the first file available for this type\n",
+ " \n",
+ " with rasterio.open(sample_file) as src:\n",
+ " # If it's true color (3 bands), we read all. Otherwise, just one.\n",
+ " if src.count >= 3:\n",
+ " arr = src.read([1, 2, 3])\n",
+ " # Normalize for display if needed\n",
+ " arr = np.ma.transpose(arr, (1, 2, 0))\n",
+ " else:\n",
+ " arr = src.read(1)\n",
+ " \n",
+ " img = show(src, ax=ax, title=f\"Product: {p_type}\\n{os.path.basename(sample_file)}\")\n",
+ " print(f\"✓ Generated plot for {p_type}\")\n",
+ "\n",
+ " plt.tight_layout()\n",
+ " plt.show()\n",
+ "\n",
+ " # List file sizes below the images\n",
+ " print(\"\\nFile Details:\")\n",
+ " print(\"=\" * 50)\n",
+ " for p_type, files in sorted(product_types.items()):\n",
+ " print(f\"\\n{p_type}:\")\n",
+ " for f in files:\n",
+ " size = os.path.getsize(f) / (1024*1024)\n",
+ " print(f\" - {os.path.basename(f)} ({size:.1f} MB)\")\n",
+ " else:\n",
+ " print(\"No finished GeoTIFF files found. Check if the processing script is still running.\")\n",
+ "else:\n",
+ " print(f\"Output directory not found in: {potential_paths}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Output File Naming Convention #\n",
+ "\n",
+ "Files are named with the event prefix and formatted date:\n",
+ "\n",
+ "**Original:** `S2B_MSIL2A_colorInfrared_20251111_161419_T17RLN.tif`\n",
+ "\n",
+ "**With Event Naming:** `202311_Example_Event_S2B_MSIL2A_colorInfrared_161419_T17RLN_2025-11-11_day.tif`\n",
+ "\n",
+ "This naming convention:\n",
+ "- Adds event prefix for organization\n",
+ "- Removes date from middle position\n",
+ "- Adds formatted date (YYYY-MM-DD) at the end\n",
+ "- Includes `_day` suffix for AWS/cloud compatibility"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Next Steps #\n",
+ "\n",
+ "You can now:\n",
+ "1. Load and visualize the GeoTIFF files using libraries like `rasterio` or `GDAL`\n",
+ "2. Upload the COG files to cloud storage (S3, GCS, etc.)\n",
+ "3. Process additional dates or tiles by modifying the configuration variables\n",
+ "4. Generate additional products by updating the `PRODUCTS` list"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.12.11"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/_quarto.yml b/_quarto.yml
index 7d7d409..a675beb 100644
--- a/_quarto.yml
+++ b/_quarto.yml
@@ -58,6 +58,7 @@ website:
- Jupyterhub/jupyterhub-training-guide.qmd
- Jupyterhub/loading-disaster-hub-image.qmd
- Jupyterhub/clone_conversion_repo.ipynb
+ - Jupyterhub/sentinel2_workflow-testUpdates.ipynb
- section: workflow.qmd
text: Data Workflow Diagrams
contents: