All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- SPS Version 3.1.0 (new)
- OGC API Version 2.1.0 (unchanged)
- OGC Python Client Version 2.1.0 (unchanged)
This release includes several updates to various Science Processing System components, focusing on scalability and testability:
- An optional High-Load configuration has been built up to support increased processing loads; this may be enabled at install-time.
- Test coverage has been greatly expanded to include the Airflow and OGC APIs, as well as the CWL classic and CWL modular DAGs.
- Additionally, the Cognito authentication interaction has been worked into the SPS integration tests, allowing for the tests to properly access the OGC API.
- Some commonly used DAGs are configured to be automatically installed any time the SPS is deployed.
- The SPS Initiators tooling is now included in the general release.
Additionally, there are a number of small adjustments:
- A number of DAGs have included changes involving instance sizing and type.
- The SPS database instance has been updated to a more recent version, and given increased resource allocation.
- There are no changes in the Terraform variables required for a standard SPS deployment, but the .tfvars files should be
regenerated to pick up the new default values (like the version of the Airflow Docker image).
Optionally, before deployment, the administrator may decide to install a more scalable SPS by using this terraform settimng:
- helm_values_template = "values_high_load.tmpl.yaml"
- This release does not include updating the Python unity-sds OGC client library: https://github.com/unity-sds/unity-monorepo/tree/main/libs/unity-py/unity_sds_client which will not work with any new SPS deployment. This upgrade is expected to be included in the next SPS release.
- unity-sps: https://github.com/unity-sds/unity-sps/releases/tag/3.1.0
- unity-sps-ogc-processes-api: https://github.com/unity-sds/unity-sps-ogc-processes-api/releases/tag/2.1.0
- unity-sps-ogc-processes-api-client-python: https://github.com/unity-sds/unity-sps-ogc-processes-api-client-python/releases/tag/2.0.1
- ghcr.io/unity-sds/unity-sps/sps-airflow:3.1.0
- ghcr.io/unity-sds/unity-sps/sps-docker-cwl:3.1.0
- ghcr.io/unity-sds/unity-sps-ogc-processes-api/unity-sps-ogc-processes-api:2.1.0
- For Administrators:
- For Developers:
- For Users:
- EPIC: Airflow/Cognito Integration
- EPIC: Deployment Enhancements
- EPIC: Continue Application Package Standardization
- EPIC: Demonstrate Space Use Case
- SPS Version 3.0.0 (new)
- OGC API Version 2.0.0 (unchanged)
- OGC Python Client Version 2.0.1 (unchanged)
This release supports full integration of Cognito authentication with the Unity Science Processing System:
- Users authenticate to the SPS Airflow User Interface using their Unity Cognito credentials.
- After Cognito authentication, the Airflow native authentication is disabled, so that users only have to login once.
- Programmatic clients will need to use Cognito tokens to interact with the Airflow and OGC APIs. Because of the changes above, this SPS release is NOT compatible with previous releases (user and client authentication will notwork across releases).
It also contains several efficiency and performance improvements to support scalability:
- Using EC2 types with attached SSD storage
- Prioritization of downstream tasks within the same DAG Run, as opposed to starting new Tasks in other DAG Runs
- Support for sharing large amounts of data across Tasks of the same DAG using S3 mount-points
- Usability improvements for the CWL classic DAG and CWL modular DAG
- Support for periodic clean-up of the Airflow database
- Upgrading SPS to use EKS version 1.31
Finally, this SPS release includes a DAG to create an OGC Application Package from a GitHub repository: the resulting Docker container can be included in a CWL workflow to process a mission or project specific data assets.
- Users must obtain a Unity Cognito account to authenticate with the SPS Airflow UI, or to interact with the Airflow and OGC APIs.
- There are no changes in the Terraform variables required for an SPS deployment, but the .tfvars files should be regenerated to pick up the new default values (like the version of the Airflow Docker image).
- This release does not include updating the Python unity-sds OGC client library: https://github.com/unity-sds/unity-monorepo/tree/main/libs/unity-py/unity_sds_client which will not work with any new SPS deployment. This upgrade is expected to be included in the next SPS release.
-
EPIC: Airflow/Cognito Integration
-
EPIC: SPS Infrastructure
- [Enhancement]: Upgrade SPS to latest EKS version 1.30
- [New Feature]: Usability improvements to CWL DAGs
- [Bug]: Investigate why Airflow tasks get stuck
- [Enhancement]: Improve scheduling and execution of tasks
- [New Feature]: Investigate new features of Airflow 3.0
- [Task] Update tests for CWL (classic) DAG to use the latest version of the Data Services container (9.11.1)
- [New Feature]: Create DAG to periodically clean up the Airflow database
- [New feature]: Use S3 mount-points to share data across tasks in the same DAG
-
EPIC: Application Package Generation Enhancements
-
EPIC: TROPESS Support
- unity-sps: https://github.com/unity-sds/unity-sps/releases/tag/3.0.0
- unity-sps-ogc-processes-api: https://github.com/unity-sds/unity-sps-ogc-processes-api/releases/tag/2.0.0
- unity-sps-ogc-processes-api-client-python: https://github.com/unity-sds/unity-sps-ogc-processes-api-client-python/releases/tag/2.0.1
- ghcr.io/unity-sds/unity-sps/sps-airflow:3.0.0
- ghcr.io/unity-sds/unity-sps/sps-docker-cwl:3.0.0
- ghcr.io/unity-sds/unity-sps-ogc-processes-api/unity-sps-ogc-processes-api:2.0.0
- For Administrators:
- For Developers:
- For Users:
- SPS Version 2.6.0 (new)
- OGC API Version 2.0.0 (unchanged)
- OGC Python Client Version 2.0.1 (unchanged)
This release contains several performance improvements to increase the scalability of the SPS system, including:
- New EC2 types available when using the CWL DAG or CWL Modular DAG, including EC2 types with attached SSD storage.
- Prioritization of downstream tasks of running DAGs with respect to upstream tasks of queued DAGs.
- Removal access to shared EFS partition
- Using EC2 types with increased CPU and memory for the core EKS node and core Airflow Pods
Also, the Airflow database is backed up every day, and restored when the SPS system is redeployed.
Additionally, this release includes much expanded support for the SRL (Sample Return Lander) use case, demonstrating triggering and chaining of data procesisng workflows for generation pof EDR and RDR products and image format conversion.
Finally, all documentation has been revised and updated.
- Unless ports are specifically opened, this SPS installation is only accessible through the top-level MDPS proxies. Users will need to login first with their Cognito credentials, then with their Airflow credentials. This issue will be fixed when the Airflow/Cognito integration is complete.
- For the same reason, unless ports are specifically opened, the Airflow and OGC APIs will not be accessible.
- This release is backward compatible with the previous version: it includes additional functionality and performance improvements, but there are no API changes.
- Because some configuration parameters have changed, the .tfvars files used to deploy EKS, Karpenter and Airflow via Terraform should be re-generated to pick up the new values.
- unity-sps: https://github.com/unity-sds/unity-sps/releases/tag/2.6.0
- unity-sps-ogc-processes-api: https://github.com/unity-sds/unity-sps-ogc-processes-api/releases/tag/2.0.0
- unity-sps-ogc-processes-api-client-python: https://github.com/unity-sds/unity-sps-ogc-processes-api-client-python/releases/tag/2.0.1
- EPIC: SPS Infrastructure
- [Enhancement]: Update the Management Console to use the latest SDS version
- [Bug]: Fix placement of Pods over Nodes
- [Enhancement] Remove access to shared EFS partition in the CWL DAG
- [Bug]: Update CPU and memory limits for worker nodes
- [New Feature]: Experiment with using EC2 types with attached storage for better performance
- [New Feature]: Log the EC2 Instance Id
- [New Feature]: Scale up the SPS core resources
- [New Feature]: Prioritize downstream tasks
- EPIC: Deployment Enhancements
- EPIC: Documentation Updates
- EPIC: Continue Application Package Standardization
- EPIC: TROPESS Support
- EPIC: Demonstrate Space Use Case
- [Enhancement]Create SRL Initiator for EDRGen (.dat + .emd)
- [New Feature]: Airflow DAG for EDRgen
- [New Feature]: Airflow DAG for RDRgen
- [New Feature]: Initiators for EDRgen processing
- [New Feature]: Initiators for RDRgen processing
- [New Feature]: Prototype routing via a DAG
- [New Feature]: Investigate router DAG configuration via Airflow Configuration, SSM, S3 file
- ghcr.io/unity-sds/unity-sps/sps-airflow:2.6.0
- ghcr.io/unity-sds/unity-sps/sps-docker-cwl:2.6.0
- ghcr.io/unity-sds/unity-sps-ogc-processes-api/unity-sps-ogc-processes-api:2.0.0
- For Administrators:
- For Developers:
- For Users:
- SPS Version 2.4.0 (new)
- OGC API Version 2.0.0 (unchanged)
- OGC Python Client Version 2.0.1 (new)
-
The CWL DAG ("cwl_dag.py") has changed: instead of requesting a certain amount of CPU ("request_cpu") and memory ("request_memory)" separately, the client need to specify the desired EC2 instance ("request_instance_type"). The value needs to be one of those shown in the drop-down menu of the User Interface (for example, "r7i.xlarge", "c6i.8xlarge", etc.). The client can still specify the desired size of the attached disk ("request_storage").
-
The SPS User Interface (Airflow) can be accessed via either of the following methods:
- The top-level URL (for example, "https://www.dev.mdps.mcp.nasa.gov:4443/unity/dev/sps/home"), which will require a Cognito login and (temporarily) an additional Airflow login.
- The mid-level proxy (for example, "http://unity-dev-httpd-alb-1659665968.us-west-2.elb.amazonaws.com:8080/unity/dev/sps/"), which requires an Airflow login, and is only accessible within the JPL VPC.
-
A new prototype CWL Modular DAG ("cwl_dag_modular.py") is available, which will run the stage-in, process and stage-out tasks as separate CWL workflow, but all in the same EC2 node. To use this DAG, users will have to rewrite and modularize their existing CWL workflows which were written as a single document.
- unity-sps: https://github.com/unity-sds/unity-sps/releases/tag/2.4.0
- unity-sps-ogc-processes-api: https://github.com/unity-sds/unity-sps-ogc-processes-api/releases/tag/2.0.0
- unity-sps-ogc-processes-api-client-python: https://github.com/unity-sds/unity-sps-ogc-processes-api-client-python/releases/tag/2.0.1
Release tickets: https://github.com/orgs/unity-sds/projects/3/views/53
- EPIC: Airflow/WPS-T Integration
- EPIC:
SPS Infrastructure - EPIC: Documentation
- EPIC: Airflow/Cognito Integration
- EPIC: Demonstrated Scalability
- EPIC: Unity Marketplace
- EPIC: Application Package Standardization
- ghcr.io/unity-sds/unity-sps/sps-airflow:2.4.0
- ghcr.io/unity-sds/unity-sps/sps-docker-cwl:2.4.0
- ghcr.io/unity-sds/unity-sps-ogc-processes-api/unity-sps-ogc-processes-api:2.0.0
- For Administrators:
- For Developers:
- For Users:
- SPS Version 2.2.0
- OGC API Version 2.0.0
- OGC Python Client Version 2.0.0
- unity-sps: https://github.com/unity-sds/unity-sps/releases/tag/2.2.0
- unity-sps-ogc-processes-api: https://github.com/unity-sds/unity-sps-ogc-processes-api/releases/tag/2.0.0
- unity-sps-ogc-processes-api-client-python: https://github.com/unity-sds/unity-sps-ogc-processes-api-client-python/releases/tag/2.0.0
- EPIC:
Security - EPIC:
Scaling- [New Feature]: Increase ephemeral disk space for Airflow workers
- [New Feature]: Enable users to select the EC2 type to execute a workload
- [New Feature]: Set the DAG run status to "failed" if the main worker task failed
- [[New Feature]: Demonstrate use of ECR within an Airflow DAG (#186)
- EPIC:
Airflow/WPS-T Integration - EPIC:
Production Venue Deployments - EPIC:
SPS Infrastructure
- ghcr.io/unity-sds/unity-sps/sps-airflow:2.2.0
- ghcr.io/unity-sds/unity-sps/sps-docker-cwl:2.2.0
- ghcr.io/unity-sds/unity-sps-ogc-processes-api/unity-sps-ogc-processes-api:2.0.0
- For Administrators:
- For Developers:
- For Users:
- SPS Version 2.1.0
- OGC API Version 1.0.0
- OGC Python Client Version 1.0.0
- unity-sps: https://github.com/unity-sds/unity-sps/releases/tag/2.1.0
- unity-sps-ogc-processes-api: https://github.com/unity-sds/unity-sps-ogc-processes-api/releases/tag/1.0.0
- unity-sps-ogc-processes-api-client-python: https://github.com/unity-sds/unity-sps-ogc-processes-api-client-python/releases/tag/1.0.0
- EPIC:
Airflow/WPS-T Integration - EPIC:
Airflow Scaling Improvements- [New Feature]: Implement autoscaling of Kubernetes worker nodes
- [New Feature]: Nightly test for SBG end-to-end workflow
- [Dependency]: PSE to provide analysis of resources needed by each SBG Task
- [New Feature] Only use MCP Golden AMIs
- [New Feature] Configure multiple pools of Kubernetes nodes
- [New Feature] Retrieve venue-dependent parameters from SSM
- [New Feature] Store deployment parameters as AIRFLOW variables
- [New Feature] Enable Airflow "plugins" folder
- [Task] Add TESTING.md file to SPS repo
- EPIC:
SPS Infrastructure - EPIC:
SPS Security
- ghcr.io/unity-sds/unity-sps/sps-airflow:2.1.0
- ghcr.io/unity-sds/unity-sps/sps-docker-cwl:2.1.0
- ghcr.io/unity-sds/unity-sps-ogc-processes-api/unity-sps-ogc-processes-api:1.0.0
- SPS Deployment with Terraform
- Interacting with an Existing SPS Deployment
- SPS Airflow Custom Docker Image Build Instructions
- Tutorial: Register, Execute, and Unregister a Process using the OGC API - Processes
- Tutorial: Register and Execute a CWL Workflow
Version 2.0.0
- unity-sps: https://github.com/unity-sds/unity-sps/releases/tag/2.0.0
- unity-sps-ogc-processes-api: https://github.com/unity-sds/unity-sps-ogc-processes-api/releases/tag/2.0.0
- EPIC:
Airflow Deployment- [New Feature]: Deploy EKS to MCP venues with Terraform
- [New Feature]: Deploy Airflow to MCP venues with Terraform (no adaptation yet)
- [New Feature]: Create a CWL Operator or simply a CWL DAG to execute a CWL workflow
- [New Feature]: Using shared disk space for inter-task communication
- [New Feature]: Implement autoscaling of Kubernetes pods on a given set of worker nodes
- [Task] Migrate the new SPS with Airflow code to unity-sps repository
- [New Feature] Allow choosing EC2 types when deploying an EKS cluster
- [New Feature]: Allow propagation of permissions to Docker containers
- [New feature] Add Terraform parameters to customize the amount of disk available to each Pod
- EPIC:
SBG Venue Deployment - EPIC:
SPS Infrastructure
- ghcr.io/unity-sds/unity-sps/sps-airflow:2.0.0
- ghcr.io/unity-sds/unity-sps/sps-docker-cwl:2.0.0
- ghcr.io/unity-sds/unity-sps-ogc-processes-api/unity-sps-ogc-processes-api:2.0.0
- SPS EKS Cluster Provisioning with Terraform
- SPS Airflow Deployment with Terraform
- SPS Airflow Custom Docker Image Build Instructions
- unity-sps-prototype: https://github.com/unity-sds/unity-sps-prototype/releases/tag/1.2.0
- unity-sps-api: https://github.com/unity-sds/unity-sps-api/releases/tag/1.2.0
- unity-sps-register_job: https://github.com/unity-sds/unity-sps-register_job/releases/tag/1.2.0
- unity-sps-workflows: https://github.com/unity-sds/unity-sps-workflows/releases/tag/1.2.0
- ades_wpst: https://github.com/unity-sds/ades_wpst/releases/tag/1.2.0
- unity-sps : https://github.com/unity-sds/unity-sps/releases/tag/1.2.0
- EPIC:
workflow metadata- [unity-sps-prototype #210] [New Feature]: Store arbitrary labels in the Jobs Database
- [unity-sps-prototype #211] [New Feature]: Query the Jobs Metadata by arbitrary labels
- [unity-sps-prototype #217] [Risk]: Evaluate scalability of WPS-T API for querying the Jobs Database
- EPIC:
chirp-workflow-execution- [unity-sps-workflows #8] [Dependency]: Stub implementation for CHIRP workflow
- [unity-sps-workflows #10] [Dependency]: Docker container for execution of CHIRP rebinning PGE
- [unity-sps-workflows #11] [Dependency]: Full implementation of CHIRP workflow
- [unity-sps-workflows #12] [Dependency]: Docker container for CMR search
- [unity-sps-workflows #13] [Dependency]: Docker container for Cataloging
- [unity-sps-workflows #14] [Risk]: Processing large data volumes with CWL and Docker
- [unity-sps-prototype #192 ] [New Feature]: Add an optional PVC to an SPS deployment
- [unity-sps-prototype #195] [Dependency]: Docker container for stage-in step
- [unity-sps-prototype #227] [New Feature]: Use stage EFS as data temporary location for CHIRP execution
- EPIC:
no magic- [unity-sps-prototype #132] As a user, i want to be explicit about inputs into my process execution (no magic)!
- [unity-sps-prototype #157] [New Feature]: Pass environment variables to a Docker container execution
- EPIC:
sps-improvements-23.3- [unity-sps-prototype #221] [Dependency]: Tagging U-SPS resources
- [unity-sps-prototype #222] [Dependency]: Add SPS API URL to the SSM store
- [unity-sps-prototype #230] [Bug]: One SPS deployment can accidentally execute another SPS deployments deployed process containers
- EPIC:
workflow-label-inputs- [unity-sps-prototype #160] [New Feature]: Pass arbitrary labels when requesting a job execution
- [unity-sps-prototype #181] [Risk]: Define how labels should be specified during workflow submission
- ghcr.io/unity-sds/unity-sps-prototype/hysds-core:unity-v1.2.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-ui-remote:unity-v1.2.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-mozart:unity-v1.2.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-grq2:unity-v1.2.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-verdi:unity-v1.2.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-factotum:unity-v1.2.0
- ghcr.io/unity-sds/unity-sps-prototype/ades-wpst-api:unity-v1.2.0
- ghcr.io/unity-sds/unity-sps-prototype/sps-api:unity-v1.2.0
- ghcr.io/unity-sds/unity-sps-prototype/sps-hysds-pge-base:unity-v1.2.0
- docker.elastic.co/elasticsearch/elasticsearch:7.9.3
- docker.elastic.co/logstash/logstash:7.10.2
- rabbitmq:3.11.13-management
- redis:7.0.10
- docker:23.0.3-dind
- busybox:1.36.0
- Tutorial on using the WPS-T API to register an application package and to execute a job
- SPS API with examples
- Tutorial on pre-warming a U-SPS cluster
- Description of the Jobs Database: architecture and usage
- unity-sps : https://github.com/unity-sds/unity-sps/releases/tag/1.1.0
- unity-sps-prototype: https://github.com/unity-sds/unity-sps-prototype/releases/tag/1.1.0
- unity-sps-api: https://github.com/unity-sds/unity-sps-api/releases/tag/1.1.0
- unity-sps-register_job: https://github.com/unity-sds/unity-sps-register_job/releases/tag/1.1.0
- unity-sps-workflows: https://github.com/unity-sds/unity-sps-workflows/releases/tag/1.1.0
- ades_wpst: https://github.com/unity-sds/ades_wpst/releases/tag/1.1.0
- EPIC #7:
jobs-database- [unity-sps-prototype #185] [Epic]: Implement Jobs Database
- [unity-sps-prototype #186] [New Feature]: Send SNS message for job creation task
- [unity-sps-prototype #187] [New Feature]: Send SNS message for job completion task
- [unity-sps-prototype #188] [New Feature]: Consume SNS message for job creation task and job completion tasks
- [unity-sps-prototype #189] [New Feature]: Deploy jobs database
- [unity-sps-prototype #193) [New Feature]: Create SNS topic, SQS queue and Lambda function for Jobs Database as part of U-SPS deployment
- [unity-sps-prototype $194] [Bug]: Prevent duplicate documents in jobs database
- EPIC #10:
chirp-workflow-execution- [unity-sps-workflows #8] [Dependency]: Stub implementation for CHIRP workflow
- [unity-sps-workflows #12] [Dependency]: Docker container for CMR search
- [unity-sps-workflows #14] [Risk]: Processing large data volumes with CWL and Docker
- [unity-sps-prototype #192] [New Feature]: Add an optional PVC to an SPS deployment
- EPIC #1:
min/max scaling- [unity-sps-prototype #141] As a project Manager, i want to set maximum and minimum limits on scaling for processing at venue creation time (static)
- EPIC #4:
no magic- [unity-sps-prototype #132] As a user, i want to be explicit about inputs into my process execution (no magic)!
- ghcr.io/unity-sds/unity-sps-prototype/hysds-core:unity-v1.1.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-ui-remote:unity-v1.1.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-mozart:unity-v1.1.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-grq2:unity-v1.1.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-verdi:unity-v1.1.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-factotum:unity-v1.1.0
- ghcr.io/unity-sds/unity-sps-prototype/ades-wpst-api:unity-v1.1.0
- ghcr.io/unity-sds/unity-sps-prototype/sps-api-fork:unity-v1.1.0
- ghcr.io/unity-sds/unity-sps-prototype/sps-hysds-pge-base:unity-v1.1.0
- docker.elastic.co/logstash/logstash:7.10.2
- rabbitmq:3.11.13-management
- redis:7.0.10
- docker:23.0.3-dind
- busybox:1.36.0
- Tutorial on using the WPS-T API to register an application package and to execute a job
- SPS API with examples
- Tutorial on pre-warming a U-SPS cluster
- MCP Test:
- Processing Endpoint (WPS-T API): http://a720fb4de892844bf884f037c17bb583-1070798053.us-west-2.elb.amazonaws.com:5001
- Scaling Endpoint (SPS API): http://a7096fc6842e84da688b45586d194498-2116025617.us-west-2.elb.amazonaws.com:5002
- MCP Dev:
- Processing Endpoint (WPS-T API): http://aa17aedf4454a4cc596a67a1efb73411-1350404365.us-west-2.elb.amazonaws.com:5001
- Scaling Endpoint (SPS API): http://a440158f49fab4278bdcf2bcb145082b-625745.us-west-2.elb.amazonaws.com:5002
- unity-sps : https://github.com/unity-sds/unity-sps/releases/tag/1.0.0
- unity-sps-prototype: https://github.com/unity-sds/unity-sps-prototype/releases/tag/1.0.0
- unity-sps-api: https://github.com/unity-sds/unity-sps-api/releases/tag/1.0.0
- unity-sps-register_job:
- unity-sps-workflows: https://github.com/unity-sds/unity-sps-workflows/releases/tag/1.0.0
- ades_wpst: https://github.com/unity-sds/ades_wpst/releases/tag/1.0.0
- EPIC #130:
ancillary data- [unity-sps-prototype #148] Mount EFS partitions on U-SPS cluster
- EPIC #146:
automatic testing- [unity-sps-prototype #153] Smoke test to deploy a simple process and execute a job request
- EPIC #132:
no magic- [unity-sps #154]: Pass explicit input parameters to a WPS-T job request
- [unity-sps-prototype #159]: Return the input parameters as part of the WPS-T DescribeProcess method
- [unity-sps-prototype #167]: The WPS-T method to register a process may time out while building the PGE Docker image
- EPIC #142:
processing-instance-types- [unity-sps-prototype #142]: As a project manager, i want to set default compute types for processing nodes at deploy time, so that i can predict costs for the system (Static)
- EPIC #162:
u-sps-api- [unity-sps-prototype #163]: New SPS API with stub pre-warm method
- [unity-sps-prototype #167]: Refactor the HySDS workers as a Kubernetes DaemonSet
- [unity-sps-prototype #167]: Scale the number of worker nodes in the Kubernetes cluster
- ghcr.io/unity-sds/unity-sps-prototype/hysds-core:unity-v1.0.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-ui-remote:unity-v1.0.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-mozart:unity-v1.0.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-grq2:unity-v1.0.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-verdi:unity-v1.0.0
- ghcr.io/unity-sds/unity-sps-prototype/hysds-factotum:unity-v1.0.0
- ghcr.io/unity-sds/unity-sps-prototype/ades-wpst-api:unity-v1.0.0
- ghcr.io/unity-sds/unity-sps-prototype/sps-api-fork:unity-v1.0.0
- ghcr.io/unity-sds/unity-sps-prototype/sps-hysds-pge-base:unity-v1.0.0
- docker.elastic.co/logstash/logstash:7.10.2
- rabbitmq:3.11.13-management
- redis:7.0.10
- docker:23.0.3-dind
- busybox:1.36.0
- Tutorial on using the WPS-T API to register an application package and to execute a job
- SPS API with examples
- Tutorial on pre-warming a U-SPS cluster
- MCP Test:
- Processing Endpoint (WPS-T API): http://a720fb4de892844bf884f037c17bb583-1070798053.us-west-2.elb.amazonaws.com:5001
- Scaling Endpoint (SPS API): http://a7096fc6842e84da688b45586d194498-2116025617.us-west-2.elb.amazonaws.com:5002
- MCP Dev:
- Processing Endpoint (WPS-T API): http://aa17aedf4454a4cc596a67a1efb73411-1350404365.us-west-2.elb.amazonaws.com:5001
- Scaling Endpoint (SPS API): http://a440158f49fab4278bdcf2bcb145082b-625745.us-west-2.elb.amazonaws.com:5002