Skip to content

Commit 857a9d9

Browse files
trisperasourcery-ai[bot]
trispera
and
sourcery-ai[bot]
authored
WIP: docs(guides): Add new admin guides (#32)
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
1 parent 9dec863 commit 857a9d9

File tree

16 files changed

+7851
-414
lines changed

16 files changed

+7851
-414
lines changed

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
site/
22
_site/
3+
.venv

‎Makefile‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,11 +40,17 @@ i: install
4040
.PHONY: venv
4141
venv:
4242
@echo "\nCreating a virtual environment ++++++++++++++++++++++++++++++++++++++++++++++++\n"
43-
@python -m venv .venv
43+
@if command -v python3 >/dev/null 2>&1; then \
44+
python3 -m venv .venv; \
45+
elif command -v python >/dev/null 2>&1; then \
46+
python -m venv .venv; \
47+
else \
48+
echo "Python interpreter not found" >&2; exit 1; \
49+
fi
4450
@echo "\nSummary +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n"
4551
@echo "Virtual environment created successfully."
4652
@echo "To activate the environment for this shell session, run:"
4753
@echo "source .venv/bin/activate"
4854

4955
.PHONY: v
50-
v: venv
56+
v: venv

‎docs/guides/guide-admin/cloudregistry.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
## Synopsis
2+
3+
GA4GH Service Registry API implementation for the ELIXIR Cloud.
4+
5+
Service entries comply with the [external service schema](https://github.com/ga4gh-discovery/ga4gh-service-registry/blob/8c45be52940db92c2fa1cd821519c271c22b1c4c/service-registry.yaml#L158) defined in the [GA4GH Service Registry API][ga4gh-service-registry]
6+
7+
Developers can find the API documentation [here](https://cloud-registry.readthedocs.io/en/latest/)
8+
9+
## Installation
10+
11+
You can find a Helm chart in the [GitHub repository](https://github.com/elixir-cloud-aai/cloud-registry/tree/dev/deployment) of Cloud-registry
12+
13+
Follow these instructions
14+
15+
- Install [Helm][helm-install]
16+
- Clone the [Cloud-registry repository](https://github.com/elixir-cloud-aai/cloud-registry/)
17+
18+
```sh
19+
git clone https://github.com/elixir-cloud-aai/cloud-registry.git
20+
```
21+
22+
- Browse to `deployment` to find the `Chart.yaml` and the `values.yaml` files
23+
24+
## Usage
25+
26+
First you must create a namespace in Kubernetes in which to deploy Cloud-registry. The
27+
commands below assume that everything is created in the context of this
28+
namespace. How the namespace is created depends on the cluster, so we won't
29+
document it here.
30+
31+
You need to edit the `values.yaml` file
32+
33+
After this you can deploy Cloud-registry using `helm`:
34+
35+
```bash
36+
helm install cloud-registry . -f values.yaml
37+
```
38+
39+
### Updates
40+
41+
If you want to edit any of the Deployments, you can update them with
42+
`helm` and the `values.yaml` file. Once edited, you can run this command:
43+
44+
```bash
45+
helm upgrade cloud-registry . -f values.yaml
46+
```
47+
48+
## Technical details
49+
50+
### MongoDB
51+
52+
The MongoDB database is deployed using:
53+
54+
- `templates/mongo-deploy.yaml`
55+
56+
### Cloud-registry
57+
58+
TRS-Filer is deployed using:
59+
60+
- `templates/cloud-registry-deploy.yaml`
61+
62+
## Destroy
63+
64+
Simply run:
65+
66+
```bash
67+
helm uninstall cloud-registry
68+
```
69+

‎docs/guides/guide-admin/cwlwes.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
## Synopsis
2+
3+
Microservice implementing the [Global Alliance for Genomics and
4+
Health][ga4gh] (GA4GH) [Workflow Execution Service][ga4gh-wes] (WES)
5+
API specification for the execution of workflows written in the [Common
6+
Workflow Language](https://www.commonwl.org/) (CWL).
7+
8+
cwl-WES is a core service of the [ELIXIR Cloud & AAI
9+
project][elixir-cloud-aai-github].
10+
11+
## Description
12+
13+
cwl-WES (formerly: WES-ELIXIR) is a Flask/Gunicorn
14+
application that makes use of [Connexion](https://github.com/ga4gh/workflow-execution-service-schemas) to implement the
15+
[GA4GH WES OpenAPI specification][ga4gh-wes]. It enables clients/users
16+
to execute [CWL](https://www.commonwl.org) workflows in the cloud via a [GA4GH Task Execution
17+
Service][ga4gh-tes] (TES)-compatible execution backend (e.g.,
18+
[TESK][tesk] or [Funnel][funnel]). Workflows can be sent for execution,
19+
previous runs can be listed, and the status and run information of individual
20+
runs can be queried. The service leverages [cwl-tes][res-cwl-tes] to
21+
interpret [CWL](https://www.commonwl.org) workflows, break them down into individual tasks and
22+
emit [GA4GH TES][ga4gh-tes]-compatible HTTP requests to a configured
23+
[TES][ga4gh-tes] instance. Access to endpoints can be configured to require
24+
JSON Web Token-based access tokens, such as those issued by
25+
[ELIXIR AAI](https://elixir-europe.org/platforms/compute/aai). Run information is stored in a
26+
MongoDB database.
27+
28+
Note that development is currently in beta stage.
29+
Further test deployments can be found at the [ELIXIR Cloud & AAI's resource
30+
listings](https://github.com/elixir-cloud-aai/elixir-cloud-aai/blob/dev/resources/resources.md).
31+
32+
cwl-WES is developed and maintained by the [ELIXIR Cloud & AAI
33+
project][elixir-cloud], a multinational effort aimed at establishing and
34+
implementing [FAIR][fair] research in the Life Sciences.
35+
36+
## Installation
37+
38+
You can find a Helm chart in the [GitHub repository](https://github.com/elixir-cloud-aai/cwl-wes/tree/dev/deployment) of CWL-WES
39+
40+
Follow these instructions
41+
42+
- Install [Helm][helm-install]
43+
- Clone the [CWL-WES repository](https://github.com/elixir-cloud-aai/cwl-wes/)
44+
45+
```sh
46+
git clone https://github.com/elixir-cloud-aai/cwl-wes.git
47+
```
48+
49+
- Browse to `deployment` to find the `Chart.yaml` and the `values.yaml` files
50+
51+
## Usage
52+
53+
First you must create a namespace in Kubernetes in which to deploy CWL-WES. The
54+
commands below assume that everything is created in the context of this
55+
namespace. How the namespace is created depends on the cluster, so we won't
56+
document it here.
57+
58+
You need to edit the `values.yaml` file
59+
60+
After this you can deploy CWL-WES using `helm`:
61+
62+
```bash
63+
helm install CWL-WES . -f values.yaml
64+
```
65+
66+
### Updates
67+
68+
If you want to edit any of the Deployments, you can update them with
69+
`helm` and the `values.yaml` file. Once edited, you can run this command:
70+
71+
```bash
72+
helm upgrade CWL-WES . -f values.yaml
73+
```
74+
75+
## Technical details
76+
77+
### MongoDB
78+
79+
The MongoDB database is deployed using:
80+
81+
- `templates/mongodb-deployment.yaml`
82+
83+
### RabbitMQ
84+
85+
The message broker RabbitMQ that allows the app to communicate with the
86+
worker is deployed using:
87+
88+
- `templates/rabbitmq/rabbitmq-deployment.yaml`
89+
90+
### CWL-WES
91+
92+
CWL-WES consists of a Flask server and a Celery worker.
93+
There are deployed using:
94+
95+
- `templates/wes-deployment.yaml`
96+
- `templates/celery-deployment.yaml`
97+
98+
## Destroy
99+
100+
Simply run:
101+
102+
```bash
103+
helm uninstall cwl-wes
104+
```
105+

‎docs/guides/guide-admin/drsfiler.md‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
## Synopsis
2+
3+
Microservice implementing the [Global Alliance for Genomics and
4+
Health (GA4GH)][ga4gh] [Data Repository Service][ga4gh-drs] (DRS)
5+
API specification.
6+
7+
## Installation
8+
9+
You can find a Helm chart in the [GitHub repository](https://github.com/elixir-cloud-aai/drs-filer/tree/dev/deployment) of DRS-Filer
10+
11+
Follow these instructions
12+
13+
- Install [Helm][helm-install]
14+
- Clone the [DRS-Filer repository](https://github.com/elixir-cloud-aai/drs-filer/)
15+
16+
```sh
17+
git clone https://github.com/elixir-cloud-aai/drs-filer.git
18+
```
19+
20+
- Browse to `deployment` to find the `Chart.yaml` and the `values.yaml` files
21+
22+
## Usage
23+
24+
First you must create a namespace in Kubernetes in which to deploy DRS-Filer. The
25+
commands below assume that everything is created in the context of this
26+
namespace. How the namespace is created depends on the cluster, so we won't
27+
document it here.
28+
29+
You need to edit the `values.yaml` file
30+
31+
After this you can deploy DRS-Filer using `helm`:
32+
33+
```bash
34+
helm install drs-filer . -f values.yaml
35+
```
36+
37+
### Updates
38+
39+
If you want to edit any of the Deployments, you can update them with
40+
`helm` and the `values.yaml` file. Once edited, you can run this command:
41+
42+
```bash
43+
helm upgrade drs-filer . -f values.yaml
44+
```
45+
46+
## Technical details
47+
48+
### MongoDB
49+
50+
The MongoDB database is deployed using:
51+
52+
- `templates/mongo-deploy.yaml`
53+
54+
### DRS-Filer
55+
56+
DRS-Filer is deployed using:
57+
58+
- `templates/drs-filer-deploy.yaml`
59+
60+
## Destroy
61+
62+
Simply run:
63+
64+
```bash
65+
helm uninstall drs-filer
66+
```
67+

‎docs/guides/guide-admin/funnel.md‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
Follow these instructions if you wish to deploy a TES endpoint in front of your
2+
HPC/HTC cluster (currently tested with [Slurm][slurm] and [OpenPBS][openpbs].
3+
4+
- Make sure the build dependencies `make` and [Go 1.21+][go-install] are
5+
installed, `GOPATH` is set and `GOPATH/bin` is added to `PATH`.
6+
7+
For example, in Ubuntu this can be achieved via:
8+
9+
```sh
10+
sudo apt update
11+
sudo apt install make golang-go
12+
export GOPATH=/your/desired/path
13+
export PATH=$GOPATH/bin:$PATH
14+
go version
15+
```
16+
17+
- Clone the repository:
18+
19+
```sh
20+
git clone https://github.com/ohsu-comp-bio/funnel.git
21+
```
22+
23+
- Build Funnel:
24+
25+
```sh
26+
cd funnel
27+
make
28+
```
29+
30+
- Test the installation by starting the Funnel server with:
31+
32+
```sh
33+
funnel server run
34+
```
35+
36+
If all works, Funnel should be ready for deployment on your HPC/HTC.
37+
38+
Alternatively, you can install Funnel via Homebrew:
39+
40+
```sh
41+
brew tap ohsu-comp-bio/formula
42+
brew install funnel@0.11
43+
```
44+
45+
Source: [Funnel website](https://ohsu-comp-bio.github.io/funnel/)
46+
47+
### Slurm
48+
49+
For the use of Funnel with Slurm, make sure the following conditions are met:
50+
51+
1. The `funnel` binary must be placed in a server with access to Slurm.
52+
2. A config file must be created and placed on the same server. [This
53+
file][funnel-config-slurm] can be used as a starting point.
54+
3. If we would like to deploy Funnel as a Systemd service,
55+
[this file][funnel-config-slurm-service] can be used as a template. Set the
56+
correct paths to the `funnel` binary and config file.
57+
58+
If successful, Funnel should be listening on port `8080`.
59+
60+
### OpenPBS
61+
62+
!!! warning "Under construction"
63+
More info coming soon...

‎docs/guides/guide-admin/images/overview_protes.svg‎

Lines changed: 1 addition & 0 deletions
Loading

0 commit comments

Comments
 (0)