Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,9 @@
# kube-deploy
Kustomizations and Argo Applications to deploy OpenCHAMI and supporting services

Kustomizations and Argo Applications to deploy OpenCHAMI and supporting
services.

https://github.com/OpenCHAMI/kube-deploy/blob/main/kustomize/README.md
describes a demo workflow that installs services and a virtualized test
node. You can use these in a KinD cluster to try a Kubernetes deployment of
OpenCHAMI on your laptop.
5 changes: 4 additions & 1 deletion example-local/services/app.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,10 @@ spec:
- ServerSideApply=true
- SkipDryRunOnMissingResource=true
source:
path: 'kustomize/services'
# These must match the private repo path. The demo requires copying the
# full example-local directory into the private repo, rather than its
# contents into the top level of the private repo.
path: 'example-local/services'
repoURL: 'https://github.com/OpenCHAMI/kube-deploy'
targetRevision: main
destination:
Expand Down
6 changes: 3 additions & 3 deletions example-local/services/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -132,13 +132,13 @@ replacements:
images:
- name: power-control-stub
newName: ghcr.io/openchami/pcs
newTag: v2.11
newTag: v2.12.0
- name: bss-stub
newName: ghcr.io/openchami/bss
newTag: 1.32
- name: smd-stub
newName: ghcr.io/openchami/smd
newTag: 2.19
- name: cloud-init-stub
newName: ghcr.io/openchami/smd
newTag: 1.3
newName: ghcr.io/openchami/cloud-init
newTag: 1.4
5 changes: 4 additions & 1 deletion example-local/test-node/app.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,10 @@ spec:
- ServerSideApply=true
- SkipDryRunOnMissingResource=true
source:
path: 'kustomize/test-node'
# These must match the private repo path. The demo requires copying the
# full example-local directory into the private repo, rather than its
# contents into the top level of the private repo.
path: 'example-local/test-node'
repoURL: 'https://github.com/OpenCHAMI/kube-deploy'
targetRevision: main
destination:
Expand Down
8 changes: 4 additions & 4 deletions example-local/test-node/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- https://github.com/OpenCHAMI/kube-deploy//kustomize/test-node/namespace/?timeoute=120&ref=rainest-argo
- https://github.com/OpenCHAMI/kube-deploy//kustomize/test-node/kubevirt/?timeoute=120&ref=rainest-argo
- https://github.com/OpenCHAMI/kube-deploy//kustomize/test-node/kubevirtbmc/?timeoute=120&ref=rainest-argo
- https://github.com/OpenCHAMI/kube-deploy//kustomize/test-node/vm/?timeoute=120&ref=rainest-argo
- https://github.com/OpenCHAMI/kube-deploy//kustomize/test-node/namespace/?timeoute=120&ref=main
- https://github.com/OpenCHAMI/kube-deploy//kustomize/test-node/kubevirt/?timeoute=120&ref=main
- https://github.com/OpenCHAMI/kube-deploy//kustomize/test-node/kubevirtbmc/?timeoute=120&ref=main
- https://github.com/OpenCHAMI/kube-deploy//kustomize/test-node/vm/?timeoute=120&ref=main
# TODO installing cert-manager using Argo makes Argo unhappy with webhook certs.
# unknown issue prevents Argo from trusting cert-manager's. Pending a fix for that,
# it must be installed separately:
Expand Down
2 changes: 1 addition & 1 deletion example-local/test-node/userdata
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ users:
#
# will provide a passwd string you can use with hashed_passwd for password login:
#
# hashed_passwd: "$6$rounds=4096$bEi3oQdeI7.odStr$tW/Tvw.kwYCVIiNse11C9yyBOzb/kAen2WUa1kUZXYSisL8DHrH.cRJqjcES9J1T8JL3duiV6Z6DcMkQtltj7."
hashed_passwd: "$6$rounds=FAKEVALUE.FAKEVALUE.FAKEVALUE.FAKEVALUE"
# Password is disabled by default, which has the knockon effect of not allowing sudo either
lock_passwd: false
sudo: "ALL=(ALL) ALL"
Expand Down
85 changes: 56 additions & 29 deletions kustomize/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ Docker network.
Install cert-manager and ArgoCD:

```
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.18.2/cert-manager.yaml
kubectl apply --server-side -f https://github.com/cert-manager/cert-manager/releases/download/v1.18.2/cert-manager.yaml
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
kubectl apply --server-side -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
```

I'd originally included cert-manager as part of the virtual machine
Expand All @@ -42,7 +42,7 @@ change the admin Service to a LoadBalancer and set up your account:
kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}'
```

Retrieve the Service external IP and initial password:
Retrieve the Service external IP and initial password. For example:

```
kubectl get svc -n argocd argocd-server -ojson | jq .status.loadBalancer.ingress[0].ip -r
Expand All @@ -53,20 +53,36 @@ kubectl get svc -n argocd argocd-server -ojson | jq .status.loadBalancer.ingress
argocd admin initial-password -n argocd
```

Store the external IP in a variable for easy access:

```
ARGOCD_SERVER=$(kubectl get svc -n argocd argocd-server -ojson | jq .status.loadBalancer.ingress[0].ip -r)
```

Log in using the initial password:

```
argocd login 172.19.128.1
argocd login ${ARGOCD_SERVER}
```

You will see a warning regarding a self-signed certificate. You can respond `y`
to proceed insecurely for the demo.

The username is `admin` and the password is the value you saw earlier after
running `argocd admin initial-password`.

Update the password to something else:

```
argocd account update-password
```

You'll now be able to log in to the web UI at https://172.19.128.1/ (the
username is `admin`). This guide will use the CLI to create and sync
username is `admin`). Note that your IP may differ. Your browser cannot use the
shell variable, and you'll need to use the IP directly if you don't have a
means to spoof DNS resolution for ti.

This guide will use the CLI to create and sync
Applications, but the UI's useful for seeing the status of managed resources.

### Set up a private repository
Expand All @@ -77,17 +93,26 @@ Application manifests, resources must be available in a git repository.
This repository seeks to include generic manifests that can be applied in any
environment, and does not include site-local configuration or secrets.

You'll need to create your own private fork of https://github.com/rainest/openchami-kustomize-local
to hold your overlay.

In your fork:

1. Change `remote/services/app.yaml` and `remote/test-node/app.yaml` to use
your fork in `spec.source.repoURL`.
1. Edit `remote/services/kustomization.yaml` to set your DB passwords
under the `secretGenerator` section.
1. Edit `remote/test-node/userdata` to set an SSH key and hashed password for
the virtual machine.
To manage your own installation, you will want to create your own private
repository, with a copy of the [example-local
directory](https://github.com/OpenCHAMI/kube-deploy/tree/main/example-local).

In your private copy:

1. Change `services/app.yaml` and `test-node/app.yaml` to use your fork in
`spec.source.repoURL`. `spec.source.targetRevision` must match your desired branch (the
example applications use `main` by default).
1. Edit `services/kustomization.yaml` to set your DB passwords under the
`secretGenerator` section. Replace the `CHANGEME` placeholder value with
something else.
1. Edit `test-node/userdata` to set an SSH key for the virtual machine. Replace
the `ssh-ed25519 CHANGEME username@host` placeholder with the value of one
of your SSH public keys (for example, the value in `~/.ssh/id_ed25519.pub`.
1. Edit `test-node/userdata` to set a hashed password for the virtual machine
user. Run the `echo "CHANGEME" | mkpasswd --method=SHA-512 --rounds=4096 --stdin`
command with your desired password instead of `CHANGEME`. Replace the
`hashed_passwd: "$6$rounds=FAKEVALUE.FAKEVALUE.FAKEVALUE.FAKEVALUE"`
placeholder with the actual value returned by `mkpasswd`.

Create a deploy key (for [GitLab](https://docs.gitlab.com/user/project/deploy_keys/)
or [GitHub](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/managing-deploy-keys))
Expand All @@ -102,8 +127,8 @@ argocd repo add git@github.com:myuser/myfork --username myuser --ssh-private-key
From your fork checkout directory:

```
kubectl apply -n argocd -f test-node/app.yaml
kubectl apply -n argocd -f services/app.yaml
kubectl apply --server-side -n argocd -f test-node/app.yaml
kubectl apply --server-side -n argocd -f services/app.yaml
```

### Deploy the virtual machine
Expand Down Expand Up @@ -166,10 +191,17 @@ directly via cURL:
kubectl patch svc -n kubevirtbmc-system default-fred-virtbmc -p '{"spec": {"type": "LoadBalancer"}}'
```

Store the BMC address in a variable:

```
BMC=$(kubectl get svc -n kubevirtbmc-system default-fred-virtbmc -ojson | jq .status.loadBalancer.ingress[0].ip -r)
```

Then, send a request to its reset endpoint:

```
BMC=$(kubectl get svc -n kubevirtbmc-system default-fred-virtbmc -ojson | jq .status.loadBalancer.ingress[0].ip -r); curl -svX POST \
-H "Content-Type: application/json" -u admin:password \
http://172.19.128.4/redfish/v1/Systems/1/Actions/ComputerSystem.Reset \
curl -svX POST -H "Content-Type: application/json" -u admin:password \
http://${BMC$}/redfish/v1/Systems/1/Actions/ComputerSystem.Reset \
-d '{"ResetType":"ForceRestart"}'
```

Expand All @@ -190,13 +222,10 @@ lacks Redfish endpoints that the [OpenCHAMI discovery tool](https://github.com/O
needs. We can hopefully improve this, but for now it's easiest to manually
enter machine details.

Released versions of the CLI manual discovery lack support for some information
that PCS needs. Pending approval and release of
https://github.com/OpenCHAMI/ochami/pull/47 and https://github.com/OpenCHAMI/ochami/pull/51
this uses yet another custom build:
To add a test machine to SMD, you'll need the OpenCHAMI `ochami` CLI tool:

```
go install github.com/rainest/ochami@v0.6.0-beta-pcs
go install github.com/github.com/OpenCHAMI/ochami@v0.7.0
```

Somewhat amusingly, PCS doesn't actually make use of almost any of the
Expand Down Expand Up @@ -327,9 +356,7 @@ the machine is more visceral proof:
ochami --config /tmp/ochami-demo.yaml pcs transition monitor f8579afb-2843-417c-9bff-65135c0765a3
```

However, something's broken re API compatibility and while `transition start`
works, `transition show` and `transition list` have started 404ing. Pending
investigation of that, you can check the VM to see that it's restarting:
You can also check the virtual machine status directly:

```
$ kubectl get virtualmachine
Expand Down
4 changes: 2 additions & 2 deletions kustomize/services/bss/suite/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ resources:
labels:
- includeSelectors: false
pairs:
app.kubernetes.io/version: v1.32.1
app.kubernetes.io/version: v1.32
app.kubernetes.io/component: boot-script-service

images:
- name: bss-stub
newName: ghcr.io/openchami/bss
newTag: v1.32.1
newTag: 1.32
4 changes: 2 additions & 2 deletions kustomize/services/cloud-init/suite/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ resources:
labels:
- includeSelectors: false
pairs:
app.kubernetes.io/version: v1.3.0
app.kubernetes.io/version: 1.4
app.kubernetes.io/component: cloud-init

images:
- name: cloud-init-stub
newName: ghcr.io/openchami/cloud-init
newTag: v1.4.0
newTag: 1.4
4 changes: 2 additions & 2 deletions kustomize/services/pcs/suite/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ resources:
labels:
- includeSelectors: false
pairs:
app.kubernetes.io/version: v2.7.0
app.kubernetes.io/version: v2.12.0
app.kubernetes.io/component: power-service

images:
- name: power-control-stub
newName: ghcr.io/openchami/pcs
newTag: v2.10.0
newTag: v2.12.0
6 changes: 3 additions & 3 deletions kustomize/services/smd/suite/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ resources:
labels:
- includeSelectors: false
pairs:
app.kubernetes.io/version: v2.18.0
app.kubernetes.io/version: 2.19
app.kubernetes.io/component: state-manager

images:
- name: smd-stub
newName: docker.io/traines/smd
newTag: v2.0.2-debug.4
newName: ghcr.io/openchami/smd
newTag: 2.19
4 changes: 2 additions & 2 deletions kustomize/services/suite/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@ resources:
# nice if you could render either with a valid image using the stock manifests.
images:
- name: power-control-stub
newName: docker.io/traines/power-control
newTag: v1.1.0-debug.14444
newName: ghcr.io/openchami/pcs
newTag: v2.12.0
4 changes: 2 additions & 2 deletions kustomize/test-node/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,5 @@ resources:
# nice if you could render either with a valid image using the stock manifests.
images:
- name: power-control-stub
newName: docker.io/traines/power-control
newTag: v1.1.0-debug.14444
newName: ghcr.io/openchami/pcs
newTag: v2.12.0