Skip to content

Commit e67e043

Browse files
pscheitpscheit
authored
Merge pull request #89 from packagist/kubernetes-migration-guide
Self-Hosted: provide migration guide to help move from native to kubernetes
2 parents e5356ca + 8361d23 commit e67e043

File tree

1 file changed

+174
-0
lines changed

1 file changed

+174
-0
lines changed

‎docs/self-hosted/kubernetes-migration-guide.md‎

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
# Migrating to Private Packagist Self-Hosted Kubernetes
2+
##
3+
4+
## Requirements
5+
6+
To start the migration process, your Private Packagist Self-Hosted Replicated Native instance needs to run version 1.12.5 or newer.
7+
To reduce the disruption to a minimum, we recommend that you set up your Private Packagist Self-Hosted Kubernetes
8+
instance on a different machine before you start the migration process.
9+
10+
Please note that it is possible to install Private Packagist Self-Hosted Kubernetes on the same machine as your current
11+
Private Packagist Self-Hosted Installation. You can do so after you follow the instructions below to back up your existing data
12+
and before you start restoring the application data. However, the Private Packagist Self-Hosted installation process will uninstall
13+
certain components used by your current Private Packagist Self-Hosted installation.
14+
15+
## Backing up the data
16+
17+
Private Packagist Self-Hosted stores data in three different ways. A PostgreSQL database, a Redis database used as storage
18+
for installation and usage statistics, cache, and job queue, and a file/blob storage for uploaded artifacts and dist
19+
files used during composer install.
20+
21+
To avoid any data inconsistency and random errors during application usage all three of them need to be backed up and restored at the same time.
22+
23+
Please note that you won't be able to restore Private Packagist Self-Hosted Kubernetes from snapshots created by Private Packagist Self-Hosted Replicated Native.
24+
25+
### Stop the Private Packagist application
26+
27+
```
28+
docker stop packagist-worker packagist-web packagist-repo
29+
```
30+
31+
### Backup the dist and artifact files
32+
33+
At the end of this step, you should have a `packagist_storage.tar.gz` file in your working directory.
34+
35+
Please note that depending on the number of packages and versions used in your projects and the size of your dependencies
36+
it can take several minutes for this command to finish.
37+
38+
```
39+
docker exec -it packagist-ui /srv/manager/bin/console packagist:self-hosted:migrate-storage export
40+
docker cp packagist-ui:/tmp/storage.tar.gz packagist_storage.tar.gz
41+
docker exec -it packagist-ui rm /tmp/storage.tar.gz
42+
docker stop packagist-ui
43+
```
44+
45+
### Backup the PostgreSQL database
46+
47+
At the end of this step, you should have a `packagist_db.sql` file in your working directory.
48+
49+
```
50+
docker exec -it packagist-postgres pg_dump --clean --if-exists -U postgres -d packagist_db > packagist_db.sql
51+
docker stop packagist-postgres
52+
```
53+
54+
### Backup the Redis database
55+
56+
At the end of this step, you should have a `packagist_redis.rdb` file in your working directory.
57+
58+
```
59+
docker exec -it packagist-redis /usr/local/bin/redis-backup.sh
60+
docker cp packagist-redis:/data/dump.rdb packagist_redis.rdb
61+
docker stop packagist-redis
62+
```
63+
64+
### Start Private Packagist again (optional)
65+
66+
Now that you have exported all the data, you may restart the Private Packagist application. This will allow your developers
67+
to continue with their work while you import the data in the Private Packagist Self-Hosted Kubernetes setup.
68+
69+
```
70+
docker start packagist-postgres packagist-redis packagist-ui packagist-repo packagist-web packagist-worker
71+
```
72+
73+
Please note that any changes made in Private Packagist will now be lost and will have to be reapplied once everyone switches to Private Packagist Self-Hosted Kubernetes.
74+
75+
## Installing Private Packagist Self-Hosted Kubernetes
76+
If you haven't done so already, now is the time to follow the [installation guide](./kubernetes.md) to install Private Packagist Self-Hosted Kubernetes.
77+
78+
## Import the application data in Private Packagist Self-Hosted Kubernetes
79+
80+
All commands assume that you are logged in to the server where Private Packagist Self-Hosted Kubernetes is installed
81+
and your working directory contains the three files generated by the previous steps.
82+
83+
If you installed Private Packagist Kubernetes in an existing cluster and are not using the default namespace,
84+
make sure to run the commands in the correct namespace or switch the namespace before running any of the command.
85+
86+
```
87+
kubectl config set-context --current --namespace=<namespace-name>
88+
```
89+
90+
To avoid any data inconsistency, we highly recommend going through the steps below in order and only start the application
91+
once all the data has been restored.
92+
93+
### Stop the Private Packagist application
94+
95+
```
96+
kubectl scale deployment ui repo worker --replicas 0
97+
```
98+
99+
To verify that all ui, repo, and worker pods have been run execute the command below and verify that the pods don't appear in the list anymore.
100+
```
101+
kubectl get pods
102+
```
103+
104+
### Import the PostgreSQL database
105+
106+
Follow the instructions below to restore the PostgreSQL database in the cluster.
107+
If you are using your own PostgreSQL instance, use the `packagist_db.sql` file directly to import the data into your instance.
108+
109+
Please note that the backup will overwrite any existing data in the database and any changes you made during the setup will be lost.
110+
111+
After each scale command, use `kubectl get pods` again to verify that the postgres pod is in the correct running state.
112+
113+
```
114+
kubectl scale statefulset postgres --replicas 0
115+
kubectl delete pvc postgres-data-postgres-0
116+
kubectl scale statefulset postgres --replicas 1
117+
cat packagist_db.sql | kubectl exec -it statefulsets/postgres -- psql -U postgres -d packagist_db
118+
```
119+
120+
### Import the Redis database
121+
122+
Follow the instructions below to restore the Redis database for Private Packagist Self-Hosted without an existing Kubernetes cluster.
123+
If you are using your own Redis instance, use the `packagist_redis.rdb` file directly to [restore Redis from an RDB file](https://redis.io/learn/guides/import#restore-an-rdb-file) or follow the instructions of your hosted service provider.
124+
If you are installing Private Packagist Self-Hosted in an existing cluster and aren't planning on using your own Redis then please contact us.
125+
126+
After each scale command, use `kubectl get pods` again to verify that the redis pod is in the correct running state.
127+
128+
```
129+
export PV_NAME=$(kubectl get pvc redis-data-redis-0 -ojsonpath='{.spec.volumeName}')
130+
kubectl scale statefulset -n default redis --replicas 0
131+
rm /var/openebs/local/$PV_NAME/appendonly.aof
132+
cp packagist_redis.rdb /var/openebs/local/$PV_NAME/dump.rdb
133+
kubectl scale statefulset redis --replicas 1
134+
```
135+
136+
### Import the dist and artifact files
137+
138+
Importing the dist and artifact files requires the UI pods to be back online. Start them with the command below. This will take a few seconds.
139+
You can use the second command to verify once 2/2 pods are running.
140+
```
141+
kubectl scale statefulset ui --replicas 1
142+
kubectl get pods -w
143+
```
144+
145+
Please note that depending on the size of the `packagist_storage.tar.gz` file and the type of blob storage that you have configured
146+
it can take several minutes for this command to finish.
147+
148+
```
149+
export UI_POD=$(kubectl get pods --no-headers -o custom-columns=":metadata.name"|grep ui-)
150+
kubectl cp packagist_storage.tar.gz $UI_POD:/tmp/packagist_storage.tar.gz -c ui
151+
kubectl exec $UI_POD -c ui -- /bin/sh -c "/srv/manager/bin/console packagist:self-hosted:migrate-storage import /tmp/packagist_storage.tar.gz && rm /tmp/packagist_storage.tar.gz"
152+
```
153+
154+
### Start the Private Packagist application
155+
156+
Once your data has been restored. Start the application:
157+
158+
```
159+
kubectl scale deployment ui --replicas 1
160+
```
161+
162+
This can take a few minutes. You can run the command below to see when all pods are back up and running.
163+
164+
```
165+
kubectl get pods -w
166+
```
167+
168+
Once all the pods are back up and running, log in to Private Packagist and verify that your organization is shown in the UI.
169+
Ideally, run a `composer update` command in one of your projects to assert that the Composer repository responds as expected.
170+
171+
### Adjust domain names
172+
173+
In case you initially set up Private Packagist Self-Hosted Kubernetes with a different domain name, you can now edit the
174+
domain names in the admin panel. If necessary, adjust your DNS entries, and shut down the old Private Packagist Self-Hosted instance.

0 commit comments

Comments
 (0)