|
| 1 | +# Migrating to Private Packagist Self-Hosted Kubernetes |
| 2 | +## |
| 3 | + |
| 4 | +## Requirements |
| 5 | + |
| 6 | +To start the migration process, your Private Packagist Self-Hosted Replicated Native instance needs to run version 1.12.5 or newer. |
| 7 | +To reduce the disruption to a minimum, we recommend that you set up your Private Packagist Self-Hosted Kubernetes |
| 8 | +instance on a different machine before you start the migration process. |
| 9 | + |
| 10 | +Please note that it is possible to install Private Packagist Self-Hosted Kubernetes on the same machine as your current |
| 11 | +Private Packagist Self-Hosted Installation. You can do so after you follow the instructions below to back up your existing data |
| 12 | +and before you start restoring the application data. However, the Private Packagist Self-Hosted installation process will uninstall |
| 13 | +certain components used by your current Private Packagist Self-Hosted installation. |
| 14 | + |
| 15 | +## Backing up the data |
| 16 | + |
| 17 | +Private Packagist Self-Hosted uses three different storages. A PostgreSQL database used as primary storage, |
| 18 | +a Redis database used as secondary storage, cache, and job queue, and a file/blob storage for uploaded artifacts and dist |
| 19 | +files used during composer install. |
| 20 | + |
| 21 | +To avoid any data inconsistency and random errors during application usage all three of them need to be backed up and restored at the same time. |
| 22 | + |
| 23 | +Please note that you won't be able to restore Private Packagist Self-Hosted Kubernetes from snapshots created via Private Packagist Self-Hosted Replicated Native. |
| 24 | + |
| 25 | +### Stop the Private Packagist application |
| 26 | + |
| 27 | +``` |
| 28 | +docker stop packagist-worker packagist-web packagist-repo |
| 29 | +``` |
| 30 | + |
| 31 | +### Backup the dist and artifact files |
| 32 | + |
| 33 | +At the end of this step, you should have a `packagist_storage.tar.gz` file in your working directory. |
| 34 | + |
| 35 | +Please note that depending on the number of packages and versions used in your projects and the size of your dependencies |
| 36 | +it can take several minutes for this command to finish. |
| 37 | + |
| 38 | +``` |
| 39 | +docker exec -it packagist-ui /srv/manager/bin/console packagist:self-hosted:migrate-storage export |
| 40 | +docker cp packagist-ui:/tmp/storage.tar.gz packagist_storage.tar.gz |
| 41 | +docker exec -it packagist-ui rm /tmp/storage.tar.gz |
| 42 | +docker stop packagist-ui |
| 43 | +``` |
| 44 | + |
| 45 | +### Backup the PostgreSQL database |
| 46 | + |
| 47 | +At the end of this step, you should have a `packagist_db.sql` file in your working directory. |
| 48 | + |
| 49 | +``` |
| 50 | +docker exec -it packagist-postgres pg_dump --clean --if-exists -U postgres -d packagist_db > packagist_db.sql |
| 51 | +docker stop packagist-postgres |
| 52 | +``` |
| 53 | + |
| 54 | +### Backup the Redis database |
| 55 | + |
| 56 | +At the end of this step, you should have a `packagist_redis.rdb` file in your working directory. |
| 57 | + |
| 58 | +``` |
| 59 | +docker exec -it packagist-redis /usr/local/bin/redis-backup.sh |
| 60 | +docker cp packagist-redis:/data/dump.rdb packagist_redis.rdb |
| 61 | +docker stop packagist-redis |
| 62 | +``` |
| 63 | + |
| 64 | +### Start Private Packagist again (optional) |
| 65 | + |
| 66 | +Now that you have exported all the data, you may restart the Private Packagist application. This will allow your developers |
| 67 | +to continue with their work while you import the data in the Private Packagist Self-Hosted Kubernetes setup. |
| 68 | + |
| 69 | +``` |
| 70 | +docker start packagist-postgres packagist-redis packagist-ui packagist-repo packagist-web packagist-worker |
| 71 | +``` |
| 72 | + |
| 73 | +Please note that any changes made in Private Packagist will now be lost and will have to be reapplied once everyone switches to Private Packagist Self-Hosted Kubernetes. |
| 74 | + |
| 75 | +## Installing Private Packagist Self-Hosted Kubernetes |
| 76 | +If you haven't done so already, now is the time to follow the [installation guide](./kubernetes.md) to install Private Packagist Self-Hosted Kubernetes. |
| 77 | + |
| 78 | +## Restore application data in Kubernetes |
| 79 | + |
| 80 | +All commands assume that you are logged in to the server where Private Packagist Self-Hosted Kubernetes is installed, |
| 81 | +your working directory contains the three files generated by the previous steps, and that |
| 82 | +Private Packagist Self-Hosted Kubernetes is installed in the Kubernetes namespace "default". |
| 83 | + |
| 84 | +To avoid any data inconsistency, we highly recommend going through the steps below in order and only restart the application |
| 85 | +once all the data has been restored. |
| 86 | + |
| 87 | +### Stop the Private Packagist application |
| 88 | + |
| 89 | +``` |
| 90 | +kubectl scale deployment ui repo worker --replicas 0 -n default |
| 91 | +``` |
| 92 | + |
| 93 | +### Restore the PostgreSQL database |
| 94 | + |
| 95 | +Follow the instructions below to restore the PostgreSQL database in the cluster. |
| 96 | +If you are using your own PostgreSQL instance, use the `packagist_db.sql` file directly to import the data into your instance. |
| 97 | + |
| 98 | +Please note that the backup will overwrite any existing data in the database and any changes you made during setup will be lost. |
| 99 | + |
| 100 | +``` |
| 101 | +kubectl scale statefulset postgres --replicas 0 -n default |
| 102 | +kubectl delete pvc postgres-data-postgres-0 -n default |
| 103 | +kubectl scale statefulset postgres --replicas 1 -n default |
| 104 | +cat packagist_db.sql | kubectl exec -it -n default statefulsets/postgres -- psql -U postgres -d packagist_db |
| 105 | +``` |
| 106 | + |
| 107 | +### Restore the Redis database |
| 108 | + |
| 109 | +Follow the instructions below to restore the Redis database in the cluster. |
| 110 | +If you are using your own Redis instance, use the `packagist_redis.rdb` file directly to [restore Redis from an RDB file](https://redis.io/learn/guides/import#restore-an-rdb-file) or follow the instructions of your hosted service provider. |
| 111 | + |
| 112 | +Follow these steps if you are using Private Packagist Self-Hosted with an embeded Kubernetes cluster |
| 113 | +``` |
| 114 | +export PV_NAME=$(kubectl get pvc -n default redis-data-redis-0 -ojsonpath='{.spec.volumeName}') |
| 115 | +kubectl scale statefulset -n default redis --replicas 0 |
| 116 | +rm /var/openebs/local/$PV_NAME/appendonly.aof |
| 117 | +cp packagist_redis.rdb /var/openebs/local/$PV_NAME/dump.rdb |
| 118 | +kubectl scale statefulset -n default redis --replicas 1 |
| 119 | +``` |
| 120 | + |
| 121 | +### Restore the dist and artifact files |
| 122 | + |
| 123 | +Please note that depending on the size of the `packagist_storage.tar.gz` file and the type of blob storage that you have configured |
| 124 | +it can take several minutes for this command to finish. |
| 125 | + |
| 126 | +``` |
| 127 | +kubectl scale statefulset -n default ui --replicas 1 |
| 128 | +kubectl cp -n default packagist_storage.tar.gz ui:/tmp/packagist_storage.tar.gz |
| 129 | +kubectl exec -n default ui /srv/manager/bin/console packagist:self-hosted:migrate-storage import /tmp/packagist_storage.tar.gz && rm /tmp/packagist_storage.tar.gz |
| 130 | +``` |
| 131 | + |
| 132 | +### Start the Private Packagist application |
| 133 | + |
| 134 | +Once your data has been restored. Restart the application, log in to Private Packagist, and verify that your organization |
| 135 | +and packages are there again. |
| 136 | + |
| 137 | +``` |
| 138 | +kubectl scale deployment ui --replicas 1 -n default |
| 139 | +``` |
| 140 | + |
| 141 | +### Adjust domain names |
| 142 | + |
| 143 | +In case you initially set up Private Packagist Self-Hosted Kubernetes with a different domain name, you can now edit the |
| 144 | +domain names in the Admin panel, if necessary adjust your DNS entries, and shut down the old Private Packagist Self-Hosted instance. |
0 commit comments