Checkout the Madek master-project https://github.com/Madek/madek including
submodules (recursively). See either Initial Checkout or Upgrading below.
Descend into the deploy directory and continue with configuration of the
inventory.
git clone --recursive https://github.com/Madek/madek.git
The version which is going to be deployed is exactly the one that is checked out in the master-project. So the master-project must be updated to upgrade you installation of Madek. From within the master-project:
git pull
git submodule update --init --recursive
It is possible to run into troubles if the submodule structure has changed. The following command might help. Backup your changes beforehand!
git reset --hard origin/master && git clean --force -fd
git submodule foreach --recursive 'git reset --hard HEAD && git clean --force -fd’
You can as well start from scratch.
See the ./inventories directory for examples.
bin/depoy -i $MY_INVENTORY_PATH
bin/ansible-playbook -i my_hosts play_create-admin.yml -e 'admin_password=YOUR-SECRETE-PASSWORD'
This will create (or update) a user with the login admin and the given
password.
The setup generates a default madek_master_secret based on various properties
of the target host and your application. These properties might be guessable or
not, depending on the setup of your host. It is highly recommended to override
the generated secret as shown in the invocations above.
Alternatives to the command line arguments are variables defined in the Ansible Inventory. This is convenient in particular in conjunction with the Ansible Vault.
Caveat: the Ansible rules of precedence should be understood when proceeding in this manner. At any rate, invoking
bin/ansible-playbook -i my_hosts play_info.yml --vault-password-file inventories/zhdk/vault-pass
(possibly with augmented arguments) will show the actual used values and is recommended to be consulted for confirmation.
Example:
bin/ansible-playbook -i inventories/zhdk/staging-v3-pdata play_dump-and-fetch-data.yml
This will dump and fetch data as well as structure_and_data into
tmp/{{ansible_host_name}}/tmp/ . Override dump_items to fetch only either
of them.
From the datalayer directory (of the webapp) a restore would look like the following example:
rake db:pg:truncate_tables db:pg:data:restore FILE=../../../deploy/tmp/staging-v3-pdata/tmp/madek_data.pgbin
See the examples.
The reverse-proxy role can restrict access by client country via Apache's
mod_maxminddb against the DB-IP Lite Country database (CC BY 4.0, no
account / API key required, refreshed monthly). This is intended as an
emergency lever during abusive traffic spikes that have repeatedly
approached DDoS levels — it keeps the service available for the primary
user base while shedding traffic from regions where load is misbehaving.
Caveats — please read before enabling. Country-level IP mapping is only ~99% accurate. Legitimate users on VPNs, Tor, mobile CGNAT or foreign cloud egress will be misclassified and locked out. Unknown / unmapped IPs are denied as well. This is not a security boundary; do not enable for normal operation. Loopback, RFC1918 and IPv6 ULA addresses are always allowed so local health checks and internal probes keep working.
mod_maxminddb is not packaged for Debian, so the reverse-proxy-pre-setup
role builds it once from the pinned upstream release tarball
(mod_maxminddb_version, mod_maxminddb_sha256 in that role's defaults).
The mmdb file is fetched and refreshed by a systemd timer
(madek-update-geoip-db.timer).
Configuration (defaults from roles/reverse-proxy-madek/defaults/main.yml):
| Variable | Default | Purpose |
|---|---|---|
geo_whitelist_enabled |
false |
Master switch. When false the mmdb is still installed and MM_COUNTRY_CODE env vars are set (useful for logging) but no requests are blocked. |
geo_whitelist_countries |
"{{ geo_whitelist_dach }}" |
List of ISO 3166-1 alpha-2 country codes that are allowed through. |
geo_whitelist_dach |
[DE, AT, CH, LI] |
Predefined group: D-A-CH + Liechtenstein. |
geo_whitelist_eu27 |
EU member states | Predefined group. |
geo_whitelist_eu_eea_uk_ch |
EU + EEA + UK + CH | Predefined group ("broad western Europe"). |
geo_whitelist_db_path |
/var/lib/GeoIP/dbip-country-lite.mmdb |
Where the mmdb is installed. |
geo_whitelist_db_refresh_on |
Mon *-*-* 04:17:00 |
systemd OnCalendar for the refresh timer (DB-IP publishes monthly). |
geo_whitelist_db_refresh_randomized_delay_seconds |
7200 |
Random jitter for the timer. |
Example — enable on a single host (e.g. inventory host_vars/test.yml):
geo_whitelist_enabled: true
geo_whitelist_countries: "{{ geo_whitelist_dach }}"
To apply, deploy the reverse-proxy role:
bin/ansible-playbook -i $MY_INVENTORY play_setup-and-deploy.yml -t reverse-proxy
To turn it off, set geo_whitelist_enabled: false and re-run the same
command. Blocked requests receive HTTP 403.
Example
rsync -e ssh -Pav root@madek.*************.**:/var/local/madek-file-storage/* /opt/madekdata/
This is handled via https://github.com/elasticdog/transcrypt.
The transcrypt bash script is stored within this repo: ./bin/transcrypt.
To unlock the encrypted files run ./bin/unlock.
Unlocking is rarely necessary. Some cases include
- deployments to a ZHdK Server,
- changing ZHdK Secrets.
Run ./bin/transcrypt --flush-credentials to bring the files back to their
encrypted state.
See https://github.com/elasticdog/transcrypt?tab=readme-ov-file#rekeying for key rollover.
We store the symmetric key via gpg in the project. Run the following to encrypt a new key:
gpg -a --encrypt \
-r Thomas.Schank@zhdk.ch \
-r ci-exec-user@ci-executor-madek-2024 \
-r jan.pistor@zhdk.ch \
-r manuel.radl@zhdk.ch \
-r matus.kmit@zhdk.ch \
-r urs.vanbinsbergen@zhdk.ch \
.transcrypt_key.txt