diff --git a/docs.json b/docs.json index e84345fd8..fe3d8080a 100644 --- a/docs.json +++ b/docs.json @@ -305,6 +305,7 @@ "pages": [ "ecosystem/nodes/cpp/setup-mytonctrl", "ecosystem/nodes/cpp/run-validator", + "ecosystem/nodes/cpp/run-archive-liteserver", "ecosystem/nodes/cpp/integrating-with-prometheus", "ecosystem/nodes/cpp/setup-mylocalton", { diff --git a/ecosystem/nodes/cpp/run-archive-liteserver.mdx b/ecosystem/nodes/cpp/run-archive-liteserver.mdx new file mode 100644 index 000000000..c52c30fbd --- /dev/null +++ b/ecosystem/nodes/cpp/run-archive-liteserver.mdx @@ -0,0 +1,463 @@ +--- +title: "Run an archive liteserver" +description: "Run an archive liteserver node with MyTonCtrl" +--- + +import { Aside } from '/snippets/aside.jsx'; + +## Overview + +An **archive liteserver node** is a type of full node that stores the entire block history of the TON blockchain. If you are building a blockchain explorer, an indexer, or any application requiring access to historical data, running an archive liteserver node is the recommended approach. + +This guide explains how to set up an archive liteserver using MyTonCtrl and ZFS for efficient storage management. + +## Step 1: Prepare environment + +### 1.1 Minimal hardware requirements + +Running an archive liteserver requires significant storage and network resources. + +- 16-core CPU +- 128 GB RAM +- At least 16TB of SSD storage, 20TB if you will not use ZFS with compression (NVMe recommended), or Provisioned 64k+ IOPS storage +- 1 Gbit/s symmetric connectivity (both inbound and outbound), ~16 TB/month at peak load +- Fixed (static) public IP address + + + +### 1.2 OS and system requirements + +- Ubuntu 22.04 LTS, Ubuntu 24.04 LTS, or Debian 11 +- Python 3.10 or higher +- Open Files Limit must be set above 4,000,000 + +### 1.3 Subscribe to official channels + +Subscribe and follow the announcements provided for liteservers in the following Telegram channels: + +| Channel | Network | +| -------------------------------------------- | ----------- | +| [@tonstatus](https://t.me/tonstatus) | TON Mainnet | +| [@testnetstatus](https://t.me/testnetstatus) | TON Testnet | + +### 1.4 Install ZFS and prepare volume + +Archive nodes benefit significantly from ZFS due to its native compression and snapshot capabilities. + +#### 1.4.1 Install ZFS + +```bash +sudo apt update +sudo apt install -y zfsutils-linux +``` + +#### 1.4.2 Verify physical block size + +For NVMe drives, it is critical to align the ZFS pool's sector size with the drive's physical block size to avoid significant performance degradation. Most modern NVMe drives use 4K blocks. Verify your drive's physical block size: + +```bash +# Replace nvme0n1 with your device name +cat /sys/block/nvme0n1/queue/physical_block_size +``` + +If the result is `4096`, you must use the `-o ashift=12` parameter during pool creation. + +#### 1.4.3 Create a storage pool + +Create a ZFS pool named `data`. Use `-o ashift=12` for 4K blocks (standard for most NVMe drives): + + + ```bash Single drive + # Replace with your device identifier (e.g., /dev/nvme1n1) + sudo zpool create -o ashift=12 data + ``` + + ```bash Multiple drives (Stripe) + # Combine multiple disks to increase capacity and performance + sudo zpool create -o ashift=12 data + ``` + + + + +### 1.4.4 Enable compression + +We recommend enabling `lz4` compression to save disk space with minimal CPU overhead: + +```bash +sudo zfs set compression=lz4 data +``` + +#### 1.4.5 Create dataset and mountpoint + +Create the dataset for TON data and set the mount point to `/var/ton-work`: + +```bash +sudo zfs create data/ton-work +sudo zfs set mountpoint=/var/ton-work data/ton-work +``` + +### 1.5 Prepare the operator account + +If you need a dedicated operator user, create it and switch to it before installing MyTonCtrl: + +- Create a non-root user + +```bash +# Create a non-root operator user +sudo adduser +sudo usermod -aG sudo +``` + +- Switch to it by reconnecting to the server via `ssh` + +```bash +# Reconnect as the new user. If you changed the SSH port in Step 1.6, use the -p flag. +exit +ssh @ [-p ] +``` + +### 1.6 Harden server security + + + +#### SSH hardening + +Apply the following SSH configuration changes in `/etc/ssh/sshd_config`: + +- Enable key-based authentication and disable password login: + +```text +PasswordAuthentication no +PubkeyAuthentication yes +``` + +- Disable root login: + +```text +PermitRootLogin no +``` + +- Change the default SSH port: + +```text +Port +``` + +`` — a non-default port number (for example, `2222`). + +- Restrict SSH access to specific IP addresses using the `Match Address` directive: + +```text +Match Address + AllowUsers +``` + +Restart the SSH service after changes: + +```bash +sudo systemctl restart sshd +``` + +#### Firewall configuration + +Enable the firewall and allow only the SSH port. The node UDP port and liteserver port are added after installation in [step 2.1.3](#2-1-3-open-the-node-udp-port-and-the-liteserver-port). + +```bash +sudo ufw allow +sudo ufw enable +sudo ufw status +``` + +#### Additional security measures + +- Use a unique, strong password for the root user. + +- Set a GRUB bootloader password to prevent unauthorized boot modifications. + +- Enable Fail2ban for SSH brute-force protection: + + ```bash + sudo apt install -y fail2ban + sudo systemctl enable fail2ban + sudo systemctl start fail2ban + ``` + +- Configure two-factor authentication for SSH using `libpam-google-authenticator` or a similar PAM module. + +## Step 2: Archive liteserver installation + +The installation process consists of three lengthy stages (in total, this can take up to a week): + +- Downloading historical blocks from TON Storage and installing the archive liteserver +- Importing downloaded data into the archive liteserver database +- Final synchronization of the archive liteserver + +### 2.1 Downloading historical blocks from TON Storage and installing the archive liteserver + +This process can take from one to several days depending on your internet connection speed. + +### 2.1.1 Install prerequisites and download installer (MyTonCtrl) + +```bash + sudo apt update + sudo apt install -y curl wget git ca-certificates python3-pip + wget https://raw.githubusercontent.com/ton-blockchain/mytonctrl/master/scripts/install.sh +``` + +### 2.1.2 Run archive liteserver installation + +Run the installer from the operator account with `sudo` so it can create system users and services: + + + ```bash Mainnet + nohup sudo bash install.sh -m liteserver -n mainnet --archive > mytonctrl_installation.log 2>&1 & + ``` + + ```bash Testnet + nohup sudo bash install.sh -m liteserver -n testnet --archive > mytonctrl_installation.log 2>&1 & + ``` + + +Installation will run in the background. + +You can monitor the progress using the following command: + +```bash +tail -f mytonctrl_installation.log +``` + +During the download process, you can find messages like the following in the log: + +```text +[info] 24.04.2026, 14:26:47.609 (UTC) STARTING DOWNLOADING e88e7e805dfa16dc6e7d3864fcc00159a1ee70edde7b421428783c2164453875 +[info] 24.04.2026, 14:27:07.611 (UTC) DOWNLOADING e88e7e805dfa16dc6e7d3864fcc00159a1ee70edde7b421428783c2164453875 0% (0.0 / 4296.493726 MB), speed: 0.0 MB/s +[info] 24.04.2026, 14:27:07.623 (UTC) DOWNLOADING 7683b6bc2c19e007fc6d363dc233ff9a405c0fee7533c2904c06943f759c155f 3% (130.766577 / 4295.81461 MB), speed: 15.078028 MB/s +[info] 28.04.2026, 11:43:00.687 (UTC) DOWNLOADING fd41a820efc4f459bb1518c45fcc23023b0a95d9446c683705802bfb9d50a0c9 95% (4072.892954 / 4296.3534 MB), speed: 28.962779 MB/s +[info] 28.04.2026, 11:43:20.700 (UTC) DOWNLOADED fd41a820efc4f459bb1518c45fcc23023b0a95d9446c683705802bfb9d50a0c9 +``` + +Upon successful completion of the installation, you will see the following line in the log: + +```text +[5/5] Mytonctrl installation completed +``` + +### 2.2 Importing downloaded data into the archive liteserver database + +This process starts automatically after installation and can take from one to several days depending on your server performance. + +You can monitor the progress using the MyTonCtrl console: + +```bash +mytonctrl +MyTonCtrl> status +``` + +Check the field **Local validator initial sync status**. The value indicates how old the last imported block was and should decrease over time. + +### 2.2.1 Open the node UDP port and the liteserver port + +At this stage, the node UDP port and liteserver port should be opened to make the archive liteserver available for syncing blocks from other nodes. + +Identify the `engine.addr` port and liteserver port from the `config.json` file: + +```bash +sudo grep -A5 '"addrs"' -n /var/ton-work/db/config.json | grep '"port"' | head -1 +sudo grep -A5 '"liteservers"' -n /var/ton-work/db/config.json | grep '"port"' | head -1 +``` + +Update security groups or configure `ufw` on bare-metal hosts: + +```bash +sudo apt install -y ufw +sudo ufw allow +sudo ufw allow +sudo ufw status +``` + +### 2.3 Final synchronization of archive liteserver + +This process starts automatically after the importing process finishes and can take from one to several days depending on your server performance. + +You can monitor the progress using the MyTonCtrl console: + +```bash +mytonctrl +MyTonCtrl> status +``` + +Check the field **Local validator initial sync status**. The value indicates how old the last imported block was. On a fully synchronized node, this value should be less than 20 seconds. + +## Step 3: Maintenance + +### 3.1 Set up alerting + +Set up alerting in MyTonCtrl to get a notification of critical issues with the archive liteserver. For more information, see [MyTonCtrl private alerting bot](/ecosystem/nodes/cpp/mytonctrl/alerting). + +### 3.2 Set up monitoring + +Set up monitoring dashboards for RAM, disk, network, CPU usage, and other metrics. + +For system-level metrics, [integrate Prometheus with `node_exporter` with MyTonCtrl](/ecosystem/nodes/cpp/integrating-with-prometheus). + +It is critical to use the monitoring system to: + +- monitor server stability +- monitor synchronization parameters +- check for memory leaks + +For technical assistance, contact [@mytonctrl\_help\_bot](https://t.me/mytonctrl_help_bot). + +### 3.3 Performing software updates + +Follow the @tonstatus channel, turn on notifications, and be prepared for urgent updates if needed. + +Before performing any updates, it is highly recommended to create a ZFS snapshot of your data. This allows you to quickly roll back in case the update process fails or corrupts the database. + +```bash +sudo zfs snapshot data/ton-work@before-update-$(date +%Y-%m-%d) +``` + +To update your node software and MyTonCtrl to the latest versions, use the MyTonCtrl console: + +```bash +MyTonCtrl> update master +MyTonCtrl> upgrade master +``` + +These commands will check for new versions of the TON node binaries and MyTonCtrl, download them, and apply the updates. + + + +After you have verified that the update was successful and the node is running correctly, it is recommended to delete the snapshot to reclaim disk space. + +### 3.4 ZFS Snapshots + +ZFS allows you to create snapshots for easy rollbacks if data corruption occurs. + +#### Create a snapshot +Snapshots can be created **without** stopping the node: + +```bash +sudo zfs snapshot data/ton-work@backup-date +``` + +#### List snapshots +To see all existing snapshots for the `data/ton-work` dataset: + +```bash +sudo zfs list -t snapshot data/ton-work +``` + +#### Roll back to a snapshot +To restore data from a snapshot, you **must** stop the validator service first: + +```bash +# Stop the service +sudo systemctl stop validator.service + +# Roll back to the snapshot +sudo zfs rollback data/ton-work@backup-date + +# Start the service +sudo systemctl start validator.service +``` + +#### Delete a snapshot +Once a snapshot is no longer needed, you can delete it: + +```bash +sudo zfs destroy data/ton-work@backup-date +``` + +### 3.5 Archiving ZFS snapshots + +Creating a snapshot is instantaneous and happens on the same physical disks where your data is stored. To protect against hardware failure, you should export your snapshots to external storage or a remote server using `zfs send`. + +#### Exporting a snapshot to a file + +Before exporting, estimate the size of the snapshot: + +```bash +sudo zfs send -pc -nv data/ton-work@backup-date +``` + +Example output: +```text +full send of data/ton-work@backup-date estimated size is 4.07T +total estimated size is 4.07T +``` + +To save the snapshot to a file, use the `-c` flag to preserve `lz4` compression (without it, the file will be significantly larger): + +```bash +# Ensure the destination directory exists and has enough free space +sudo zfs send -c data/ton-work@backup-date > /external_storage/backup_ton_work.zfs +``` + + + +#### Transferring a snapshot via SSH + +You can stream a snapshot directly to a remote ZFS-enabled server: + +```bash +sudo zfs send -c data/ton-work@backup-date | ssh user@remote_host "sudo zfs recv remote_data/ton-work" +``` + +## Troubleshooting + +### Monitoring import logs + +To see detailed logs of the block import process, increase the log verbosity from the MyTonCtrl console: + +```bash +mytonctrl +MyTonCtrl> installer set_node_argument --verbosity 3 +``` + +Then follow the log file from a separate terminal: + +```bash +tail -f /var/ton-work/log* +``` + +You should see entries like: + +```text +[ 2][t49][2025-01-01 00:00:00.632][import-db-slice-local.cpp:629][!archiveimport] Imported archive in 2.75s : mc_seqno=761229 shard_seqno=761229 +``` + +Set verbosity back to `1` after checking logs to avoid excessive disk I/O overhead: + +```bash +mytonctrl +MyTonCtrl> installer set_node_argument --verbosity 1 +``` + +### Performance issues + +If you see logs like `Importing archive for masterchain seqno #... from net` accompanied by timeout errors, it likely indicates that your storage performance is insufficient to keep up with the block import process. Ensure your disk meets the IOPS requirements listed in [section 1.1](#step-1-prepare-environment). + +## Support + +For technical assistance, join the official support channel: [@ton_node_help](https://t.me/ton_node_help). + +## See also + +- [TON node types](/ecosystem/nodes/overview) +- [Run a node with MyTonCtrl](/ecosystem/nodes/cpp/setup-mytonctrl)