ROSETTA-QTUM IS CONSIDERED ALPHA SOFTWARE. USE AT YOUR OWN RISK! COINBASE ASSUMES NO RESPONSIBILITY NOR LIABILITY IF THERE IS A BUG IN THIS IMPLEMENTATION.
rosetta-qtum provides a reference implementation of the Rosetta API for
Qtum in Golang. If you haven't heard of the Rosetta API, you can find more
information here.
- Rosetta API implementation (both Data API and Construction API)
- UTXO cache for all accounts (accessible using the Rosetta
/account/balanceAPI) - Stateless, offline, curve-based transaction construction from any SegWit-Bech32 Address
As specified in the Rosetta API Principles,
all Rosetta implementations must be deployable via Docker and support running via either an
online or offline mode.
YOU MUST INSTALL DOCKER FOR THE FOLLOWING INSTRUCTIONS TO WORK. YOU CAN DOWNLOAD DOCKER HERE.
Clone this repository and run the following command to create a Docker image called rosetta-qtum:latest.
make build-local
Running the following commands will start a Docker container in
detached mode with
a data directory at <working directory>/qtum-data and the Rosetta API accessible
at port 8080.
docker run -d --rm --ulimit "nofile=100000:100000" -v "$(pwd)/qtum-data:/data" -e "MODE=ONLINE" -e "NETWORK=MAINNET" -e "PORT=8080" -p 8080:8080 -p 8333:8333 rosetta-qtum:latest
If you cloned the repository, you can run make run-mainnet-online.
docker run -d --rm -e "MODE=OFFLINE" -e "NETWORK=MAINNET" -e "PORT=8081" -p 8081:8081 rosetta-qtum:latest
If you cloned the repository, you can run make run-mainnet-offline.
docker run -d --rm --ulimit "nofile=100000:100000" -v "$(pwd)/qtum-data:/data" -e "MODE=ONLINE" -e "NETWORK=TESTNET" -e "PORT=8080" -p 8080:8080 -p 18333:18333 rosetta-qtum:latest
If you cloned the repository, you can run make run-testnet-online.
docker run -d --rm -e "MODE=OFFLINE" -e "NETWORK=TESTNET" -e "PORT=8081" -p 8081:8081 rosetta-qtum:latest
If you cloned the repository, you can run make run-testnet-offline.
rosetta-qtum has been tested on an GCP T2d-standard-16 instance.
This instance type has 16 vCPU with 64 gb of RAM, 10gb of swap and 100gb of ssd storage.
To increase the load rosetta-qtum can handle, it is recommended to tune your OS
settings to allow for more connections. On a linux-based OS, you can run the following
commands (source):
sysctl -w net.ipv4.tcp_tw_reuse=1
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
sysctl -w net.ipv4.tcp_max_syn_backlog=10000
sysctl -w net.core.somaxconn=10000
sysctl -p (when done)
We have not tested rosetta-qtum with net.ipv4.tcp_tw_recycle and do not recommend
enabling it.
You should also modify your open file settings to 100000. This can be done on a linux-based OS
with the command: ulimit -n 100000.
rosetta-qtum uses memory-mapped files to
persist data in the indexer. As a result, you must run rosetta-qtum on a 64-bit
architecture (the virtual address space easily exceeds 100s of GBs).
If you receive a kernel OOM, you may need to increase the allocated size of swap space on your OS. There is a great tutorial for how to do this on Linux here.
rosetta-qtum uses the syncer, storage, parser, and server package
from rosetta-sdk-go instead
of a new Qtum-specific implementation of packages of similar functionality. Below
you can find a high-level overview of how everything fits together:
+------------------------------------------------------------------+
| |
| +--------------------------------------+ |
| | | |
| | indexer | |
| | | |
| | +--------+ | |
+-------------------+ pruner <----------+ | |
| | +--------+ | | |
+-----v----+ | | | |
| qtumd | | +------+--------+ | |
+-----+----+ | +--------> block_storage <----+ | |
| | | +---------------+ | | |
| | +---+----+ | | |
+-------------------> syncer | | | |
| +---+----+ | | |
| | +--------------+ | | |
| +--------> coin_storage | | | |
| +------^-------+ | | |
| | | | |
+--------------------------------------+ |
| | |
+-------------------------------------------------------------------------------------------+ |
| | | | |
| +------------------------------------------------------------+ | | |
| | | | |
| | +---------------------+-----------------------+------+ | |
| | | | | | |
| +-------+---------+ +-------+---------+ +-------+-------+ +-----------+----------+ | |
| | account_service | | network_service | | block_service | | construction_service +--------+
| +-----------------+ +-----------------+ +---------------+ +----------------------+ |
| |
| server |
| |
+-------------------------------------------------------------------------------------------+
- Automatically prune qtumd while indexing blocks
- Reduce sync time with concurrent block indexing
- Use Zstandard compression to reduce the size of data stored on disk without needing to write a manual byte-level encoding
To speed up indexing, rosetta-qtum uses concurrent block processing
with a "wait free" design (using channels instead of sleeps to signal
which threads are unblocked). This allows rosetta-qtum to fetch
multiple inputs from disk while it waits for inputs that appeared
in recently processed blocks to save to disk.
+----------+
| qtumd |
+-----+----+
|
|
+---------+ fetch block data / unpopulated txs |
| block 1 <------------------------------------+
+---------+ |
+--> tx 1 | |
| +---------+ |
| | tx 2 | |
| +----+----+ |
| | |
| | +---------+ |
| | | block 2 <-------------------+
| | +---------+ |
| +-----------> tx 3 +--+ |
| +---------+ | |
+-------------------> tx 4 | | |
| +---------+ | |
| | |
| retrieve previously synced | +---------+ |
| inputs needed for future | | block 3 <--+
| blocks while waiting for | +---------+
| populated blocks to save to +---> tx 5 |
| disk +---------+
+------------------------------------> tx 6 |
| +---------+
|
|
+------+--------+
| coin_storage |
+---------------+
To validate rosetta-qtum, install rosetta-cli
and run one of the following commands:
rosetta-cli check:data --configuration-file rosetta-cli-conf/testnet/config.json- This command validates that the Data API information in thetestnetnetwork is correct. It also ensures that the implementation does not miss any balance-changing operations.rosetta-cli check:construction --configuration-file rosetta-cli-conf/testnet/config.json- This command validates the blockchain’s construction, signing, and broadcasting.rosetta-cli check:data --configuration-file rosetta-cli-conf/mainnet/config.json- This command validates that the Data API information in themainnetnetwork is correct. It also ensures that the implementation does not miss any balance-changing operations.
Interested in helping fix issues in this repository? You can find to-dos in the Issues section with the help wanted tag. Be sure to reach out on our community before you tackle anything on this list.
make depsto install dependenciesmake testto run testsmake lintto lint the source codemake salusto check for security concernsmake build-localto build a Docker image from the local contextmake coverage-localto generate a coverage report
This project is available open source under the terms of the Apache 2.0 License.
© 2021 Coinbase