Add node infrastructure guides and restructure section #1246

brunopgalvao · 2025-11-18T05:46:10Z

Summary

Resolves #1216
Resolves #1217
Resolves #1185

Refactored the "Nodes and Validators" section to accommodate Polkadot Hub, collators, and system parachains with an emphasis on running a Polkadot Hub RPC node. With this, the section has been renamed to "Node Infrastructure" to better accommodate what is contained in this section (Polkadot Hub RPC node, system parachain RPC node, relay chain nodes, collators, validators, etc)

New content to:

setup and run a Polkadot Hub RPC node
setup and run a system parachain RPC node
setup and run a block-producing collator

- Add comprehensive guide for running system parachain RPC nodes - Covers People Chain, Bridge Hub, and Coretime Chain - Provides both Docker and systemd deployment options - Includes snapshot setup, monitoring, and security best practices - Add conclusion sections to RPC and collator guides - polkadot-hub-rpc.md: summarize benefits and next steps - system-parachain-rpc.md: highlight adaptability across system parachains - collator.md: emphasize operator role and responsibilities - Restructure documentation from nodes-and-validators to node-infrastructure

- Resolve conflict in .nav.yml by keeping 'Node Infrastructure' and adding 'Technical Reference' - Integrate latest changes from staging/product-ia

Update polkadot-omni-node from stable2506-4 to v1.20.2 and Rust from 1.86 to 1.91.1 in the system parachain RPC documentation. This ensures users are working with the latest stable releases. Changes: - Docker image: parity/polkadot-omni-node:v1.20.2 - Cargo package: polkadot-omni-node@0.11.0 - Rust toolchain: 1.91.1 All versions have been tested and verified to work correctly.

Resolve merge conflict in .nav.yml by keeping the Node Infrastructure navigation structure from this branch.

- Update Rust toolchain to 1.91.1 (from 1.86) - Update polkadot-omni-node to v0.11.0 for collator (from v0.5.0) - Update Docker image to v1.20.2 for RPC nodes (from stable2506-4) - Update chain-spec-builder to v14.0.0 (from v10.0.0) - Update AI-generated page files to reflect latest changes - Clean up old nodes-and-validators AI page files

dawnkelly09

Not done yet but wanted to surface a couple of questions for @brunopgalvao

dawnkelly09 · 2025-11-21T15:44:43Z

node-infrastructure/run-a-collator.md

+
+## Complete Initial Sync
+
+Your collator must sync both the relay chain and parachain before producing blocks.


I'm confused as to why we are telling them they must do this, but we give no steps/guidance as to how to do it. Is there a guide somewhere else we can at least point them toward?

When they run the collator it will do this automatically.

dawnkelly09 · 2025-11-21T15:46:22Z

node-infrastructure/run-a-collator/collator.md

+
+## Install Dependencies
+
+1. Install Rust using the following commands:


Please clarify whether the user should run both of these Rust installation commands (# Install Rust vs # Install specific Rust version) or pick one or the other.

Yes they should run all commands inside that snippet.

…cker

nhussein11

Left some comment for polkadot-hub-rpc page, but please check that most of them also apply to parachain-rpc.md

node-infrastructure/overview.md

node-infrastructure/run-a-collator.md

node-infrastructure/run-a-collator/collator.md

node-infrastructure/run-a-collator.md

node-infrastructure/run-a-collator/collator.md

node-infrastructure/run-a-collator.md

brunopgalvao · 2025-11-26T05:12:17Z

Images for collator.md, "Register Collator for Selection" > "Registration Process" section.

collatorSelection.invulnerables()

session.setKeys

collatorSelection.registerAsCandidate

nhussein11 · 2025-11-26T12:04:31Z

node-infrastructure/run-a-collator.md

+  ```
+
+  The output will be similar to the following:
+  ```bash


This should be a termynal element

nhussein11 · 2025-11-26T12:06:57Z

node-infrastructure/run-a-node/relay-chain/full-node.md

There is a termynal element that seems broken here

artkaseman

Added some comments.

.ai/pages/node-infrastructure-run-a-collator-collator.md

.ai/pages/node-infrastructure-run-a-node-parachain-rpc.md

.ai/pages/node-infrastructure-run-a-node-polkadot-hub-rpc.md

.ai/pages/node-infrastructure-run-a-collator-collator.md

…nfra/running-a-collator

- Remove 9944/9615 from external firewall ports (localhost only) - Add pruning flags for para and relay chains (--blocks-pruning=256, --state-pruning=256) - Use --database=paritydb and --sync=fast for collators - Disable relay chain RPC with --rpc-port=0 - Add --pool-limit=0 for relay chain on collators - Replace archive-canonical with pruned relay chain config - Add chown root:root for binary ownership

brunopgalvao · 2025-12-04T05:49:26Z

Thanks @artkaseman - valuable feedback. Incorporated all your suggestions.

node-infrastructure/run-a-node/polkadot-hub-rpc.md

eshaben · 2025-12-08T16:25:59Z

node-infrastructure/run-a-node/polkadot-hub-rpc.md

+                    rm files.txt
+                    ```
+
+                    **Relay chain pruned snapshot** (~200 GB):


So they should always use the pruned relay chain snapshot, even if they used the archived asset hub snapshot?

node-infrastructure/run-a-node/polkadot-hub-rpc.md

eshaben · 2025-12-08T16:27:19Z

node-infrastructure/run-a-node/polkadot-hub-rpc.md

+                    # Check https://snapshots.polkadot.io/ for the latest snapshot URL
+                    export SNAPSHOT_URL_ASSET_HUB="https://snapshots.polkadot.io/polkadot-asset-hub-rocksdb-prune/LATEST"


Can this be run separately from the rclone commands? Curious about this so I can possibly update formatting of this section.

Yes it can. It's only use is for the rclone command though.

node-infrastructure/run-a-node/polkadot-hub-rpc.md

eshaben · 2025-12-08T16:31:57Z

node-infrastructure/run-a-node/polkadot-hub-rpc.md

+            sudo journalctl -u polkadot-hub-rpc -f
+            ```
+
+    7. You can use a few different commands to verify your node is running properly:


These commands can be used regardless of docker and systemd setup, right?

The sudo journalctl... commands are systemd specific. The docker ... commands are ... docker specific.

the ones I was talking about were curl commands 😆

haha so yes for the curl commands, they apply for both docker and systemd setup.

eshaben

just looking for clarity on what steps are for what setup option (docker vs systemd)

eshaben · 2025-12-08T19:10:42Z

node-infrastructure/run-a-collator/collator.md

+- **Docker-based Setup**: Best for simpler setup and maintenance
+- **Manual/systemd Setup**: Best for production environments requiring more control
+
+=== "Docker Setup"


Are these steps required before generating a node key?

No. Theses steps are part of installing dependencies that will be used later in this guide.

eshaben · 2025-12-08T19:11:28Z

node-infrastructure/run-a-collator.md

+  sudo mkdir -p /var/lib/polkadot-collator
+  ```
+
+2. Generate your node key using Docker with the following command:


This is the same regardless of whether the user wants to go with docker or systemd setup?

eshaben · 2025-12-08T19:11:45Z

node-infrastructure/run-a-collator.md

+
+Generate an account for on-chain transactions as follows:
+
+1. Generate an account key with `sr25519` scheme using the following command:


This is the same regardless of whether the user wants to go with docker or systemd setup?

eshaben · 2025-12-08T19:13:30Z

node-infrastructure/run-a-collator/collator.md

+
+## Create User and Directory Structure
+
+1. Create a dedicated user with the following command:


are these systemd-specific instructions?

Nice catch. Updated to have a docker setup and a systemd setup

eshaben · 2025-12-08T19:16:17Z

node-infrastructure/run-a-collator.md

+
+## Register Collator for Selection
+
+System parachains use different collator selection mechanisms. Explore the following tabs to see how mechanisms compare and determine the most suitable mechanism for registering your collator.


most suitable mechanism? Is this right? It's not like the user gets to decide, right? Like the parachain defines how to register collators

If my understanding is correct, I'm struggling with why we're listing out this information. Can't we just say these are some of the mechanisms seen in the wild, refer to your parachain's documentation to determine how to register your collator? I could maybe add specifics for like system parachains here and explicitly state they use invulnerables, right?

Even with the fellowship decisions, "Technical Fellowship may manage some system parachain collators" - this just seems a little vague to me. What are we trying to say?

most suitable mechanism? Is this right? It's not like the user gets to decide, right? Like the parachain defines how to register collators

Correct, it is up to how the parachain is configured.

If my understanding is correct, I'm struggling with why we're listing out this information. Can't we just say these are some of the mechanisms seen in the wild, refer to your parachain's documentation to determine how to register your collator? I could maybe add specifics for like system parachains here and explicitly state they use invulnerables, right?

Yes. We can remove this information and say, consult the specific parachain's documentation for onboarding a collator.

Even with the fellowship decisions, "Technical Fellowship may manage some system parachain collators" - this just seems a little vague to me. What are we trying to say?

For system parachains there is no place where users can go to figure out how to onboard, they would have to drop into a Polkadot Element channel and ask around or communicate with Parity to ask what the onboarding process is.

eshaben · 2025-12-08T19:22:51Z

@brunopgalvao I've made a lot of changes locally, just awaiting your responses so I can make modifications as needed. Thanks!

- Switch from rocksdb to paritydb snapshots where available - Update relay chain pruned snapshot size from ~200 GB to ~822 GB - Fix non-existent pruned snapshot URLs (Asset Hub, People Chain) - Clarify storage requirements: archive nodes (~1.2 TB) vs pruned nodes (200+ GB) - Update system parachain snapshot sizes with accurate values

Remove duplicate tabs for archive/pruned snapshots since the same archive snapshots are used for both node types.

Restructure Run the Collator section to provide both Docker and systemd setup options in tabs, consistent with other node documentation.

…into bruno/infra/running-a-collator

eshaben · 2025-12-09T20:57:02Z

node-infrastructure/run-a-node/parachain-rpc.md

-                    - `--size-only`: Only transfers if sizes differ (prevents unnecessary re-downloads)
+            2. Download the snapshots:
+
+                **Parachain archive snapshot** (People Chain example, ~71 GB):


Why is this number 71 GB, but above you said a parachain archive is 900 GB?

Because you need to take into account ~822 GB for the relay chain.

eshaben · 2025-12-09T20:57:20Z

node-infrastructure/run-a-node/parachain-rpc.md

 - **Storage**:
-    - Storage requirements vary by parachain. System parachains: Asset Hub (~600-800 GB), Bridge Hub (~500-600 GB), Collectives (~400-500 GB), People Chain (~300-400 GB), Coretime (~300-400 GB). For non-system parachains, check the [snapshot sizes](https://snapshots.polkadot.io/){target=\_blank} if available.
-    - Additional 200+ GB for relay chain pruned database
+    - Archive node: Storage varies by parachain. Using snapshots, system parachain totals are: Asset Hub (~1.2 TB), Bridge Hub (~1.1 TB), Collectives (~1 TB), People Chain (~900 GB), Coretime (~900 GB). For non-system parachains, check the [snapshot sizes](https://snapshots.polkadot.io/){target=\_blank} and add ~822 GB for the relay chain.


822 GB for the archived relay chain?

I guess I'm confused why we're combining the values here (I'm assuming you're adding archive parachain snapshot + archive relay chain - although was not clear initially to me from this sentence alone) but our instructions do not include using an archived relay chain.

~822 GB for a pruned snapshot of the relay chain. So you need to add the storage size needed for the parachain + the relay chain. An archived version of the relay chain would be ~3.19 TB...

eshaben · 2025-12-09T21:22:05Z

node-infrastructure/run-a-node/parachain-rpc.md

+- **CPU**: 8+ cores (16+ cores for high traffic)
+- **Memory**: 64 GB RAM minimum (128 GB recommended for high traffic)
+- **Storage**:
+    - **Archive node**: Storage varies by parachain. Using snapshots, system parachain totals are: Asset Hub (~1.2 TB), Bridge Hub (~1.1 TB), Collectives (~1 TB), People Chain (~900 GB), Coretime (~900 GB). For non-system parachains, check the [snapshot sizes](https://snapshots.polkadot.io/){target=\_blank} and add ~822 GB for the relay chain


The link doesn't provide info on non-system parachains - seems like only relay chains?

Yes. https://snapshots.polkadot.io only supports sysetm parachains and relay chains.
For non-system parachains, the user would need to consult the associated parachain's documentation, perhaps they have a snapshot service or maybe not.

I believe there is a not around line 108:

!!! note

Snapshots are available for system parachains and the Polkadot relay chain. For other parachains, check with the parachain team for snapshot availability or sync from genesis.

eshaben · 2025-12-09T21:23:05Z

node-infrastructure/run-a-node/parachain-rpc.md

+            mkdir -p my-node-data/chains/polkadot/db
+            ```
+
+        2. Choose between archive (complete history; ~71 GB for People Chain) or pruned (recent state; TODO: ERIN) snapshots of the parachain and set the snapshot URL accordingly:


do we know pruned storage amounts? Or should I say something generic like depends on specified number of blocks to keep?

I have a few TODO: ERINs scattered around this PR, all are trying to add pruned storage amounts

There is no pruned snapshot for people parachain just archive snapshot.

eshaben · 2025-12-09T21:25:42Z

node-infrastructure/run-a-node/parachain-rpc.md

+            === "Archive"
+
+                ```bash
+                # Check https://snapshots.polkadot.io/ for the latest snapshot URL
+                export SNAPSHOT_URL_PARACHAIN="https://snapshots.polkadot.io/polkadot-people-rocksdb-archive/INSERT_LATEST"
+                ```
+
+            === "Pruned"
+
+                ```bash
+                # Check https://snapshots.polkadot.io/ for the latest snapshot URL
+                export SNAPSHOT_URL_PARACHAIN="https://snapshots.polkadot.io/polkadot-people-rocksdb-prune/INSERT_LATEST"
+                ```
+


I had this change locally before you committed a bunch of stuff. I saw you got rid of the pruned info. So I'm curious why you got rid of it? Should we give the user the option? This seems like an easy way to do that

Because I later found out that there was no pruned snapshot for people... which is fine because they can run the parachain in pruned mode and it will sync quickly.

eshaben · 2025-12-09T21:41:31Z

node-infrastructure/run-a-collator.md

+The registration process varies by system parachain. General steps include the following:
+
+1. Check the existing collators for your target parachain:
+    <!--TODO: What is the URL? Also, why is the user following this step? Will this query return info needed in next steps or ???-->


This should still be here? When can this be addressed?

Can remove that. Personally, as a best practice, I am not in favor of adding TODO's in code unless it is a dire need (usually there is another way around it).

I have included an example screenshot for collatorSelection.invulnerables() here:

Add node infrastructure guides and restructure section #1246 (comment)

brunopgalvao self-assigned this Nov 18, 2025

brunopgalvao changed the title ~~Add system parachain RPC guide and enhance documentation~~ Add Polkadot Hub RPC node, system parachain RPC node, and collator pages Nov 18, 2025

brunopgalvao added B0 - Needs Review Pull request is ready for review C1 - Medium Medium priority task A0 - New Content Pull request contains new content pages labels Nov 18, 2025

Merge branch 'staging/product-ia' into bruno/infra/running-a-collator

43339dc

- Resolve conflict in .nav.yml by keeping 'Node Infrastructure' and adding 'Technical Reference' - Integrate latest changes from staging/product-ia

brunopgalvao changed the title ~~Add Polkadot Hub RPC node, system parachain RPC node, and collator pages~~ Add node infrastructure guides and restructure section Nov 18, 2025

brunopgalvao added 2 commits November 19, 2025 14:39

Merge staging/product-ia into bruno/infra/running-a-collator

6dc4298

Resolve merge conflict in .nav.yml by keeping the Node Infrastructure navigation structure from this branch.

brunopgalvao marked this pull request as ready for review November 19, 2025 09:56

brunopgalvao requested a review from a team as a code owner November 19, 2025 09:56

brunopgalvao requested a review from nhussein11 November 19, 2025 09:56

kapetan3sid assigned dawnkelly09 Nov 19, 2025

kapetan3sid requested a review from dawnkelly09 November 19, 2025 14:13

brunopgalvao and others added 4 commits November 20, 2025 18:01

Update content

ed97e71

initial look at collator.md

656920e

tweak admo

b04156a

wip - collator page

dbd2d98

dawnkelly09 reviewed Nov 21, 2025

View reviewed changes

dawnkelly09 and others added 9 commits November 21, 2025 12:39

finish collator pre-tech review pass

fc25788

clean up formatting for RPC node page

2de9f46

formatting edits

3b791ac

Update guides

b84d71d

Merge remote changes with tabbed content structure

7870c65

Use polkadot-parachain binary instead of polkadot-omni-node

a9f8340

Update llms

35d51a0

Download polkadot-parachain binary from GitHub releases instead of Do…

fda3e1c

…cker

Use MkDocs admonition syntax for notes and warnings

e517594

wip rpc page cleanup

6822d57

nhussein11 reviewed Nov 24, 2025

View reviewed changes

Update content

65132ed

brunopgalvao requested a review from nhussein11 November 26, 2025 05:40

nhussein11 approved these changes Nov 26, 2025

View reviewed changes

kapetan3sid assigned eshaben Dec 1, 2025

kapetan3sid requested a review from eshaben December 1, 2025 14:37

artkaseman reviewed Dec 3, 2025

View reviewed changes

brunopgalvao added 3 commits December 4, 2025 12:19

Merge remote-tracking branch 'origin/staging/product-ia' into bruno/i…

1f55b3e

…nfra/running-a-collator

Update AI index pages for node infrastructure docs

5569d63

eshaben unassigned dawnkelly09 Dec 4, 2025

eshaben reviewed Dec 8, 2025

View reviewed changes

brunopgalvao and others added 8 commits December 9, 2025 16:06

Simplify snapshot sections to single archive instructions

decf9d5

Remove duplicate tabs for archive/pruned snapshots since the same archive snapshots are used for both node types.

Add Docker setup option for collator deployment

41f3548

Restructure Run the Collator section to provide both Docker and systemd setup options in tabs, consistent with other node documentation.

LLMS files

9a200f2

cleanup docs

1993fdb

Merge remote-tracking branch 'origin/bruno/infra/running-a-collator' …

10dc550

…into bruno/infra/running-a-collator

add bruno's changes to my changes

cc37481

add bruno's changes to my changes

88cd077

eshaben reviewed Dec 9, 2025

View reviewed changes

fix link

8485f32

eshaben reviewed Dec 9, 2025

View reviewed changes


		## Complete Initial Sync

		Your collator must sync both the relay chain and parachain before producing blocks.


		## Install Dependencies

		1. Install Rust using the following commands:

		# Check https://snapshots.polkadot.io/ for the latest snapshot URL
		export SNAPSHOT_URL_ASSET_HUB="https://snapshots.polkadot.io/polkadot-asset-hub-rocksdb-prune/LATEST"


		Generate an account for on-chain transactions as follows:

		1. Generate an account key with `sr25519` scheme using the following command:


		## Create User and Directory Structure

		1. Create a dedicated user with the following command:


		## Register Collator for Selection

		System parachains use different collator selection mechanisms. Explore the following tabs to see how mechanisms compare and determine the most suitable mechanism for registering your collator.

Add node infrastructure guides and restructure section #1246

Are you sure you want to change the base?

Add node infrastructure guides and restructure section #1246

Uh oh!

Conversation

brunopgalvao commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

dawnkelly09 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nhussein11 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

brunopgalvao commented Nov 26, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

artkaseman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

brunopgalvao commented Dec 4, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eshaben Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eshaben left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brunopgalvao commented Nov 18, 2025 •

edited

Loading

eshaben Dec 9, 2025 •

edited

Loading