Add async backing technical reference #1179
Conversation
- still need to finalize diagrams and give better context of them
charlesHetterich
had a problem deploying
to
n8n-sending
GitHub Actions
Failure
charlesHetterich
had a problem deploying
to
n8n-sending
GitHub Actions
Failure
charlesHetterich
temporarily deployed
to
n8n-sending
GitHub Actions
Inactive
charlesHetterich
temporarily deployed
to
n8n-receiving
GitHub Actions
Inactive
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
…tions - improved timeline graphs of inclusion pipeline
charlesHetterich
changed the title
charlesHetterich
changed the title
There was a problem hiding this comment.
Pull Request Overview
This PR adds new documentation for Polkadot's asynchronous backing and inclusion pipeline features, explaining how rollup blocks are processed and validated in the relay chain.
- Introduces comprehensive documentation for asynchronous backing, including configuration options and visual comparisons between synchronous and asynchronous processing
- Adds a new page explaining the inclusion pipeline stages (Generation, Backing, and Inclusion)
- Updates the llms.txt index to reflect the new documentation pages
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 9 comments.
| File | Description |
|---|---|
| reference/parachains/consensus/inclusion-pipeline.md | New documentation page explaining the multi-step pipeline for rollup block processing, including Generation, Backing, and Inclusion stages |
| reference/parachains/consensus/async-backing.md | New comprehensive documentation on asynchronous backing, including governance configurations, processing diagrams, and multi-core scenarios |
| llms.txt | Updated documentation index with new page entries and incremented page count from 269 to 271 |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
This is ready for feedback. I split this up into two technical reference pages
Everything is in good condition besides the section titled Additionally there are 3 empty links in
Which, I was hoping there are definition these could link to? |
charlesHetterich
added
A0 - New Content
brunopgalvao
left a comment
brunopgalvao
left a comment
There was a problem hiding this comment.
Left my review. Overall I would focus on making sure the concept of async backing flows well for readers. I left some comments that can help in the direction.
|
|
||
| Notice how `P2` starts before the backing stage of `P1` | ||
|
|
||
| In the multi-core scenario |
There was a problem hiding this comment.
This heading has no explanatory text - just a diagram. Need to add context explaining what the multi-core/elastic scaling scenario demonstrates.
|
|
||
| *in progress* | ||
|
|
||
| In the synchronous scenario, both the collators and validators draw context from the relay parent of the prior parablock, which lives on the relay chain. This makes the Backing and Generation steps tightly coupled to the prior parablock completing the inclusion pipeline. As a result, one parablock can be processed every other relay block, and only `0.5` seconds are assigned for execution. |
There was a problem hiding this comment.
I would recommend adding context here for developers who are new to Polkadot - noting that synchronous backing was the old way, and the new, better way which is baked into the protocol is asynchronous backing. Something like this:
The Polkadot-parachain protocol originally operated in synchronous mode, where both collators and validators drew context exclusively from the relay parent of the prior parablock, which lives on the relay chain. This made the Backing and Generation steps tightly coupled to the prior parablock completing the entire inclusion pipeline. As a result, one parablock could only be processed every other relay block, with just 0.5 seconds assigned for execution.
The modern protocol now uses asynchronous backing, where both collators and validators have access to /reference/parachains/consensus/inclusion-pipeline as an additional context source. The Backing and Generation steps are no longer coupled to the prior block completing the full inclusion pipeline. Instead, the prior parablock only needs to complete the generation step and be added to the Unincluded Segments before the next parablock can begin the Backing and Generation steps.
This results in one parablock being processed every relay block (instead of every other relay block), and allows for more time to execute during the Generation step (0.5s → 2s).
There was a problem hiding this comment.
And add links to key terms.
There was a problem hiding this comment.
LGTM! Left some minor comments but approved :) @charlesHetterich, please ensure you mark the conversions as solved once you've fixed those comments, whether Bruno's or mine.
| ## Synchronous VS. Asynchronous Processing | ||
|
|
||
| <!-- In the synchronous scenario, both the collators and validators draw context from the relay parent of the prior parablock, which lives on the relay chain. This makes the Backing and Generation steps tightly coupled to the prior parablock completing the inclusion pipeline. As a result, one parablock can be processed *every other* relay block, and only `0.5` seconds are assigned for execution. --> |
There was a problem hiding this comment.
Please remove all these commented lines.
|
|
||
| * **Unincluded Segments**: Chains of candidate parablocks that have yet to be included in the relay chain, i.e. they can contain blocks at any stage pre-inclusion. The core functionality that [Async Backing](/reference/parachains/consensus/async-backing) brings is the ability to build on these unincluded segments of block ancestors rather than building only on ancestors included in the relay chain state. This context source lives on the collators. | ||
|
|
||
| **Generation**: Collators *execute* their blockchain's core functionality to generate a new block, producing a [proof-of-validity](https://paritytech.github.io/polkadot-sdk/book/types/availability.html?#proof-of-validity) (PoV), which is passed to validators selected for backing. The PoV is composed of: |
There was a problem hiding this comment.
Not sure how up to date is this link, or if it's maintained. I saw that Reinhard used a link to the wiki for the ticket he was addressing, maybe it'd be good if we use the same one here just to keep consistency
Co-authored-by: Nicolás Hussein <80422357+nhussein11@users.noreply.github.com>
📝 Description
Provide a clear and concise description of your changes.
🔍 Review Preference
Choose one:
🤖 AI-Ready Docs
If content changed, regenerate AI files:
python3 scripts/generate_llms.py✅ Checklist