Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
8a1b7e3
first pass async backing technical reference
charlesHetterich Nov 5, 2025
30e9e3d
split "async backing" into "async backing" & "inclusion pipeline" sec…
charlesHetterich Nov 7, 2025
f21afb1
renamed parachain language to rollup
charlesHetterich Nov 7, 2025
5bc9f0f
committing all difs generated by workflow script on commits
charlesHetterich Nov 7, 2025
e7e40dc
replaced all references of *rollup* --> *parachain/parablock*
charlesHetterich Nov 7, 2025
9ddd44f
added description to inclusion-pipeline.md
charlesHetterich Nov 7, 2025
353d84a
fixed typos pointed out by copilot and updated diagrams
charlesHetterich Nov 19, 2025
3f7c4ad
committing generated content
charlesHetterich Nov 19, 2025
9c21f6b
added links to definitions for , , and
charlesHetterich Nov 19, 2025
a8561ce
removed unnecessary from internal link
charlesHetterich Nov 19, 2025
214af0d
improve `Generation` and `Backing` definitions within the Inclusion P…
charlesHetterich Nov 25, 2025
864b3bf
adding generated files
charlesHetterich Nov 25, 2025
426bf7a
added intro to inclusion pipeline page
charlesHetterich Nov 25, 2025
1ee9875
add ai generated pages
charlesHetterich Nov 25, 2025
1aed941
improved backing definition in the inclusion pipeline reference
charlesHetterich Nov 25, 2025
86f6a2f
improved sync vs async section of async backing
charlesHetterich Nov 26, 2025
317b284
fixed mermaid chart
charlesHetterich Nov 26, 2025
ce479bc
Apply suggestions from code review
0xlukem Dec 2, 2025
aca7036
WIP inclusion-pipeline
0xlukem Dec 2, 2025
bc7f4f1
styling feedback
0xlukem Dec 4, 2025
85df8ce
Merge branch 'staging/product-ia' into charles/async-backing-techninc…
0xlukem Dec 4, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2,327 changes: 159 additions & 2,168 deletions .ai/categories/basics.md

Large diffs are not rendered by default.

3,786 changes: 247 additions & 3,539 deletions .ai/categories/dapps.md

Large diffs are not rendered by default.

2,633 changes: 128 additions & 2,505 deletions .ai/categories/infrastructure.md

Large diffs are not rendered by default.

2,625 changes: 124 additions & 2,501 deletions .ai/categories/networks.md

Large diffs are not rendered by default.

2,235 changes: 107 additions & 2,128 deletions .ai/categories/parachains.md

Large diffs are not rendered by default.

3,235 changes: 140 additions & 3,095 deletions .ai/categories/polkadot-protocol.md

Large diffs are not rendered by default.

493 changes: 0 additions & 493 deletions .ai/categories/reference.md

Large diffs are not rendered by default.

2,888 changes: 74 additions & 2,814 deletions .ai/categories/smart-contracts.md

Large diffs are not rendered by default.

6,234 changes: 1,143 additions & 5,091 deletions .ai/categories/tooling.md

Large diffs are not rendered by default.

2,863 changes: 0 additions & 2,863 deletions .ai/site-index.json

Large diffs are not rendered by default.

272 changes: 271 additions & 1 deletion .reference/parachains/consensus/async-backing.md
Original file line number Diff line number Diff line change
@@ -1 +1,271 @@
TODO
---
title: Asynchronous Backing
description: Understand how asynchronous backing pipelines parachain block production, the protocol changes it introduces on the Relay Chain, and how parachains participate safely and efficiently.
categories: Polkadot Protocol
---

# Asynchronous Backing

## Introduction

Asynchronous backing often shortened to _Async Backing_ is a parachain protocol feature that significantly improves performance, enabling parachains to produce blocks twice as fast (every 6 seconds instead of every 12) and to provide 4x more execution time per block (2 seconds instead of 0.5).

Technically, async backing is a parachain [configuration](https://paritytech.github.io/polkadot-sdk/master/cumulus_primitives_core/relay_chain/struct.AsyncBackingParams.html){target=\_blank} that allows collators and validators to build blocks ahead of the relay chain during the generation and backing stages of the [Inclusion Pipeline](/reference/parachains/consensus/inclusion-pipeline){target=\_blank} by using unincluded segments, which are chains of parachain blocks that have not yet been fully included in the relay chain. This decouples parachain block production from relay chain inclusion, improves coretime efficiency, and enables the parallel processing required for parachains to further scale throughput using [Elastic Scaling](/reference/parachains/consensus/elastic-scaling){target=\_blank}.

## Configurations
The following configurations can be set by on-chain governance, dictating how many blocks ahead of the relay chain a given parachain's collators can run:

- [**`max_candidate_depth`**](https://paritytech.github.io/polkadot-sdk/master/cumulus_primitives_core/relay_chain/struct.AsyncBackingParams.html#structfield.max_candidate_depth){target=\_blank}: the number of parablocks a collator can produce that are not yet included in the relay chain. A value of `2` means that there can be a maximum of 3 unincluded parablocks at any given time.
- [**`allowed_ancestry_len`**](https://paritytech.github.io/polkadot-sdk/master/cumulus_primitives_core/relay_chain/struct.AsyncBackingParams.html#structfield.allowed_ancestry_len){target=\_blank}: the oldest relay parent a parablock can be built on top of. A value of `1` means collators can start building blocks 6 seconds in advance.

## Synchronous VS. Asynchronous Processing

<!-- In the synchronous scenario, both the collators and validators draw context from the relay parent of the prior parablock, which lives on the relay chain. This makes the Backing and Generation steps tightly coupled to the prior parablock completing the inclusion pipeline. As a result, one parablock can be processed *every other* relay block, and only `0.5` seconds are assigned for execution. -->
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove all these commented lines.


The Polkadot-parachain protocol originally operated in synchronous mode, where both collators and validators drew context exclusively from the relay parent of the prior parablock, which lives on the relay chain. This made the Backing and Generation steps tightly coupled to the prior parablock completing the entire inclusion pipeline. As a result, one parablock could only be processed every other relay block, with just 0.5 seconds assigned for execution.

```mermaid
---
displayMode: compact
config:
themeCSS: "
#item1 { fill: #450693; stroke: #450693; } \n
#item2 { fill: #8C00FF; stroke: #8C00FF; } \n
#item3 { fill: #FFC400; stroke: #FFC400; } \n
#r { fill: #eb4172; stroke:none; font-size: 20px; } \n
svg text { font-size: 20px !important; } \n
svg .sectionTitle { font-size: 20px !important; } \n #p1padTop { display: none; } \n

/* Hide ALL task labels by default */
text.taskText,
text.taskTextOutside,
[class*='taskText'] tspan { display: none !important; } \n

/* Show labels for the 'r' group (inside or outside, incl. tspans) */
text.taskText[id^='r'],
text.taskTextOutside[id^='r'],
text[id^='r'] tspan { display: inline !important; font-size: 20px; color: #000 !important; } \n

/* Keep section titles styled */
.sectionTitle { fill: #000 !important; font-weight: 700; font-size: 18px; } \n

/* Hide the first two section titles (F1, F2). Change indexes if needed. */
.sectionTitle:nth-of-type(1),
.sectionTitle:nth-of-type(2) { display: none !important; } \n

/* Also hide SPACING row labels on the left */
text.taskTextOutside[id^='p1padTop'] { display: none !important; } \n
"
themeVariables:
sectionBkgColor: '#fff'
gantt:
numberSectionStyles: 1
barHeight: 70
gridLineStartPadding: 100
---
%%{init: {"gantt": {"barHeight": 70 }}}%%
gantt
dateFormat YYYY
axisFormat %y
tickInterval '10year'

section F1
R1 : r, 1905, 1907
R2 : r, 1911, 1913
R3 : r, 1917, 1919
R4 : r, 1923, 1925

section F2
SPACING : p1padTop, 1901, 1924

section P1
X : item1, 1900, 1901
Backing : item2, 1901, 1906
Inclusion : item3, 1906, 1912

section P2
X : item1, 1912, 1913
Backing : item2, 1913, 1918
Inclusion : item3, 1918, 1924


```

<!-- In the asynchronous scenario, where both the collators and validators have access to [Unincluded Segments](/reference/parachains/consensus/inclusion-pipeline) as an additional context source, the Backing and Generation steps are no longer coupled to the prior block completing the full inclusion pipeline. Instead, the prior parablock only needs to complete the generation step and be added to the Unincluded Segments before the next parablock can begin the Backing and Generation steps.

This results in one parablock being processed *every* relay block, and allows for `2` seconds of execution. -->

The modern protocol now uses asynchronous backing, where both collators and validators have access to [unincluded segments](/reference/parachains/consensus/inclusion-pipeline){target=\_blank} as an additional context source. The Backing and Generation steps are no longer coupled to the prior block completing the full inclusion pipeline. Instead, the prior parablock only needs to complete the generation step and be added to the Unincluded Segments before the next parablock can begin the Backing and Generation steps.

This results in one parablock being processed every relay block (instead of every other relay block), and allows for more time to execute during the Generation step (0.5s β†’ 2s).

```mermaid
---
displayMode: compact
config:
themeCSS: "
#item1 { fill: #450693; stroke: #450693; } \n
#item2 { fill: #8C00FF; stroke: #8C00FF; } \n
#item3 { fill: #FFC400; stroke: #FFC400; } \n
#r { fill: #eb4172; stroke:none; font-size: 20px; } \n
svg text { font-size: 20px !important; } \n
svg .sectionTitle { font-size: 20px !important; } \n #p1padTop { display: none; } \n

/* Hide ALL task labels by default */
text.taskText,
text.taskTextOutside,
[class*='taskText'] tspan { display: none !important; } \n

/* Show labels for the 'r' group (inside or outside, incl. tspans) */
text.taskText[id^='r'],
text.taskTextOutside[id^='r'],
text[id^='r'] tspan { display: inline !important; font-size: 20px; color: #000 !important; } \n

/* Keep section titles styled */
.sectionTitle { fill: #000 !important; font-weight: 700; font-size: 18px; } \n

/* Hide the first two section titles (F1, F2). Change indexes if needed. */
.sectionTitle:nth-of-type(1),
.sectionTitle:nth-of-type(2) { display: none !important; } \n

/* Also hide SPACING row labels on the left */
text.taskTextOutside[id^='p1padTop'] { display: none !important; } \n
"
themeVariables:
sectionBkgColor: '#fff'
gantt:
numberSectionStyles: 1
barHeight: 70
gridLineStartPadding: 100
---
%%{init: {"gantt": {"barHeight": 70 }}}%%
gantt
dateFormat YYYY
axisFormat %y
tickInterval '10year'

section F1
R1 : r, 1905, 1907
R2 : r, 1911, 1913
R3 : r, 1917, 1919
R4 : r, 1923, 1925
R5 : r, 1929, 1931

section F2
SPACING : p1padTop, 1901, 1930

section P1
X : item1, 1900, 1902
Backing : item2, 1902, 1912
Inclusion : item3, 1912, 1918

section P2
X : item1, 1906, 1908
Backing : item2, 1908, 1918
Inclusion : item3, 1918, 1924

section P3
X : item1, 1912, 1914
Backing : item2, 1914, 1924f
Inclusion : item3, 1924, 1930

section P4
X : item1, 1918, 1920
Backing : item2, 1920, 1930
```

<!-- In the multi-core scenario, by using elastic scaling we can further increase the the number of parablocks processed for each relay block. By using two cores, here we double the number of parablocks being processed in a 30 second period:

```mermaid
---
displayMode: compact
config:
themeCSS: "
#item1 { fill: #450693; stroke: #450693; } \n
#item2 { fill: #8C00FF; stroke: #8C00FF; } \n
#item3 { fill: #FFC400; stroke: #FFC400; } \n
#r { fill: #eb4172; stroke: none; } \n
#p1padTop { display: none; } \n
text.taskText[id^=p1padTop] { fill: none !important; color: #000 !important; } \n


/* Default inside task text: white */ \n
.taskText { fill: #fff !important; color: #fff !important; font-weight: 700; font-size: 18px; } \n

/* Hide text inside bars */
.taskText { display: none !important; } \n

/* Outside labels and section titles: black */ \n
.taskTextOutside, .sectionTitle { fill: #000 !important; color: #000 !important; font-weight: 700; font-size: 18px; } \n

/* Inside text for #r items: black */ \n
text.taskText[id^=r] { fill: #fff !important; } \n
"
themeVariables:
sectionBkgColor: '#fff'
gantt:
numberSectionStyles: 1
barHeight: 70
gridLineStartPadding: 100
---
%%{init: {"gantt": {"barHeight": 70 }}}%%
gantt
dateFormat YYYY
axisFormat %y
%% this next line doesn't recognise 'decade' or 'year', but will silently ignore
tickInterval '10year'

section F1
R1 : r, 1905, 1907
R2 : r, 1911, 1913
R3 : r, 1917, 1919
R4 : r, 1923, 1925
R5 : r, 1929, 1931

section F2
SPACING : p1padTop, 1901, 1930

section P1
X : item1, 1900, 1902
Backing : item2, 1902, 1912
Inclusion : item3, 1912, 1918

section P2
X : item1, 1906, 1908
Backing : item2, 1908, 1918
Inclusion : item3, 1918, 1924

section P3
X : item1, 1912, 1914
Backing : item2, 1914, 1924
Inclusion : item3, 1924, 1930

section F20
SPACING : p1padTop, 1901, 1930

section F21
R1 : r, 1905, 1907
R2 : r, 1911, 1913
R3 : r, 1917, 1919
R4 : r, 1923, 1925
R5 : r, 1929, 1931

section F22
SPACING : p1padTop, 1901, 1930


section P4
X : item1, 1900, 1902
Backing : item2, 1902, 1912
Inclusion : item3, 1912, 1918

section P5
X : item1, 1906, 1908
Backing : item2, 1908, 1918
Inclusion : item3, 1918, 1924

section P6
X : item1, 1912, 1914
Backing : item2, 1914, 1924
Inclusion : item3, 1924, 1930
``` -->
61 changes: 61 additions & 0 deletions .reference/parachains/consensus/inclusion-pipeline.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
title: Inclusion Pipeline
description: Learn how Polkadot validates parachain blocks through the Inclusion Pipeline, detailing context, generation, backing, and inclusion stages.
categories: Polkadot Protocol
---

# Inclusion Pipeline

## Introduction

The inclusion pipeline is the multi-stage process through which every parachain block (parablock) is validated and secured before being finalized in the Polkadot relay chain. This pipeline ensures that all parachain blocks meet validity requirements through progressive verification by multiple sets of validators.

The pipeline exists to provide Polkadot's security guarantees, rather than relying on a single validator group. Each parablock passes through multiple validation stages with different validator sets, ensuring that invalid blocks cannot be finalized even if some validators are malicious or compromised.

Whether a parachain uses synchronous or [asynchronous backing](/reference/parachains/consensus/async-backing){target=\_blank}, all parablocks follow the same inclusion pipeline. The difference is in the timing: asynchronous backing allows multiple blocks to be at different stages of the pipeline simultaneously (pipelining), while synchronous backing processes one block through the entire pipeline before starting the next.

## Pipeline Stages

The inclusion pipeline consists of three main stages:

```mermaid
%%{init: {"flowchart": {"nodeSpacing": 40, "rankSpacing": 60}}}%%
flowchart LR
%% Keep the pipeline on one row (container is hidden)
subgraph Row[" "]
direction LR
G["Generation"] --> B["Backing"] --> I["Inclusion"]
end
style Row fill:none,stroke:none

%% Context: plain text (no box) pointing to both G and B
C["Context"]:::nobox
C -.-> G
C -.-> B

classDef nobox fill:none,stroke:none,color:inherit;
```
### Context

To build a parablock during the generation and backing stages, collators and validators require access to the state context of the parachain. This context is derived from two sources:

- **Relay Parent**: The relay chain block to which the parablock is anchored. Note that the relay parent of a parablock is always different from the relay chain block that eventually includes it. This context source resides on the relay chain.

- **Unincluded Segments**: Chains of candidate parablocks that have not yet been included in the relay chain. These segments represent sequences of block ancestors and may contain candidates at any stage pre-inclusion. A key feature enabled by [async backing](/reference/parachains/consensus/async-backing){target=\_blank} is that collators can build new parablocks on top of these unincluded ancestors rather than being limited to ancestors already included in the relay chain state. This context source resides on the collators.

### Generation

Collators execute their blockchain core functionality to generate a new block, producing a [proof-of-validity](https://paritytech.github.io/polkadot-sdk/book/types/availability.html?#proof-of-validity){target=\_blank} (PoV), which is passed to validators selected for backing. The PoV is composed of:

- A list of state transitions called the **block candidate**
- The values in the parachain's database that the block modifies
- The hashes of the unaffected points in the Merkle tree


### Backing

A subset of active validators verify that the parablock follows the state transition rules of the parachain and sign a [validity statement](https://paritytech.github.io/polkadot-sdk/book/types/backing.html?#validity-attestation){target=\_blank} about the PoV which can have a positive or negative outcome. With enough positive statements (at least 2/3 of assigned validators), the candidate is considered backable. It is then noted in a fork on the relay chain, at which point it is considered backed, ready for the next stage of the pipeline.

### Inclusion

Validators gossip [erasure code chunks](https://paritytech.github.io/polkadot-sdk/book/types/availability.html#erasure-chunk){target=\_blank} and put the parablock through the final [approval process](https://paritytech.github.io/polkadot-sdk/book/protocol-approval.html){target=\_blank} before it is considered *included* in the relay chain.
Loading
Loading