Skip to content

feat: add sequencer subcommand with embedded store#2102

Draft
sergerad wants to merge 16 commits into
nextfrom
sergerad-embedded-store-phase-3a
Draft

feat: add sequencer subcommand with embedded store#2102
sergerad wants to merge 16 commits into
nextfrom
sergerad-embedded-store-phase-3a

Conversation

@sergerad
Copy link
Copy Markdown
Collaborator

@sergerad sergerad commented May 20, 2026

Relates to #1961. Implements Phase 3a.

Adds a miden-node sequencer subcommand that runs the store, block-producer, and RPC components in a single process, eliminating the gRPC hops between them for store reads, block application, and transaction submission.

Some of these changes (enums for remote vs local store/block-producer) will be refactored / removed in Phase 4 (cleanup).

TLDR is:

  1. miden-node sequencer command
  2. sequencer.proto provides sequencer.NtxBuilderApi for ntx-builder to point to
  3. replica.proto providers block and proof streaming service

Changes

Store crate — exposes the types needed for in-process use: State, StoreApi, ProvenTipWriter, ApplyBlockError, and a serve_replica function that runs the proof scheduler alongside a new replica.Api gRPC service (see below).

replica.proto — new internal proto that promotes store.StoreReplica out of store.proto into its own replica.Api service with its own descriptor. This decouples the replica streaming interface from the store's client-facing proto surface.

sequencer.proto — new internal proto that promotes store.NtxBuilder out of store.proto into its own sequencer.NtxBuilderApi service with its own descriptor.

Block-producer crate — adds LocalStoreClient (wraps Arc<State> directly) alongside the existing gRPC-based client, unified under a StoreClient enum. Adds EmbeddedBlockProducer, a parallel struct to BlockProducer that takes Arc<State> instead of a store URL. EmbeddedBlockProducer::start() returns a BlockProducerHandle — a cloneable in-process handle that bypasses gRPC for transaction and batch submission — together with a future that drives the gRPC server, batch builder, and block builder.

RPC crate — adds StoreBackend enum (Remote(StoreRpcClient) | Embedded(Arc<StoreApi>)) so RpcService can dispatch store reads in-process. Adds BlockProducerBackend enum (Remote(BlockProducerClient) | Embedded(BlockProducerHandle)) so transaction submission is routed directly to the in-process mempool without a loopback gRPC round-trip. Adds EmbeddedRpc, a parallel struct to Rpc that takes Arc<StoreApi> and Option<BlockProducerBackend> instead of URLs.

bin/node — adds sequencer bootstrap and sequencer start subcommands. start wires the three embedded components together: serve_replica, EmbeddedBlockProducer::start(), and EmbeddedRpc run as concurrent tasks sharing a single Arc<State>. Three service endpoints are exposed: the client-facing RPC API, the block-producer API, and the replica streaming API.

bin/ntx-builder — replaced --store.url with --sequencer.url with the intention being that only the sequencer services ntx-builder queries that are store related.

What stays the same

The existing store, block-producer, and rpc subcommands are untouched — CI stays green throughout. The sequencer is purely additive.

@@ -0,0 +1,154 @@
use std::collections::HashMap;
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Mirko-von-Leipzig having server/embedded.rs doesn't really make sense. But I think potentially this whole block-producer crate might move into the sequencer crate? Or at least we need to just return to this on phase 4 / cleanup to decide how to properly structure it when we remove the RPC path for block producer.

/// The two services expose an identical interface; the only difference is which proto package
/// hosts the service declaration. The embedded sequencer serves `sequencer.NtxBuilderApi` so
/// the ntx-builder binary can point its `--store.url` at the sequencer without depending on the
/// standalone store binary.
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note this, another thing for phase 4 / cleanup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant