diff --git a/CLAUDE.md b/CLAUDE.md index ed1aa3c9..2dc09416 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -112,4 +112,4 @@ When updating documentation: - Navigation structure must be updated in `docs.json` when adding new pages - Interactive RPC documentation is generated from the source `methods.mdx` file - Test findings in `tests/README.md` track documentation accuracy against implementation -- Use relative imports for snippets and components (e.g., `/snippets/icons.mdx`) \ No newline at end of file +- Use relative imports for snippets and components (e.g., `/snippets/icons.mdx`) diff --git a/sdk/v0.53/learn/concepts/baseapp.mdx b/sdk/v0.53/learn/concepts/baseapp.mdx new file mode 100644 index 00000000..f7f67e88 --- /dev/null +++ b/sdk/v0.53/learn/concepts/baseapp.mdx @@ -0,0 +1,214 @@ +--- +title: BaseApp Overview +--- + +`BaseApp` is the execution engine of every Cosmos SDK chain. It implements [ABCI (Application Blockchain Interface)](/sdk/next/learn/intro/sdk-app-architecture#abci-application-blockchain-interface), the protocol CometBFT uses to communicate with the application, and translates those calls into module execution, transaction processing, and state transitions. + +Every Cosmos SDK chain embeds `BaseApp`. Your `app.go` creates a `BaseApp` instance, configures it with modules, keepers, and middleware, and the resulting struct is what CometBFT communicates with directly. `BaseApp` provides the base layer of execution infrastructure to your blockchain application. Without it, every chain would need to independently implement ABCI handling, signature verification, gas metering, message routing, block hook orchestration, and state commitment. + +## Architectural position + +`BaseApp` sits between CometBFT and the modules: + +``` +CometBFT (consensus engine) + ↓ ABCI (InitChain, CheckTx, FinalizeBlock, Commit, ...) +BaseApp + ↓ orchestrates block execution +ModuleManager + ↓ dispatches to individual modules +Modules (x/auth, x/bank, x/staking, ...) + ↓ read/write +State (KVStores) +``` + +CometBFT drives the block lifecycle by calling ABCI methods on `BaseApp`. `BaseApp` handles each call, delegating to registered lifecycle hooks and routing messages to the appropriate module handlers. Modules contain the business logic, and KVStores hold the resulting state. + +## Key fields + +[`BaseApp`](https://github.com/cosmos/cosmos-sdk/blob/main/baseapp/baseapp.go) is defined in `baseapp/baseapp.go`. It holds references to everything needed to run a chain: + +```go +type BaseApp struct { + logger log.Logger + name string // application name from abci.BlockInfo + db dbm.DB // common DB backend + cms storetypes.CommitMultiStore // Main (uncached) state + storeLoader StoreLoader // function to handle store loading + grpcQueryRouter *GRPCQueryRouter // router for redirecting gRPC query calls + msgServiceRouter *MsgServiceRouter // router for redirecting Msg service messages + txDecoder sdk.TxDecoder // unmarshal []byte into sdk.Tx + mempool mempool.Mempool + anteHandler sdk.AnteHandler // ante handler for fee and auth + postHandler sdk.PostHandler // post handler, optional + // ... + sealed bool + // ... + chainID string + // ... +} +``` + +For a complete list of fields, see the [`BaseApp` struct definition](https://github.com/cosmos/cosmos-sdk/blob/main/baseapp/baseapp.go). + +- `cms` (CommitMultiStore): the root state store. All module substores are mounted here, and all state reads and writes during block execution pass through it. {/* cosmos/cosmos-sdk baseapp/baseapp.go:92 — cms storetypes.CommitMultiStore // Main (uncached) state */} +- `storeLoader`: a function that opens and mounts the individual module stores at application startup. {/* cosmos/cosmos-sdk baseapp/baseapp.go:94 — storeLoader StoreLoader // function to handle store loading, may be overridden with SetStoreLoader() */} +- `grpcQueryRouter`: routes incoming gRPC queries to the correct module's query handler. {/* cosmos/cosmos-sdk baseapp/baseapp.go:95 — grpcQueryRouter *GRPCQueryRouter // router for redirecting gRPC query calls */} +- `msgServiceRouter`: routes each message in a transaction to the correct module's `MsgServer` handler. {/* cosmos/cosmos-sdk baseapp/baseapp.go:96 — msgServiceRouter *MsgServiceRouter // router for redirecting Msg service messages */} +- `txDecoder`: decodes raw transaction bytes from CometBFT into an `sdk.Tx`. {/* cosmos/cosmos-sdk baseapp/baseapp.go:98 — txDecoder sdk.TxDecoder // unmarshal []byte into sdk.Tx */} +- `anteHandler`: runs before message execution to handle cross-cutting concerns: signature verification, sequence validation, and fee deduction. {/* cosmos/cosmos-sdk baseapp/baseapp.go:102 — anteHandler sdk.AnteHandler // ante handler for fee and auth */} +- `postHandler`: optional middleware that runs after message execution — used for tasks such as tipping or post-execution state adjustments. {/* cosmos/cosmos-sdk baseapp/baseapp.go:103 — postHandler sdk.PostHandler // post handler, optional */} +- `sealed`: set to `true` after `LoadLatestVersion` is called. Setter methods panic if called after sealing. + +## Initialization and sealing + +{/* TODO: move the full initialization and sealing explanation (two-phase config model, code example, ordering rules, why everything must be in NewApp) to the app.go page, where it's actionable. Keep only the concept here. */} + +`BaseApp` enforces a configuration lifecycle: setter methods must be called before `LoadLatestVersion` is invoked. When `LoadLatestVersion` runs, it validates required components, initializes the check state, and sets `sealed` to `true`. {/* cosmos/cosmos-sdk baseapp/baseapp.go — func (app *BaseApp) Init() validates cms and stateStore, initializes checkState, then seals */} Any setter called after sealing panics. On first launch, CometBFT calls `InitChain`. It stores `ConsensusParams` from the genesis file — block gas limit, max block size, evidence rules — in the `ParamStore`, where they can later be adjusted via on-chain governance. {/* cosmos/cosmos-sdk baseapp/abci.go:89 — InitChain calls StoreConsensusParams if req.ConsensusParams != nil */} It initializes all volatile states by branching the root store, sets the block gas meter to infinite so genesis transactions are not gas-constrained, and calls the application's `initChainer`, which runs each module's `InitGenesis` to populate initial state. {/* cosmos/cosmos-sdk baseapp/abci.go:82,122,124 — stateManager.SetState for finalize+check, WithBlockGasMeter(InfiniteGasMeter), abciHandlers.InitChainer */} How this shapes the structure of `app.go` is covered in the next section. + +## Transaction decoding + +Transactions arrive from CometBFT as raw bytes. Before `BaseApp` can validate or execute them, it must decode them into the SDK's transaction type using the `TxDecoder`: + +``` +[]byte tx + ↓ +TxDecoder + ↓ +sdk.Tx +``` + +This step happens before the transaction enters the execution pipeline. Without it, `BaseApp` cannot inspect messages, run the ante handler, or route execution to the correct module. + +## Execution modes + +`BaseApp` does not execute everything against the same mutable state. It maintains branched, copy-on-write views of the committed root state for different execution contexts: + +- `CheckTx` (`ExecModeCheck`): validates a transaction before it enters the mempool, without committing state. +- `FinalizeBlock` (`ExecModeFinalize`): executes transactions in a proposed block against a branched state that is committed at the end. +- `PrepareProposal` (`ExecModePrepareProposal`): runs when the node is the block proposer, assembling a candidate block. Executes against a branched state that is never committed. {/* cosmos/cosmos-sdk baseapp/abci.go — PrepareProposal sets prepareProposalState from app.cms and runs runTx in execModePrepareProposal */} +- `ProcessProposal` (`ExecModeProcessProposal`): runs on every validator to validate an incoming proposal. Also executes against a branched state that is never committed. {/* cosmos/cosmos-sdk baseapp/abci.go — ProcessProposal sets processProposalState from app.cms and runs runTx in execModeProcessProposal */} +- `Simulate` (`ExecModeSimulate`): runs a transaction for gas estimation without committing state. + +This separation ensures that validation, proposal handling, and simulation cannot accidentally mutate committed application state. + +## The transaction execution pipeline + +When `BaseApp` processes a transaction, it runs through a structured pipeline: + +``` +RunTx + ├─ DecodeTx → raw bytes → sdk.Tx + ├─ AnteHandler → signatures, sequence, fees, gas setup + ├─ RunMsgs → route each message to the correct module handler + └─ PostHandler → optional post-execution middleware +``` + +If the AnteHandler fails, message execution does not begin. If any message fails, the transaction reverts atomically; all messages commit or none do. + +## AnteHandler + +The `AnteHandler` is middleware that runs before any message in a transaction executes. It verifies cryptographic signatures, validates and increments the account sequence number, deducts transaction fees, and sets up the gas meter for the transaction. + +{/* TODO: move the AnteHandler configuration example (SetAnteHandler, HandlerOptions, ordering rules) to the app.go page, where it's actionable. Keep only the concept here. then link out to the app.go page. */} + +If the AnteHandler fails, the transaction is rejected and its messages never execute. If the AnteHandler succeeds but a message later fails, the AnteHandler's state writes — fee deduction and sequence increment — are already flushed to `finalizeBlockState` and will be committed with the block. {/* cosmos/cosmos-sdk baseapp/baseapp.go:932 — msCache.Write() is called immediately after AnteHandler success, before RunMsgs begins */} Fees are charged even for transactions whose messages fail. + +## Message routing + +When a transaction contains messages, `BaseApp` routes each one to the appropriate module handler using the `MsgServiceRouter`. + +```go +type MsgServiceRouter struct { + routes map[string]MsgServiceHandler + // ... +} +``` +{/* cosmos/cosmos-sdk baseapp/msg_service_router.go:29 — type MsgServiceRouter struct { routes map[string]MsgServiceHandler; ... } */} + +The routing process has three steps: + +1. Registration: During app startup, each module calls `RegisterService`, which registers its message handlers keyed by message type URL (e.g., `/cosmos.bank.v1beta1.MsgSend`). {/* cosmos/cosmos-sdk baseapp/msg_service_router.go:71 — func (msr *MsgServiceRouter) RegisterService iterates service methods and registers handlers in the routes map keyed by message type URL */} +2. Lookup: At execution time, `Handler` looks up the registered handler for the incoming message's type URL. {/* cosmos/cosmos-sdk baseapp/msg_service_router.go:54 — func (msr *MsgServiceRouter) Handler(msg sdk.Msg) MsgServiceHandler — returns handler by type URL from routes map */} +3. Execution: The retrieved handler invokes the module's `MsgServer` implementation, which validates inputs, applies business rules, and updates state through the keeper. + +This routing is entirely type-URL-based. Modules do not need to know about each other at the routing level; `BaseApp` is the neutral coordinator. + +## Queries + +For read-only access to application state, `BaseApp` uses the **`GRPCQueryRouter`** to route incoming gRPC queries to the correct module query service. Queries bypass the transaction execution pipeline and directly read committed state. They do not go through the AnteHandler, do not consume gas in the same way, and do not mutate state. + +## Store management + +`BaseApp` owns the `CommitMultiStore` that holds all module state. At app startup, each module registers its store key, and `BaseApp` mounts the corresponding store: {/* cosmos/cosmos-sdk baseapp/baseapp.go:323 — func (app *BaseApp) MountKVStores mounts IAVL or DB-backed stores for each registered KV store key */} + +```go +app.MountKVStores(keys) +``` + +Before executing each transaction, `BaseApp` creates a cached, copy-on-write view of the multistore. All writes during that transaction occur in the cache. If the transaction succeeds, the cache is flushed to the underlying store. If the transaction fails at any point, the cache is discarded and no state changes are applied. + +## CheckTx and mempool validation + +Before a transaction reaches block execution, it goes through `CheckTx`. `BaseApp` runs the AnteHandler in CheckTx mode to validate signatures, check sequence numbers, and verify fees. Transactions that fail `CheckTx` are rejected and do not enter the mempool. + +`CheckTx` does not execute messages and does not write state. It exists solely to protect the mempool from invalid or malformed transactions before they consume block space. + +## Coordinating block execution + +When CometBFT calls `FinalizeBlock`, `BaseApp` runs the full block execution pipeline in order: {/* cosmos/cosmos-sdk baseapp/abci.go:868 — func (app *BaseApp) FinalizeBlock(req *abci.RequestFinalizeBlock) — FinalizeBlock entry point */} + +``` +FinalizeBlock + ├─ PreBlock → module pre-block hooks + ├─ BeginBlock → module begin-block hooks + ├─ For each transaction: + │ ├─ AnteHandler (signature verification, fee deduction, gas setup) + │ ├─ Message routing and execution + │ └─ Commit or revert (atomic per-transaction) + └─ EndBlock → module end-block hooks + → returns AppHash +``` + +**PreBlock** runs before any block logic. It handles changes that must take effect before the block begins, such as activating a chain upgrade or modifying consensus parameters. {/* cosmos/cosmos-sdk baseapp/abci.go:757 — PreBlock called at the start of internalFinalizeBlock, before BeginBlock */} + +**BeginBlock** runs after PreBlock and handles per-block housekeeping: minting inflation rewards, distributing staking rewards, resetting per-block counters. {/* cosmos/cosmos-sdk baseapp/abci.go:762 — BeginBlock called after PreBlock, before transaction iteration */} + +**Transactions** execute sequentially in block order. Each is atomic: if any message fails, the entire transaction reverts. + +**EndBlock** runs after all transactions. It handles logic that depends on the block's cumulative state — for example, tallying governance votes after all vote messages have been processed, or updating validator power after all delegation changes. {/* cosmos/cosmos-sdk baseapp/abci.go:802 — EndBlock called after all transactions have executed */} + +After `FinalizeBlock` completes, `BaseApp` computes and returns the app hash — the Merkle root of all committed state. {/* cosmos/cosmos-sdk baseapp/abci.go:999,1013 — res.AppHash = app.workingHash() — app hash computed via cms.WorkingHash() and returned in the FinalizeBlock response */} When CometBFT subsequently calls `Commit`, `BaseApp` writes `finalizeBlockState` to the root store, resets `checkState` to the newly committed state, and clears `finalizeBlockState` to `nil` in preparation for the next block. {/* cosmos/cosmos-sdk baseapp/abci.go:1062,1085,1087 — app.cms.Commit(), stateManager.SetState(execModeCheck, ...), stateManager.ClearState(execModeFinalize) */} + +{/* todo: add link to lifecycle page and module page */} + +## Module Manager + +`BaseApp` exposes `PreBlock`, `BeginBlock`, and `EndBlock` as lifecycle hook points. Every standard SDK application wires these to a `ModuleManager`, which holds the full set of registered modules and their execution ordering. When a hook fires, `ModuleManager` iterates its ordered module list and calls each module's corresponding hook in sequence. Ordering matters: some modules depend on others having already updated state before they run. + +{/* TODO: move the ModuleManager struct, SetPreBlocker/SetBeginBlocker/SetEndBlocker wiring, mm.BeginBlock(ctx) delegation, and ordering configuration to the app.go page, where it's actionable. add link to that page*/} + +## Block proposal and vote extensions + +ABCI 2.0 added a proposal phase that runs during consensus rounds, before `FinalizeBlock` executes. `BaseApp` exposes four handlers for this phase, with default implementations wired at construction: + +- `PrepareProposal`: called on the current block proposer to assemble a block from the mempool. The default selects transactions up to the block gas limit. Chains can override this to implement custom ordering, filtering, or injection of protocol-level transactions. +- `ProcessProposal`: called on every validator to validate an incoming proposal. The default accepts any structurally valid proposal. Chains that use `PrepareProposal` to inject data typically also override this to verify that data is present and valid. +- `ExtendVote` / `VerifyVoteExtension`: allow validators to attach arbitrary data to their precommit votes and verify other validators' extensions. One major use case is oracle price feeds: validators inject off-chain data into consensus so it becomes available on-chain at block start. + +All four are configurable in `app.go` via `SetPrepareProposal`, `SetProcessProposal`, `SetExtendVoteHandler`, and `SetVerifyVoteExtensionHandler`. Chains that do not need custom behavior can leave the defaults in place. + +{/* TODO: move configuration examples and override patterns for PrepareProposal, ProcessProposal, ExtendVote, and VerifyVoteExtension to the app.go page, where it's actionable. Add link to that page. */} + +## Putting it all together + +`BaseApp` is the execution engine of a Cosmos SDK chain: + +``` +CometBFT → ABCI → BaseApp → Modules → State +``` + +It implements ABCI, coordinates the block lifecycle (PreBlock → BeginBlock → transactions → EndBlock), routes messages to module handlers via the `MsgServiceRouter`, routes queries via the `GRPCQueryRouter`, runs the AnteHandler before each transaction, and manages the multistore with copy-on-write caching for atomicity. State changes are committed at block end; validation and simulation run against branched state and never touch committed data. + +The next section explains `app.go`: how `BaseApp` is instantiated, configured, and wired with modules to produce a complete, running chain. +{/* todo: add link to app.go page */} \ No newline at end of file diff --git a/sdk/v0.53/learn/concepts/context-gas-and-events.mdx b/sdk/v0.53/learn/concepts/context-gas-and-events.mdx new file mode 100644 index 00000000..e8a224ad --- /dev/null +++ b/sdk/v0.53/learn/concepts/context-gas-and-events.mdx @@ -0,0 +1,167 @@ +--- +title: Execution Context, Gas, and Events +--- + +In the previous section, you learned how data is serialized and why every validator must encode state identically. This page covers the runtime environment that modules execute within: the context object that carries block metadata and state access, the gas system that limits computation, and the event system that allows modules to emit observable signals. + +## What is `sdk.Context` + +Every message handler, keeper method, and block hook in the Cosmos SDK receives an `sdk.Context`. It is the execution environment for a single unit of work — a transaction, a query, or a block hook — and carries everything that code needs to read state, emit events, and consume gas. + +The `Context` struct is defined in `types/context.go`: {/* cosmos/cosmos-sdk types/context.go:41 — type Context struct { ... } */} + +```go +type Context struct { + ms storetypes.MultiStore + chainID string + gasMeter storetypes.GasMeter + blockGasMeter storetypes.GasMeter + eventManager EventManagerI + // ... additional fields +} +``` + +Context is a value type, not a pointer. It is passed by value and mutated through `With*` methods that return a new copy. This means a module can safely derive a sub-context (for example, with a different gas meter) without affecting the caller's context. + +### Block metadata + +Context exposes read-only access to the current block's metadata: + +- `ctx.BlockHeight()` returns the current block number. {/* cosmos/cosmos-sdk types/context.go:91 — func (c Context) BlockHeight() int64 { return c.header.Height } */} +- `ctx.BlockTime()` returns the block's timestamp. {/* cosmos/cosmos-sdk types/context.go:92 — func (c Context) BlockTime() time.Time { return c.header.Time } */} +- `ctx.ChainID()` returns the chain identifier string. {/* cosmos/cosmos-sdk types/context.go:93 — func (c Context) ChainID() string { return c.chainID } */} +- `ctx.Logger()` returns a structured logger scoped to the current execution context. Modules use this for operational logging (e.g., logging an upgrade activation or an unexpected state) without affecting consensus. {/* cosmos/cosmos-sdk types/context.go:95 — func (c Context) Logger() log.Logger { return c.logger } */} + +These values are populated by BaseApp from the block header provided by CometBFT before any block logic runs. Modules read them to implement time-dependent logic (for example, checking whether a vesting period has elapsed) or to tag events with the block height. + +`ctx.IsCheckTx()` returns true when the context is being used for mempool validation rather than block execution. {/* cosmos/cosmos-sdk types/context.go:99 — func (c Context) IsCheckTx() bool { return c.checkTx } */} Modules that have logic specific to mempool pre-screening — such as skipping expensive computation during `CheckTx` — can branch on this flag. + +### Context and state access + +State is accessed through context. The context holds a reference to the multistore, and each keeper opens its own store through the context: + +```go +func (k Keeper) GetCount(ctx context.Context) (uint64, error) { + return k.counter.Get(ctx) +} +``` + +The keeper does not hold a direct reference to the live multistore; it opens its module's store from the context on each call. This is why context must be passed to every keeper method — it is the gateway to the current block's state, the gas meter, and the event manager for that execution unit. + +## Gas metering + +### What gas measures + +Gas is a unit of computation. In the Cosmos SDK, gas accounts for both computation and state access. Every store read, store write, and iterator step costs gas. Complex computations — such as signature verification in the AnteHandler — also cost gas. + +The gas system exists to prevent abuse. Without a gas limit, a single transaction could exhaust a node's resources with an unbounded computation or an unindexed state scan. + +### Gas limit and the transaction gas meter + +Every transaction specifies a gas limit in its `auth_info.fee.gas_limit` field. {/* cosmos/cosmos-sdk proto/cosmos/tx/v1beta1/tx.proto:235 — uint64 gas_limit = 2; — the gas_limit field in the Fee message of AuthInfo */} When BaseApp begins executing a transaction, it creates a `GasMeter` initialized with that limit and attaches it to the context. {/* cosmos/cosmos-sdk types/context.go:244 — func (c Context) WithGasMeter(meter storetypes.GasMeter) Context — used by BaseApp to attach the transaction gas meter to context */} + +The `GasMeter` interface provides two key methods: {/* cosmos/cosmos-sdk store/types/gas.go:41 — type GasMeter interface { GasConsumed() Gas; ConsumeGas(amount Gas, descriptor string); ... } */} + +```go +type GasMeter interface { + GasConsumed() Gas + ConsumeGas(amount Gas, descriptor string) + // ... +} +``` + +`GasConsumed` returns the total gas used so far in the current execution unit. {/* cosmos/cosmos-sdk store/types/gas.go:42 — GasConsumed() Gas — returns total gas consumed so far */} `ConsumeGas` adds to the running total and panics with `ErrorOutOfGas` if consumption exceeds the limit. {/* cosmos/cosmos-sdk store/types/gas.go:46 — ConsumeGas(amount Gas, descriptor string) — adds amount to consumed; panics with ErrorOutOfGas if limit exceeded */} {/* cosmos/cosmos-sdk store/types/gas.go:30 — type ErrorOutOfGas struct { Descriptor string } — the error type panicked when gas is exhausted */} + +### How gas is consumed + +Gas is consumed automatically at the store layer. Every read and write through the `GasKVStore` wrapper charges gas before delegating to the underlying store: + +- A `Get` (store read) charges a flat read cost plus a per-byte cost for the key and value. {/* cosmos/cosmos-sdk store/gaskv/store.go:66 — gs.gasMeter.ConsumeGas(gs.gasConfig.ReadCostFlat, ...) on every Get call */} +- A `Set` (store write) charges a flat write cost plus a per-byte cost for the key and value. {/* cosmos/cosmos-sdk store/gaskv/store.go:80 — gs.gasMeter.ConsumeGas(gs.gasConfig.WriteCostFlat, ...) on every Set call */} + +Modules do not need to manually track gas for ordinary state access — the store layer handles it automatically. Modules call `ctx.GasMeter().ConsumeGas(...)` directly only for computation costs that are not captured by store operations (for example, a module that performs a cryptographic operation outside the store). + +### When gas runs out + +If gas is exhausted during execution, `ConsumeGas` panics with `ErrorOutOfGas`. {/* cosmos/cosmos-sdk store/types/gas.go:116 — basicGasMeter.ConsumeGas panics with ErrorOutOfGas{descriptor} when g.consumed > g.limit */} BaseApp recovers from this panic, discards all state changes from the transaction, and returns an error to the user. Fees may still be charged for the gas consumed up to the point of failure, but none of the transaction's state changes are committed. + +### Block gas limit + +In addition to the per-transaction gas meter, there is a block-level gas meter that tracks total gas consumed by all transactions in a block. {/* cosmos/cosmos-sdk types/context.go:54 — blockGasMeter storetypes.GasMeter field in Context struct tracks block-level gas */} The block gas limit prevents a single block from consuming unbounded computation. If a transaction would cause the block's gas total to exceed the limit, it is excluded from the block. + +## Events + +### What events are + +Events are observable signals emitted during transaction and block execution. A module emits events to describe what happened: tokens were transferred, a validator was slashed, a governance proposal passed. Events carry structured key-value data alongside a type string. + +Events are not part of consensus state. They are not stored in the KVStore, do not affect the app hash, and are not required for deterministic execution. Instead, they are collected by BaseApp and included in the block result, where indexers, explorers, and relayers consume them. + +### EventManager + +Modules emit events through the `EventManager`, which is attached to the context: {/* cosmos/cosmos-sdk types/context.go:104 — func (c Context) EventManager() *EventManager { return c.eventManager } */} + +The `EventManager` is created fresh for each transaction and collects all events emitted during that execution. {/* cosmos/cosmos-sdk types/events.go:24 — type EventManager struct { events Events } — holds the slice of events accumulated during execution */} {/* cosmos/cosmos-sdk types/events.go:28 — func NewEventManager() *EventManager — creates an EventManager with an empty events slice */} + +### Emitting events + +Modules emit events using `EmitEvent` or `EmitTypedEvent`: + +```go +// emit an untyped event +ctx.EventManager().EmitEvent(sdk.NewEvent( + "increment", + sdk.NewAttribute("new_count", strconv.FormatUint(newCount, 10)), +)) +``` + +`EmitEvent` appends a raw key-value event to the manager's accumulated list. {/* cosmos/cosmos-sdk types/events.go:35 — func (em *EventManager) EmitEvent(event Event) — appends the event to em.events */} + +For events backed by protobuf message types, `EmitTypedEvent` serializes the message's fields into event attributes automatically: {/* cosmos/cosmos-sdk types/events.go:57 — func (em *EventManager) EmitTypedEvent(tev proto.Message) error — serializes proto message into an Event with attributes derived from its fields */} + +```go +ctx.EventManager().EmitTypedEvent(&types.EventCounterIncremented{ + NewCount: newCount, +}) +``` + +Using `EmitTypedEvent` is the modern approach. It provides type safety and makes the event schema explicit through proto definitions, allowing clients to deserialize events back into typed structs. + +### Block events and transaction events + +Events emitted during `BeginBlock` or `EndBlock` hooks are **block events** — they describe things that happened at the block level (inflation minted, validator updates applied). Events emitted inside a message handler are **transaction events** — they describe what a specific transaction did. + +Both types are included in the `FinalizeBlock` response that CometBFT returns to the network, but they are reported separately so clients can distinguish block-level activity from per-transaction activity. + +### Who consumes events + +Events are consumed outside the node: + +- **Block explorers** index events to show users what happened in a transaction (which tokens moved, which validator was slashed, which proposal passed). +- **Relayers** (IBC) subscribe to specific event types to detect packet sends and acknowledgments. +- **Indexers and off-chain services** build queryable databases of chain activity from event streams. Events can also be queried via the node's REST API and WebSocket endpoint. +- **Wallets and UIs** display event data to users as transaction receipts. + +Events are included in the block result that CometBFT returns after each block. They are not replayed or reprocessed; once a block is finalized, its events are fixed. + +## Putting it together + +During transaction execution, context, gas, and events work together as the runtime layer: + +``` +BaseApp creates Context for the transaction + ↓ +AnteHandler runs + → signature verification, fee deduction, gas meter initialized + ↓ +Message handler runs + → each store read/write consumes gas via GasKVStore + → module logic emits events via EventManager + ↓ +If gas exhausted → panic → state reverted, fees charged for gas consumed +If execution succeeds → state changes committed, events returned in block result +``` + +The context carries the gas meter and event manager into every keeper call. Gas is consumed transparently at the store layer. Events accumulate and are returned as part of the block result once execution completes. + +The next section explains how an SDK application is structured as a codebase: where modules live, what goes in `app/`, and how all the pieces are assembled. diff --git a/sdk/v0.53/learn/concepts/encoding.mdx b/sdk/v0.53/learn/concepts/encoding.mdx new file mode 100644 index 00000000..650352f8 --- /dev/null +++ b/sdk/v0.53/learn/concepts/encoding.mdx @@ -0,0 +1,352 @@ +--- +title: Encoding +--- + +In the previous section, you learned that state is stored as raw byte arrays in a key-value store and that modules encode structured data into those bytes before writing. This page explains how that encoding works, why the Cosmos SDK chose Protocol Buffers, and what that means for module development. + +## What is Protobuf? + +[Protocol Buffers](https://protobuf.dev/) (protobuf) is a language-neutral, binary serialization format developed by Google. You define your data structures in `.proto` files using a schema language, then generate code in your target language from that schema. The generated code handles serialization (converting structured data into bytes) and deserialization (converting bytes back into structured data). + +A simple protobuf message looks like this: + +```proto +message MsgSend { + string from_address = 1; + string to_address = 2; + repeated Coin amount = 3; +} +``` + +Each field has a name, a type, and a field number. The field numbers are what protobuf actually uses during encoding; field names are only present in the schema, not in the serialized bytes. This is why protobuf is compact and why you can rename fields without breaking wire compatibility. + +## Why the Cosmos SDK uses protobuf + +The Cosmos SDK uses protobuf for a fundamental reason: consensus requires determinism. + +Every validator in the network independently executes each block. After execution, each validator computes the AppHash, a cryptographic hash of the application state. For validators to agree on the AppHash, they must all produce exactly the same bytes for every piece of state they write. + +Protobuf satisfies this requirement. The Cosmos SDK enforces **deterministic protobuf serialization** so that identical data always produces identical bytes across all nodes. This is formalized in [ADR-027 (Deterministic Protobuf Serialization)](https://github.com/cosmos/cosmos-sdk/blob/v0.53.0/docs/architecture/adr-027-deterministic-protobuf-serialization.md), which specifies constraints such as requiring fields to appear in ascending field-number order and varint encodings to be as short as possible. The SDK validates incoming transactions against these rules before processing them, so a non-deterministically encoded transaction is rejected rather than producing divergent state. Every validator encoding the same data under these rules produces an identical byte sequence. + +Beyond determinism, protobuf provides: + +- **Compact encoding**: binary wire format is smaller than JSON or XML, which matters for transaction throughput and block size. +- **Schema evolution**: fields can be added or deprecated without breaking existing clients, which is critical for chain upgrades. +- **Code generation**: `.proto` files generate Go structs, gRPC service stubs, and REST gateway handlers automatically. +- **Cross-language support**: clients in any language can interact with the chain by generating code from the same `.proto` files. + +## Binary and JSON encoding + +The Cosmos SDK uses protobuf in two encoding modes: + +**Binary encoding** is the default for everything that participates in consensus: transactions written to blocks, state stored in KV stores, and genesis data. Binary encoding is compact and deterministic. When a transaction is broadcast to the network, it travels as protobuf binary. When a module writes state, it serializes values to protobuf binary before calling `Set` on the store. + +**JSON encoding** is used for human-readable output: the CLI, gRPC-gateway REST endpoints, and off-chain tooling. The Cosmos SDK uses protobuf's JSON encoding (`ProtoMarshalJSON`) rather than standard Go JSON, which preserves field names from the `.proto` schema and handles special types like `Any` correctly. {/* cosmos/cosmos-sdk codec/json.go:12 — var defaultJM = &jsonpb.Marshaler{OrigName: true, EmitDefaults: true, AnyResolver: nil} — OrigName:true preserves proto field names from the schema */} {/* cosmos/cosmos-sdk codec/proto_codec.go:149 — return ProtoMarshalJSON(o, pc.interfaceRegistry) — MarshalJSON delegates to ProtoMarshalJSON with the interface registry */} + +The most important rule: **binary encoding is consensus-critical**. Two validators must produce identical binary bytes for identical data. JSON is only used where humans or external clients need to read the data; it never influences the AppHash. + +```text +Consensus-critical path Human-readable path +───────────────────────── ───────────────────────── +Transaction bytes (binary) CLI output (JSON) +State KV values (binary) REST API responses (JSON) +Genesis KV state (binary) Block explorers (JSON) +``` + +Note: genesis data is distributed as JSON in `genesis.json`, but during chain initialization `InitGenesis` deserializes that JSON into protobuf structs and writes them to the KV store as binary. The KV store—and therefore the AppHash—only ever contains the binary form. + +## Transaction encoding + +Transactions are protobuf messages defined in `cosmos.tx.v1beta1`. A transaction is composed of three parts: {/* cosmos/cosmos-sdk proto/cosmos/tx/v1beta1/tx.proto:16 — message Tx { TxBody body = 1; AuthInfo auth_info = 2; repeated bytes signatures = 3; } */} + +```text +Tx + ├─ TxBody + │ └─ repeated google.protobuf.Any messages + ├─ AuthInfo + │ ├─ repeated SignerInfo (each with sequence) + │ └─ Fee + └─ repeated bytes signatures +``` + +- **TxBody** contains the messages to execute, serialized as `repeated google.protobuf.Any messages`. {/* cosmos/cosmos-sdk proto/cosmos/tx/v1beta1/tx.proto:104 — repeated google.protobuf.Any messages = 1 — TxBody.messages field */} +- **AuthInfo** contains signer information (including the per-signer sequence number) and fee. +- **signatures** contains the cryptographic signatures, one per signer. + +Messages inside the transaction are stored as `google.protobuf.Any` values so that a single transaction can contain multiple message types from different modules. + +When a user submits a transaction, the SDK encodes it as a `TxRaw`—a flat structure with the `TxBody` bytes, `AuthInfo` bytes, and signatures already serialized—and broadcasts that binary representation over the network. {/* cosmos/cosmos-sdk x/auth/tx/encoder.go:27 — return proto.Marshal(raw) — DefaultTxEncoder serializes TxRaw to binary bytes for broadcast */} + +## Transaction signing and `SignDoc` + +Transactions are not signed directly. Instead, the SDK constructs a deterministic structure called a **`SignDoc`**, which defines exactly what bytes the signer commits to: {/* cosmos/cosmos-sdk proto/cosmos/tx/v1beta1/tx.proto:50 — message SignDoc { bytes body_bytes = 1; bytes auth_info_bytes = 2; string chain_id = 3; uint64 account_number = 4; } */} + +```text +SignDoc + ├─ body_bytes (serialized TxBody) + ├─ auth_info_bytes (serialized AuthInfo, includes sequence per signer) + ├─ chain_id (prevents cross-chain replay) + └─ account_number (ties the signature to a specific on-chain account) +``` + +The `SignDoc` is serialized to protobuf binary and then signed with the user's private key: + +```text +signature = Sign(proto.Marshal(SignDoc)) +``` + +Because `SignDoc` is serialized deterministically, all validators verify the exact same bytes when checking transaction signatures. The per-signer sequence number lives in `AuthInfo.SignerInfo.sequence` and is included in `auth_info_bytes`, which is part of `SignDoc`—this is what prevents replay attacks. + +## How protobuf is used in modules + +Every piece of data that crosses a module boundary, gets stored, or travels over the wire is defined in a `.proto` file and serialized with protobuf. + +**Messages and transactions**: Each module defines its transaction messages in a `tx.proto` file. The `MsgSend` definition above is an example. When a user submits a transaction, the SDK serializes the transaction body (including its messages) to binary using protobuf before broadcasting it. {/* cosmos/cosmos-sdk x/auth/tx/encoder.go:27 — return proto.Marshal(raw) — DefaultTxEncoder serializes the transaction to protobuf binary bytes for broadcast */} + +**Queries**: Modules define their query services in `query.proto`. Request and response types are protobuf messages. The SDK uses gRPC for queries, and gRPC uses protobuf as its serialization format by definition. + +**State types**: Data stored in the KV store is protobuf-encoded. A module that stores a custom struct first marshals it to bytes using the codec, then writes those bytes to the store. When reading, it unmarshals the bytes back into the struct. + +**Genesis**: Genesis state is defined in `genesis.proto`. `InitGenesis` and `ExportGenesis` use protobuf to deserialize genesis state from `genesis.json` and serialize it back. {/* cosmos/cosmos-sdk x/auth/module.go:158 — cdc.MustUnmarshalJSON(data, &genesisState) — InitGenesis uses the JSON codec to unmarshal JSON genesis data into a protobuf-typed GenesisState struct */} {/* cosmos/cosmos-sdk x/auth/module.go:166 — return cdc.MustMarshalJSON(gs) — ExportGenesis serializes the genesis state back to JSON using the codec */} + +**State keys**: While values in the KV store are protobuf-encoded structs, keys are manually constructed byte sequences defined by each module. A module might store balances under a key like: + +```text +0x02 | address_bytes → protobuf-encoded Coin +``` + +Modules define their key layout with byte-prefix constants to avoid collisions with other modules and to maintain a predictable state layout. In practice, modules do not build raw keys by hand. Instead they create **prefix stores**, which automatically prepend a module-specific key prefix to all reads and writes: {/* cosmos/cosmos-sdk store/prefix/store.go:23 — func NewStore(parent types.KVStore, prefix []byte) Store — creates a prefix store scoped to a byte prefix */} + +```go +store := prefix.NewStore(ctx.KVStore(k.storeKey), types.KeyPrefixBalances) +// all Set/Get calls on `store` are automatically namespaced under KeyPrefixBalances +``` + +This gives each module a logical namespace inside its own store key without requiring global coordination over raw byte offsets. + +A concrete example shows how a module reads and writes typed state as bytes: {/* cosmos/cosmos-sdk codec/proto_codec.go:49 — func (pc *ProtoCodec) Marshal(o gogoproto.Message) ([]byte, error) — the Marshal method called as k.cdc.Marshal() in keeper code */} {/* cosmos/cosmos-sdk codec/proto_codec.go:78 — func (pc *ProtoCodec) Unmarshal(bz []byte, ptr gogoproto.Message) error — the Unmarshal method called as k.cdc.Unmarshal() in keeper code */} + +```go +// write: marshal the coin amount to bytes, then set in store +bz, err := k.cdc.Marshal(&amount) +store.Set(key, bz) + +// read: get bytes from store, unmarshal back to coin +var amount sdk.Coin +bz := store.Get(key) +k.cdc.Unmarshal(bz, &amount) +``` + +The codec (`k.cdc`) is the protobuf codec described in the next section. + +## The codec and interface registry + +The Cosmos SDK wraps protobuf in a **codec** that modules use for marshaling and unmarshaling. The primary implementation is [`ProtoCodec`](https://github.com/cosmos/cosmos-sdk/blob/v0.53.0/codec/proto_codec.go), which calls protobuf's `Marshal` and `Unmarshal` under the hood. {/* cosmos/cosmos-sdk codec/proto_codec.go:56 — return gogoproto.Marshal(o) — Marshal calls gogoproto.Marshal to produce binary bytes */} {/* cosmos/cosmos-sdk codec/proto_codec.go:79 — err := gogoproto.Unmarshal(bz, ptr) — Unmarshal calls gogoproto.Unmarshal to decode binary bytes */} + +```go +type ProtoCodec struct { + interfaceRegistry types.InterfaceRegistry +} + +func (pc *ProtoCodec) Marshal(o ProtoMarshaler) ([]byte, error) +func (pc *ProtoCodec) Unmarshal(bz []byte, ptr ProtoMarshaler) error +``` + +Keepers hold a reference to the codec and use it to encode and decode state: + +```go +type Keeper struct { + cdc codec.BinaryCodec + store storetypes.StoreKey +} +``` + +The codec is initialized once at app startup and passed to each keeper during initialization. + +### Interface types and `Any` + +Protobuf is strongly typed. You cannot store a field as "some implementation of an interface" directly in a protobuf message. The Cosmos SDK solves this using protobuf's [`google.protobuf.Any`](https://protobuf.dev/programming-guides/proto3/#any), which wraps an arbitrary message type alongside a URL that identifies what type it contains. + +`Any` is used anywhere the SDK needs to serialize a value whose concrete type is not known at compile time. The most common example is public keys. An account might use a secp256k1 key, an ed25519 key, or a multisig key. The `BaseAccount` stores the public key as `Any`: {/* cosmos/cosmos-sdk x/auth/types/auth.pb.go:34 — PubKey *any.Any — the generated Go struct stores pub_key as *any.Any, not a concrete type */} + +```proto +message BaseAccount { + string address = 1; + google.protobuf.Any pub_key = 2; + uint64 account_number = 3; + uint64 sequence = 4; +} +``` + +The `Any` field holds the serialized public key bytes plus a type URL like `/cosmos.crypto.secp256k1.PubKey`. When the SDK reads the account, it uses the type URL to look up the concrete Go type, then unmarshals the bytes into that type. + +#### Messages inside transactions + +Transaction messages are the most common use of `Any` in the SDK. A transaction can carry multiple message types from different modules—`bank.MsgSend`, `staking.MsgDelegate`, `gov.MsgVote`—in a single `TxBody`. Because protobuf requires concrete types at the field level, each message is packed into an `Any` before being placed inside the transaction: + +```text +MsgSend + ↓ pack into Any +Any { + type_url: "/cosmos.bank.v1beta1.MsgSend" + value: +} + ↓ placed in TxBody.messages +repeated google.protobuf.Any messages +``` + +During decoding, the SDK reads the `type_url`, looks up the concrete type in the interface registry, and unmarshals the bytes into the correct message struct. This is why every `sdk.Msg` implementation must be registered with `RegisterInterfaces` before the application starts. + +This lookup is handled by the **interface registry**. + +### Interface registry + +The [`InterfaceRegistry`](https://github.com/cosmos/cosmos-sdk/blob/v0.53.0/codec/types/interface_registry.go) is a runtime map from type URLs to Go types. {/* cosmos/cosmos-sdk codec/types/interface_registry.go:115 — type interfaceRegistry struct { ... typeURLMap map[string]reflect.Type ... } — the registry struct definition with its typeURLMap field */} When the SDK encounters an `Any` value, it queries the registry with the type URL to find the concrete Go type, then uses protobuf to unmarshal the bytes. {/* cosmos/cosmos-sdk codec/types/interface_registry.go:395 — typ, found := registry.typeURLMap[typeURL] — Resolve looks up the concrete Go type by type URL */} + +```text +Any { type_url, value_bytes } + ↓ + InterfaceRegistry.Resolve(type_url) + ↓ + concrete Go type + ↓ + proto.Unmarshal(value_bytes, concreteType) +``` + +{/* cosmos/cosmos-sdk codec/types/interface_registry.go:394 — func (registry *interfaceRegistry) Resolve(typeURL string) (proto.Message, error) — the public Resolve method that the diagram above represents */} + +Without the interface registry, the SDK cannot decode `Any` values. This is why types must be explicitly registered before they can be deserialized. + +## Registering interface implementations + +Because the interface registry is a runtime lookup table, every concrete type that implements an SDK interface must be registered before the application starts. This is done with `RegisterInterfaces`: + +```go +// in codec registration, typically in module.go or types/codec.go +func RegisterInterfaces(registry codectypes.InterfaceRegistry) { + registry.RegisterImplementations( + (*cryptotypes.PubKey)(nil), + &secp256k1.PubKey{}, + &ed25519.PubKey{}, + ) +} +``` + +This tells the registry: "a `PubKey` interface can be a `secp256k1.PubKey` or an `ed25519.PubKey`." {/* cosmos/cosmos-sdk crypto/codec/proto.go:13 — crypto module's RegisterInterfaces registers ed25519.PubKey, secp256k1.PubKey, and multisig.LegacyAminoPubKey under the cosmos.crypto.PubKey interface */} If a type is used in an `Any` field anywhere in the application and is not registered, the codec will fail to unmarshal it and return an error. {/* cosmos/cosmos-sdk codec/types/interface_registry.go:397 — return nil, fmt.Errorf("unable to resolve type URL %s", typeURL) — Resolve returns an error when the type URL has not been registered */} + +Each module calls `RegisterInterfaces` during app initialization, and `app.go` calls these registration functions through the module manager when building the app. {/* cosmos/cosmos-sdk types/module/module.go:120 — func (bm BasicManager) RegisterInterfaces(registry types.InterfaceRegistry) — iterates all modules calling m.RegisterInterfaces(registry) for each */} Custom types that implement SDK interfaces must follow the same pattern. + +## Proto-to-code generation workflow + +Writing `.proto` files produces `.pb.go` files through a code generation step. The generated Go code contains struct definitions, marshal/unmarshal methods, and gRPC service stubs. You never edit these generated files directly. + +The workflow is: + +**1. Write the `.proto` file** + +Proto files for a module live in the `proto/` directory at the repository root: + +``` +proto/myapp/mymodule/v1/ +├── tx.proto # message types (MsgAdd, MsgAddResponse, ...) +├── query.proto # query service (QueryCount, ...) +├── state.proto # on-chain state types +└── genesis.proto # genesis state +``` + +A message definition: + +```proto +syntax = "proto3"; +package myapp.mymodule.v1; + +message MsgAdd { + string sender = 1; + uint64 add = 2; +} + +message MsgAddResponse { + uint64 updated_count = 1; +} + +service Msg { + rpc Add(MsgAdd) returns (MsgAddResponse); +} +``` + +**2. Run code generation** + +```bash +make proto-gen +``` + +This runs `buf` (or `protoc` with plugins) against the `.proto` files and produces Go code under the module's `types/` directory: + +``` +x/mymodule/types/ +├── tx.pb.go # generated: MsgAdd, MsgAddResponse, Marshal/Unmarshal methods +├── query.pb.go # generated: query request/response types +├── query.pb.gw.go # generated: gRPC-gateway REST handlers +└── state.pb.go # generated: on-chain state types +``` + +**3. Use the generated types** + +The generated structs implement `proto.Message` and can be passed directly to the codec for marshaling, registered with the interface registry, and used in keeper methods and message handlers: + +```go +// handler receives the generated type +func (m msgServer) Add(ctx context.Context, req *types.MsgAdd) (*types.MsgAddResponse, error) { + count, err := m.AddCount(ctx, req.Sender, req.Add) + if err != nil { + return nil, err + } + return &types.MsgAddResponse{UpdatedCount: count}, nil +} +``` + +The generated gRPC service stub is registered with BaseApp's message router, connecting the handler to the transaction execution pipeline automatically. + +## Legacy Amino encoding + +Before adopting protobuf, the Cosmos SDK used a custom serialization format called **Amino**. The `LegacyAmino` type still exists in the SDK and wraps `*amino.Codec`: {/* cosmos/cosmos-sdk codec/amino.go:16 — type LegacyAmino struct { Amino *amino.Codec } */} + +```go +type LegacyAmino struct { + Amino *amino.Codec +} +``` + +Amino was used for transaction encoding, JSON signing documents, and interface serialization. It has been replaced by protobuf for all of these roles, but some legacy components still reference it: + +- `LegacyAmino` is still present in the codec package for backward-compatibility +- `LegacyAminoPubKey` (multisig) is registered alongside protobuf public key types +- Some older chains, hardware wallets, and client tooling depend on Amino JSON signing + +New modules and chains should use protobuf exclusively. The `LegacyAmino` codec is maintained for compatibility but is not used in the consensus-critical path. + +## Encoding in context + +Every layer of the Cosmos SDK depends on encoding: + +``` +Transaction (binary protobuf) + ↓ broadcast over p2p +CometBFT + ↓ passes raw bytes to application +BaseApp + ↓ decodes transaction, extracts messages +Module MsgServer + ↓ processes message, calls keeper +Keeper + ↓ marshals state value to bytes +KVStore (raw bytes) + ↓ committed to disk +AppHash (Merkle root over all KV bytes) +``` + +Because every step uses protobuf, every step is deterministic. Two validators executing the same transactions always produce the same bytes at every layer, and therefore always arrive at the same AppHash. + +For more details on the encoding subsystem and advanced codec usage, see the [Encoding reference](/sdk/v0.53/learn/advanced/encoding). + +The next section explains the runtime execution environment that modules operate within: `sdk.Context`, gas metering, and events. diff --git a/sdk/v0.53/learn/concepts/lifecycle.mdx b/sdk/v0.53/learn/concepts/lifecycle.mdx new file mode 100644 index 00000000..8144f473 --- /dev/null +++ b/sdk/v0.53/learn/concepts/lifecycle.mdx @@ -0,0 +1,210 @@ +--- +title: Transaction Lifecycle +--- + +In the [Transactions, Messages, and Queries](/sdk/v0.53/learn/concepts/transactions) page, you learned that transactions are the actual mechanism that authorizes and executes logic on the chain. This page explains how transactions are validated, executed, and committed in the Cosmos SDK. + +Before building with the Cosmos SDK, it's important to connect the high-level architecture from [SDK Application Architecture](/sdk/v0.53/learn/intro/sdk-app-architecture) with how blocks and transactions actually execute in code. + +A Cosmos SDK application can be conceptually split into layers: + +- [CometBFT](/cometbft) (consensus engine) — orders and proposes blocks +- [ABCI](/sdk/v0.53/learn/intro/sdk-app-architecture#abci-application-blockchain-interface) (Application-Blockchain Interface) — the protocol CometBFT uses to talk to the Cosmos SDK application +- SDK application (BaseApp + modules) — the deterministic state machine that executes transactions +- Protobuf schemas — define transactions, messages, state, and query types + +{/* todo: link to sdk and encoding pages */} + +This page maps the block and transaction lifecycle back to those layers. + +## ABCI overview + +CometBFT and the SDK application are two separate processes with distinct responsibilities. +- [CometBFT](/cometbft/v0.38/docs/introduction/intro) handles consensus: ordering transactions, managing validators, and driving block production. +- The [SDK application](/sdk/v0.53/learn/intro/sdk-app-architecture#cosmos-sdk-application) handles state: executing transactions and updating the chain's data. + +The **ABCI** (Application Blockchain Interface) is the protocol that connects them: CometBFT calls ABCI methods on the application to drive each phase of the block lifecycle, and the application responds. + +The **BaseApp** {/* todo: link to baseapp */} struct is the SDK's implementation of the ABCI interface. It receives these calls from CometBFT and orchestrates execution across modules. Modules plug into BaseApp and execute their logic during the appropriate phases. + +```python ++---------------------+ | +-------------------------+ +| CometBFT | | | SDK Application | +| (Consensus) | ABCI | (BaseApp + modules) | ++---------------------+ | +-------------------------+ + | +InitChain (once) | + Chain start -------------------|------> InitGenesis per module + | +CheckTx (per submitted tx) | + Mempool validation ------------|------> decode · verify · validate + |<------ accept → Mempool + | +PrepareProposal (proposer only) | + Build block proposal ----------|------> select txs (MaxTxBytes, MaxGas) + | +ProcessProposal (all validators) | + Evaluate proposal -------------|------> verify txs → ACCEPT / REJECT + | +FinalizeBlock (per block) | + Execute block -----------------|------> PreBlock hooks + | BeginBlock hooks + | For each tx: + | AnteHandler + | → message routing + | → MsgServer (module logic) + | EndBlock hooks + | Return AppHash +Commit | + Persist state -----------------|------> persist state to disk + |<------ return AppHash +``` + +## InitChain (genesis only) + +`InitChain` runs once when the chain starts for the first time. BaseApp loads `genesis.json`, which defines the chain's initial state, and calls each module's `InitGenesis` to populate its store. The initial validator set is established. Genesis runs before the first block begins. + +{/* todo: link to store#genesis-and-chain-initialization */} + +## CheckTx and the mempool + +Before a transaction can enter a block, it goes through `CheckTx`: + +``` +User + ↓ +Node + ↓ +ABCI: CheckTx + ↓ +Mempool +``` + +Transactions are sent as raw protobuf-encoded bytes. + +{/* todo: link to encoding section above */} + +During `CheckTx`, the SDK application's BaseApp decodes the transaction, verifies signatures and sequences, validates fees and gas, and performs basic message validation. + +{/* todo: link to signatures, sequence fees, and gas sections above */} +If validation fails, the transaction is rejected. If it passes, it enters the mempool. The mempool is a node's in-memory pool of validated transactions waiting to be included in a block. + +Validated transactions wait in the mempool until CometBFT selects a block proposer for the next round. + +## PrepareProposal + +Each round, [CometBFT](/cometbft/v0.38/docs/introduction/intro#intro-to-abci) selects one validator to propose a block. `PrepareProposal` is called on that validator only. BaseApp selects transactions from the mempool respecting the block's `MaxTxBytes` and `MaxGas` limits and returns the final transaction list. {/* todo: link to baseapp#prepareproposal */} + +## ProcessProposal + +Once the other validators receive the proposed block, CometBFT calls `ProcessProposal`. BaseApp verifies each transaction and returns `ACCEPT` or `REJECT`. No state is written. Once more than two-thirds of voting power accepts the block and consensus is reached, CometBFT calls `FinalizeBlock`. {/* todo: link to baseapp#processproposal */} + +## FinalizeBlock + +CometBFT calls `FinalizeBlock` once per block. Inside `FinalizeBlock`, BaseApp runs these phases in order: + +``` +PreBlock → BeginBlock → transaction execution → EndBlock +``` + +### PreBlock + +PreBlock runs before `BeginBlock` and is generally used for logic that must affect consensus-critical state before the block begins, such as activating a chain upgrade or modifying consensus parameters. Because these changes need to take effect before any block logic runs, they cannot happen inside `BeginBlock`. Modules may implement this via the `HasPreBlocker` extension interface on their `AppModule` (typically in `x//module.go`), and the application's `ModuleManager` invokes all registered PreBlockers during `FinalizeBlock`. + +### BeginBlock + +`BeginBlock` runs after PreBlock and handles per-block housekeeping that must happen before any transactions execute, regardless of the transactions in the block. Common uses include minting inflation rewards, distributing staking rewards, and resetting per-block state. Modules implement this via the `BeginBlock` function in `x//module.go`. + +### Transaction execution + +After BeginBlock, BaseApp iterates over each transaction in the block and runs it through a fixed pipeline. + +#### Step 1: AnteHandler + +Configured in a Cosmos SDK chain's `app.go` file, the `AnteHandler` runs first for every transaction. It verifies the transaction's signature, increments the sequence number, deducts the fee, and meters the gas. {/* todo: link to baseapp#antehandler */} + +If the AnteHandler fails, the transaction aborts and its messages do not execute. + +#### Step 2: Message routing and execution + +Each message in a transaction is routed via BaseApp's `MsgServiceRouter` {/* todo: link to baseapp#msgservicerouter */} to the appropriate module's protobuf `Msg` service. Messages are module-specific and typically defined in a module's `tx.proto`. BaseApp routes these messages to the module's registered protobuf `Msg` service handler, which calls the module's `MsgServer` implementation. + +The `MsgServer` contains the execution logic for that message type. It validates the message content, applies business rules, and updates state. State is read and written through the module's keeper, which manages access to the module's KV store and encapsulates its storage keys. {/* todo: link to msgserver and keeper sections */} + +Messages execute sequentially in the order they appear in the transaction. + +#### Step 3: Atomicity + +Transactions are atomic: all messages succeed or none are committed. + +``` +Tx + ├─ Msg 1 + ├─ Msg 2 + └─ Msg 3 +``` + +If any message fails, all state changes from that transaction are discarded and the transaction returns an error. The next transaction in the block is then executed. BaseApp uses cached stores internally to implement this. + +{/* todo: add section linking to unordered tx */} + +### EndBlock + +EndBlock runs after all transactions in the block have executed. It is used for logic that depends on the block's cumulative state, like tallying governance votes after all vote transactions have been processed, or recalculating validator power after all delegation changes in the block. Modules implement this via the `EndBlock` function in `x//module.go`. + +## Commit + +After `FinalizeBlock` returns, CometBFT calls `Commit`. This persists the state changes to the node's local disk. + +## Deterministic execution + +Across all validators, the block execution is deterministic. Blocks must contain the same ordered transactions, and transactions must use canonical protobuf binary encoding. State transitions must be deterministic, which ensures that every validator computes the same app hash during `FinalizeBlock`, which guarantees consensus safety. If validators holding more than 1/3 of voting power disagree on the app hash, consensus halts. + +## Complete lifecycle overview + +```go +CometBFT + ↓ ABCI InitChain +BaseApp → x//InitGenesis + +For each submitted transaction (async): + ↓ ABCI CheckTx + → decode, verify, validate + → insert into mempool + +For every block: + ↓ ABCI PrepareProposal (proposer only) + → select txs from mempool (MaxTxBytes, MaxGas) + → return tx list to CometBFT + ↓ ABCI ProcessProposal (all validators) + → verify txs, check gas limit + → ACCEPT or REJECT + ↓ ABCI FinalizeBlock + → PreBlock + → x//BeginBlock + For each tx (in the block): + → AnteHandler + → Message routing + → Message execution (atomic) + → x//EndBlock + ↓ ABCI Commit +BaseApp commits KVStores +``` + + +The hooks that run at each phase (the AnteHandler, BeginBlocker, EndBlocker, and InitChainer) are registered in your chain's `app.go` before any block executes. `app.go` is the configuration layer that wires modules into BaseApp. Both are covered in detail in later sections. +{/* todo: add links to app.go and BaseApp sections */} + + +CometBFT drives block processing through ABCI. BaseApp implements ABCI and orchestrates execution. + +- Transactions are validated in `CheckTx` before entering the mempool +- `PrepareProposal` runs on the proposer to build the final tx set for the block +- `ProcessProposal` runs on all validators to accept or reject the proposed block +- Each block is executed inside a single `FinalizeBlock` call +- Within `FinalizeBlock`: PreBlock → BeginBlock → transactions → EndBlock +- Each transaction runs through AnteHandler → message routing → message execution +- Transactions are atomic: all messages commit or none do +- `FinalizeBlock` computes and returns the app hash; `Commit` persists state to disk + +Protobuf ensures canonical encoding so all validators interpret transactions identically. diff --git a/sdk/v0.53/learn/concepts/modules.mdx b/sdk/v0.53/learn/concepts/modules.mdx new file mode 100644 index 00000000..eb107643 --- /dev/null +++ b/sdk/v0.53/learn/concepts/modules.mdx @@ -0,0 +1,298 @@ +--- +title: Intro to Modules +--- + +In the previous section, you saw how blocks and transactions are processed. But where does the actual application logic of a blockchain live? + +In the Cosmos SDK, **modules** define that logic. + +Modules are the fundamental building blocks of a Cosmos SDK application. Each module encapsulates a specific piece of functionality, such as accounts, token transfers, validator management, governance, or any custom logic you define. + +## Why modules exist + +A blockchain application needs to manage many independent concerns (accounts, balances, validator management, etc). Instead of placing all logic in a single monolithic state machine, the Cosmos SDK divides the application into modules. The SDK provides a base layer that allows these modules to operate together as a cohesive blockchain. + +Each module owns a slice of state, defines its messages and queries, implements its business rules, and hooks into the block lifecycle and genesis as needed. This keeps the application organized, composable, and easier to reason about. It also separates safety concerns between modules, creating a more secure system. + +## What a module defines + +A module is a self-contained unit of state and logic. + +At a high level, a module defines: + +- [State](#state): a kv store namespace that contains the module's data +- [Messages](#messages): the actions the module allows +- [Queries](#queries): read-only access to the module's state +- [`MsgServer`](#message-execution-msgserver): validates, applies business logic, and delegates to the keeper +- [Keeper](#keeper): the state access layer — the only sanctioned path to the store + +### State + +State is the data persisted on the chain: everything that transactions can read from or write to. When a transaction is executed, modules apply state transitions or deterministic updates to this stored data. + +Each module owns its own part of the blockchain state. For example: + +- The `x/auth` module stores account metadata. +- The `x/bank` module stores account balances. + +Modules do not share storage directly. Each module has its own key-value store namespace located in a `multistore`. For example, the bank module's store might contain entries like: + +``` +// x/bank store (conceptual) +balances | cosmos1abc...xyz | uatom → 1000000 +balances | cosmos1def...uvw | uatom → 500000 +``` + +The key encodes the namespace, address, and denomination. The value is the encoded amount. No other module can read or write these entries directly, only the [module's keeper](#keeper) can. + +Most state is updated by user-submitted transactions. Some state, called **params**, holds module configuration values. Params are stored in the module's KV store like any other state, but are updated through a privileged message (`MsgUpdateParams`) that only the designated `authority` address can execute. + +### Messages + +As you learned in the [Transactions, Messages, and Queries](/sdk/next/learn/concepts/transactions) section, each module defines the actions it allows via messages (`sdk.Msg`). + +Messages are defined in the module's `tx.proto` file and implemented by that module's `MsgServer`. Here is a simplified example of a message definition from the bank module: + +``` +message MsgSend { + string from_address = 1; + string to_address = 2; + repeated cosmos.base.v1beta1.Coin amount = 3; +} +``` + +This message represents a request to transfer tokens from one account to another. +When included in a transaction and executed: + +1. The sender's balance is checked. +2. The amount is deducted from `from_address`. +3. The amount is credited to `to_address`. + +When a transaction is executed, the SDK's `BaseApp` struct routes each message to the module that defines it. + +### Queries + +Modules expose read-only access to their state through query services, defined in `query.proto`. Queries do not modify state and do not go through block execution. For example: + +``` +// query.proto +rpc Balance(QueryBalanceRequest) returns (QueryBalanceResponse); +``` + +In this case, the caller provides an address and denomination; the query reads the balance from the x/bank keeper and returns it without modifying state. + +### Business logic + +While messages define intent, the `MsgServer` and `Keeper` work together to execute that intent and apply state transitions to the module. + +Business logic is conceptually split across two layers: + +- The **`MsgServer`** handles the transaction-facing logic: it validates inputs, applies message-level business rules, and where required checks authorization. Once satisfied, it delegates state transitions to the keeper. +- The **`Keeper`** is the module's state access layer: it defines the storage schema and provides the sanctioned APIs for reading and writing state. Rather than thinking of the keeper as "just storage," it is more accurate to view it as the module's domain API — it owns how state changes. + +--- + +### Message execution (`MsgServer`) + +Each module implements a `MsgServer`, which is invoked by `BaseApp`'s message router when a message is routed to that module. + +The `MsgServer` is responsible for: + +- Checking authorization when required +- Delegating to keeper methods that validate inputs, enforce business rules, and perform state transitions +- Returning a response + +{/* todo: link to Build a Module tutorial */} +The following examples are from the `x/counter` tutorial, which walks you through building a module that lets accounts increment a shared counter. + +The following is the counter module's `Add` handler which is the `MsgServer` method that processes a request to increment the counter: + +```go +func (m msgServer) Add(ctx context.Context, request *types.MsgAddRequest) (*types.MsgAddResponse, error) { + newCount, err := m.AddCount(ctx, request.GetSender(), request.GetAdd()) + if err != nil { + return nil, err + } + + return &types.MsgAddResponse{UpdatedCount: newCount}, nil +} +``` + +In this example, `Add` is permissionless; any account can call it. All validation (overflow check, `MaxAddValue` limit), fee charging, state mutation, event emission, and telemetry live in the keeper's `AddCount` method. This keeps `msg_server.go` focused on message routing and authorization, while the keeper's named methods can be reused from other entry points such as `BeginBlock` or governance proposals. + +For privileged messages like `MsgUpdateParams`, the `MsgServer` checks the caller against the stored authority before proceeding (`x/counter`): {/* todo: link to counter tutorial */} + +```go +func (m msgServer) UpdateParams(ctx context.Context, msg *types.MsgUpdateParams) (*types.MsgUpdateParamsResponse, error) { + if m.authority != msg.Authority { + return nil, sdkerrors.Wrapf( + govtypes.ErrInvalidSigner, + "invalid authority; expected %s, got %s", + m.authority, + msg.Authority, + ) + } + + if err := m.SetParams(ctx, msg.Params); err != nil { + return nil, err + } + + return &types.MsgUpdateParamsResponse{}, nil +} +``` + +## Keeper + +A module's **keeper** is its state access layer. It owns the module's KV store and provides typed methods for reading and writing state. The store fields are unexported, so nothing outside the `keeper` package can access them directly. + +The `MsgServer` and `QueryServer` both embed the keeper. The `MsgServer` handles authorization; the keeper handles everything else: input validation, business rules, fee charging, state transitions, and event emission. Because all business logic runs through keeper methods, the same rules apply whether the caller is a message handler, a block hook, or a governance proposal. + +The `x/counter` keeper holds two state items and two external dependencies: {/* todo: link to counter tutorial */} + +```go +type Keeper struct { + Schema collections.Schema + counter collections.Item[uint64] // the counter value + params collections.Item[types.Params] // max add value, fee per increment + bankKeeper types.BankKeeper // used to charge fees on increment + + // authority is the address capable of executing a MsgUpdateParams message. + // Typically, this should be the x/gov module account. + authority string +} + +// GetCount returns the current counter value. +func (k *Keeper) GetCount(ctx context.Context) (uint64, error) { + count, err := k.counter.Get(ctx) + if err != nil && !errors.Is(err, collections.ErrNotFound) { + return 0, err + } + return count, nil +} +``` + +`counter` and `params` use `collections.Item`, a typed single-value entry backed by the module's KV store. `GetCount` treats `ErrNotFound` as zero, so the counter starts at 0 without an explicit genesis initialization. + +The `authority` field holds the address allowed to call privileged messages like `MsgUpdateParams`, typically the governance module account. The `MsgServer` checks this before executing those messages. + +### Inter-module access + +Modules are isolated by default. Each module owns its state, and direct access to that state is restricted through the module's keeper. Other modules cannot arbitrarily mutate another module's storage. + +Instead, modules interact through explicitly defined keeper interfaces. `x/counter` uses this pattern to call into `x/bank`. {/* todo: link to counter tutorial */} + +For example: + +- The staking module calls methods on the bank keeper to transfer tokens. +- The governance module calls parameter update methods on other modules. + +Each module defines an `expected_keepers.go` file that declares the interfaces it requires from other modules. This makes cross-module dependencies explicit: a module can only call methods the other module has chosen to expose. + +This design keeps dependencies auditable and prevents accidental or unsafe cross-module state mutation. + +### Block hooks + +Modules may execute logic at specific points in the block lifecycle by implementing optional hook interfaces: + +- `HasBeginBlocker` — runs logic at the start of each block +- `HasEndBlocker` — runs logic at the end of each block +- `HasPreBlocker` — runs logic before BeginBlock, used for consensus parameter changes + +Hooks are optional, and modules should only implement the hooks they need. These hooks are invoked during block execution by a `ModuleManager` in the `BaseApp`, which calls each registered module's hooks in a configured order. {/* todo: link to baseapp#module-manager */} + +{/* todo: link to baseapp#module-manager — ModuleManager coordinates hook invocation order across all modules */} + +### Genesis initialization + +Modules define how their state is initialized when the chain starts. + +Each module implements: + +- `DefaultGenesis` — returns the module's default genesis state +- `ValidateGenesis` — validates the genesis state before the chain starts +- `InitGenesis` — writes the genesis state into the module's store at chain start +- `ExportGenesis` — reads the module's current state and serializes it as genesis data + +During `InitChain`, BaseApp calls each module's `InitGenesis` to populate its state from `genesis.json`. + +{/* todo: link to lifecycle page init genesis section */} + +## Built-in and custom modules + +The Cosmos SDK provides many built-in modules. Visit the [List of Modules](/sdk/v0.53/build/modules/modules) page for more information. +Applications can include any subset of these modules and can define entirely new custom modules. + +A Cosmos SDK blockchain is ultimately a collection of modules assembled into a single application. This customization of modules and application logic down to the lowest levels of a chain is what makes the Cosmos sdk so flexible and powerful. + +## Anatomy of a module (high-level) + +Modules live under the `x/` directory of an SDK application: + +``` +x/ +├── auth/ # Accounts and authentication +├── bank/ # Token balances and transfers +├── poa/ # Proof-of-authority validator management +├── gov/ # Governance system +└── mymodule/ # Your custom module +``` + +Each subdirectory under `x/` is a self-contained module. +An application composes multiple modules together to form a complete blockchain. + +Inside a module, you will typically see a structure like this: + +``` +x/mymodule/ +├── keeper/ +│ ├── keeper.go # Keeper struct and state access methods +│ ├── msg_server.go # MsgServer implementation +│ └── query_server.go # QueryServer implementation +├── types/ +│ ├── expected_keepers.go # Interfaces for other modules' keepers +│ ├── keys.go # Store key definitions +│ └── *.pb.go # Generated from proto definitions +└── module.go # AppModule implementation and hook registration +``` + +Proto files live separately at the repository root, not inside `x/`: + +``` +proto/myapp/mymodule/v1/ +├── tx.proto # Message definitions +├── query.proto # Query service definitions +├── state.proto # On-chain state types +└── genesis.proto # Genesis state definition +``` + +- **`keeper/`** + Contains the `Keeper` struct (state access) and implementations of the `MsgServer` and `QueryServer` interfaces. + +- **`types/`** + Defines the module's public types: generated protobuf structs, store keys, and the `expected_keepers.go` interfaces that declare what this module needs from other modules. + +- **`module.go`** + Connects the module to the application and registers genesis handlers, block hooks, and message and query services. + +- **`.proto` files** + Define messages, queries, state schemas, and genesis state. Go code is generated from these files and used throughout the module. + +## Modules in context + +Putting everything together you've learned so far: + +- **Accounts** authorize transactions. +- **Transactions** carry messages and execution constraints. +- **Blocks** order transactions and define commit boundaries. +- **Modules** define the business rules of execution. +- **MsgServer** validates messages and orchestrates state transitions. +- **Keeper** performs controlled reads and writes to module state. +- **State** persists the deterministic result of execution. + +``` +Account → Transaction → Message → Module → MsgServer → Keeper → State +``` + +In the next section, you will look more closely at how module state is stored, how genesis initializes it, and how the application commits deterministic state transitions. + +{/* todo: link to next page */} diff --git a/sdk/v0.53/learn/concepts/sdk-structure.mdx b/sdk/v0.53/learn/concepts/sdk-structure.mdx new file mode 100644 index 00000000..a97f87f5 --- /dev/null +++ b/sdk/v0.53/learn/concepts/sdk-structure.mdx @@ -0,0 +1,151 @@ +--- +title: Intro to SDK Structure +--- + +Before writing a module or chain, it helps to understand how the Cosmos SDK organizes code and how the pieces connect. This page maps the directory structure of an SDK application, explains what lives inside a module, and shows how modules are assembled into a running application. + +## What is an SDK application + +A Cosmos SDK application is a Go binary that implements a deterministic state machine. It runs alongside CometBFT inside a single daemon process. CometBFT drives consensus; the SDK application executes transactions and maintains state. + +Every SDK application is composed of three main elements: + +- **BaseApp**: the execution engine that implements ABCI and orchestrates transaction processing +- **Modules**: self-contained units of business logic, state, messages, and queries +- **`app.go`**: the wiring layer that instantiates BaseApp, registers modules, and configures the application at startup + +These three elements are covered in depth in the next two sections. This page focuses on how they are organized in the codebase. + +## Repository structure + +An SDK application repository follows a conventional layout: + +``` +myapp/ +├── app/ +│ └── app.go # Application wiring: creates BaseApp, registers modules +├── cmd/ +│ └── main.go # Node binary entrypoint +├── x/ +│ ├── mymodule/ # Custom module +│ └── ... +└── proto/ + └── myapp/ + └── mymodule/ + └── v1/ + ├── tx.proto + ├── query.proto + ├── state.proto + └── genesis.proto +``` + +Each directory has a distinct responsibility: + +- **`x/`** contains the modules. Each subdirectory is a separate, self-contained module. Built-in Cosmos SDK modules (`x/auth`, `x/bank`, `x/staking`, etc.) follow the same layout and are imported as Go packages. Your custom modules live alongside them. + +- **`app/`** contains `app.go`, which assembles the application: creating BaseApp, mounting stores, initializing keepers, and registering all modules with the ModuleManager. + +- **`cmd/`** contains the entrypoint for the node binary. It parses command-line flags, reads configuration files, and starts the daemon process that runs both the CometBFT node and the SDK application. + +- **`proto/`** contains the Protobuf definitions for all custom types: messages, queries, state schemas, and genesis. Go code is generated from these files and consumed throughout the module. Proto files live at the repository root, not inside `x/`, so they can be shared across languages and tooling. + +## What lives inside a module + +Each module under `x/` follows a consistent internal layout: + +``` +x/mymodule/ +├── keeper/ # Keeper (state access), MsgServer, QueryServer +├── types/ # Generated proto types, store keys, expected_keepers.go +└── module.go # AppModule: wires the module into the application +``` + +Proto definitions live separately in `proto/`, not inside `x/`. See [Intro to Modules](/sdk/v0.53/learn/concepts/modules#anatomy-of-a-module-high-level) for a complete walkthrough of each file and the role it plays. + +## How modules are assembled into an application + +Modules are assembled through the **ModuleManager** in `app.go`. The ModuleManager holds the full set of registered modules and coordinates their lifecycle hooks — `InitGenesis`, `BeginBlock`, `EndBlock`, and service registration — across the application. Module ordering is configured explicitly in `app.go` and matters: for example, `x/staking` must finalize delegation changes in `EndBlock` before `x/distribution` calculates rewards. {/* cosmos/cosmos-sdk simapp/app.go:522 — app.ModuleManager.SetOrderEndBlockers(...) — explicit end-block ordering configured in simapp, illustrating that module execution order is a deliberate application-level decision */} See [Intro to Modules](/sdk/v0.53/learn/concepts/modules) for a detailed walkthrough of how the ModuleManager works. + +## Where BaseApp fits + +BaseApp sits between CometBFT and the modules: + +``` +CometBFT (consensus) + ↓ ABCI calls +BaseApp + ↓ orchestrates blocks, routes messages +ModuleManager + ↓ dispatches to individual modules +Modules (x/auth, x/bank, x/mymodule, ...) + ↓ read/write +State (KVStores) +``` + +BaseApp implements the ABCI interface that CometBFT calls to drive block execution. When CometBFT calls `FinalizeBlock`, BaseApp runs the block through all its phases — PreBlock, BeginBlock, transactions, EndBlock — and returns the resulting app hash. BaseApp is covered in detail in [BaseApp Overview](/sdk/v0.53/learn/concepts/baseapp). + +## The role of `app.go` + +`app.go` is the single file that defines a specific chain. It is where the application is assembled from its parts: + +1. Create a `BaseApp` instance with the application name, logger, database, and codec. +2. Create a `StoreKey` for each module and mount it to the multistore. +3. Instantiate each `Keeper`, passing in the codec, store key, and references to other keepers the module depends on. +4. Create the `ModuleManager` with all module instances. +5. Configure execution ordering: which modules run first during genesis, BeginBlock, and EndBlock. +6. Register all gRPC services (message and query handlers) through the ModuleManager. +7. Set the `AnteHandler` and other middleware. + +Because `app.go` is plain Go code, it is fully customizable. A chain includes exactly the modules it needs, wires keepers together as required, and controls the execution order of all lifecycle hooks. There is no framework magic — the assembly is explicit and auditable. + +## Other files in a chain + +A complete SDK chain repository contains more than just `x/`, `app/`, `cmd/`, and `proto/`. Here is what else you will typically find. + +### Additional files in `app/` + +Real-world applications split the `app/` directory across multiple files to keep `app.go` focused on wiring: + +``` +app/ +├── app.go # Main wiring: BaseApp, keepers, module registration +├── export.go # Exports current state as a genesis file (hard forks, snapshots) +├── upgrades.go # Upgrade handlers for consensus-breaking software changes +└── genesis.go # Helpers for genesis state initialization (optional) +``` + +- **`export.go`** — Implements `ExportAppStateAndValidators`, which serializes all module state into a `genesis.json`. {/* cosmos/cosmos-sdk simapp/export.go:21 — func (app *SimApp) ExportAppStateAndValidators(forZeroHeight bool, jailAllowedAddrs, modulesToExport []string) — serializes all module state into the exported genesis format */} This is used when migrating to a new chain version (hard fork) or creating a testnet from a live chain snapshot. +- **`upgrades.go`** — Registers named upgrade handlers consumed by the `x/upgrade` module. Each handler runs exactly once, at the block where the governance-approved upgrade height is reached, and performs any necessary state migrations. {/* cosmos/cosmos-sdk x/upgrade/keeper/keeper.go:518 — k.setDone(ctx, plan.Name) — marks the upgrade plan as complete after handler execution, preventing re-execution */} + +### At the repository root + +``` +myapp/ +├── go.mod # Go module definition: SDK version and all dependencies +├── go.sum # Cryptographic checksums for all dependencies +├── Makefile # Build, test, and codegen tasks +└── scripts/ # Automation scripts (proto generation, linting) +``` + +- **`go.mod` / `go.sum`** — Standard Go module files. `go.mod` declares the Cosmos SDK version and all other imported packages. `go.sum` provides verifiable checksums for the full dependency tree. +- **`Makefile`** — The standard entry point for development tasks: `make build` compiles the binary, `make test` runs unit tests, `make proto-gen` regenerates Go code from `.proto` files. Most SDK chains include targets for linting, simulation tests, and Docker builds. +- **`scripts/`** — Shell scripts and configuration for tooling that the Makefile invokes — typically `buf` configuration for protobuf compilation, scripts for checking dependency consistency, and helpers for running local testnets. + +### The node binary (`cmd/`) + +``` +cmd/ +└── myappdaemon/ + ├── main.go # Binary entrypoint + └── root.go # Root Cobra command: subcommands (start, tx, query, keys, ...) +``` + +The `cmd/` directory produces the node daemon binary (e.g., `simd`, `gaiad`, `wasmd`). It uses [Cobra](https://github.com/spf13/cobra) to expose subcommands for starting the node, submitting transactions, querying state, managing keys, and running genesis initialization. The `start` command spins up CometBFT and the SDK application together in a single process. + +## Summary + +An SDK application is a deterministic state machine composed of modules assembled in `app.go`. The codebase follows a conventional layout: modules in `x/`, application wiring in `app/`, the binary entrypoint in `cmd/`, and Protobuf definitions in `proto/`. + +The ModuleManager assembles modules and coordinates their lifecycle hooks across the application. BaseApp provides the ABCI implementation that connects the state machine to CometBFT's consensus engine. + +The next section explains what BaseApp is and how it coordinates transaction execution in detail. diff --git a/sdk/v0.53/learn/concepts/store.mdx b/sdk/v0.53/learn/concepts/store.mdx new file mode 100644 index 00000000..31a0dc9f --- /dev/null +++ b/sdk/v0.53/learn/concepts/store.mdx @@ -0,0 +1,248 @@ +--- +title: State, Storage, and Genesis +--- + +In the previous section, you learned that modules define business logic and that keepers are responsible for reading and writing module state. This page takes a closer look at how state is stored and committed, and how a chain starts with its initial state. + +## What is state? + +State is the persistent data of the blockchain: account balances, delegations, governance proposals, module parameters, and any other data that survives between blocks. When a transaction executes, modules update state. When a block is committed, that updated state becomes the starting point for the next block: + +```go +State₀ + ↓ apply Block 1 +State₁ + ↓ apply Block 2 +State₂ +``` + +## The KVStore model + +At its lowest level, the Cosmos SDK stores state as **key-value pairs**. Both keys and values are byte arrays. {/* cosmos/cosmos-sdk store/types/store.go:302 — KVStore = GKVStore[[]byte]; keys and values are both []byte */} Modules encode structured data into those bytes using Protocol Buffers, and decode them back when reading. {/* cosmos/cosmos-sdk codec/proto_codec.go:49 — ProtoCodec.Marshal encodes using gogoproto.Marshal to []byte */} + +See [Encoding] for details on how modules serialize data into bytes. + +{/* todo: add link to encoding page */} +The following example shows a conceptual example of how the bank module stores balances: + +```go +key: 0x2 | len(address) | address_bytes | denom_bytes +value: ProtocolBuffer(amount) + +# example +key: 0x2 | 20 | cosmos1abc...xyz | uatom +value: ProtocolBuffer(1000000) +``` + +The key encodes the store prefix, address length, address, and denomination. {/* cosmos/cosmos-sdk x/bank/types/keys.go:29 — BalancesPrefix = collections.NewPrefix(2); prefix byte 0x02 followed by address and denom */} The value is a Protocol Buffer-encoded amount. {/* cosmos/cosmos-sdk types/collections.go:151 — intValueCodec.Encode calls value.Marshal() (math.Int proto encoding); x/bank/types/keys.go:42 — BalanceValueCodec uses IntValue, storing just the amount (not a full Coin) */} See [`x/bank/types/keys.go`](https://github.com/cosmos/cosmos-sdk/blob/v0.53.0/x/bank/types/keys.go) for the actual implementation. + +Each module owns its own namespace in the key-value store. {/* cosmos/cosmos-sdk store/types/store.go:416 — KVStoreKey; only the pointer value should ever be used — it functions as a capabilities key, giving each module its own isolated namespace */} + +## Multistore + +Every module has its own KVStore, and all module stores are mounted inside a [multistore](https://github.com/cosmos/cosmos-sdk/blob/v0.53.0/store/rootmulti/store.go) that is committed as a single state root. {/* cosmos/cosmos-sdk store/rootmulti/store.go:154 — MountStoreWithDB mounts each module's store by StoreKey; store/rootmulti/store.go:537 — Commit() returns CommitID{Hash: cInfo.Hash()} as the single root */} + +A module can only read and write to its own store through its keeper. Access is gated by a `StoreKey`, which is a typed capability object registered at app startup. {/* cosmos/cosmos-sdk store/types/store.go:416 — "Only the pointer value should ever be used - it functions as a capabilities key" */} Modules that don't hold the key cannot open the store. + +This isolation follows an object-capabilities model: + +1. Modules cannot directly mutate another module's state +2. Cross-module interaction must go through exposed keeper methods + +When a block finishes executing, the multistore computes a new root hash (the **app hash**) that represents the entire application state. {/* cosmos/cosmos-sdk store/rootmulti/store.go:537 — Commit() returns CommitID{Hash: cInfo.Hash()} — the root hash over all substores */} That hash is returned to CometBFT, included in the block header, and is what makes the chain's state verifiable. {/* cosmos/cosmos-sdk baseapp/abci.go:999 — res.AppHash = app.workingHash() */} {/* todo: link to lifecycle page */} + +## How state is stored (IAVL and commit stores) + +Each module's KVStore is backed by a commit store. {/* cosmos/cosmos-sdk store/iavl/store.go:31 — var _ types.CommitKVStore = (*Store)(nil) confirms iavl.Store implements CommitKVStore */} [See the store spec for more details.](/sdk/v0.53/build/spec/store/store) + +By default, the Cosmos SDK uses [**IAVL**](https://github.com/cosmos/iavl), a versioned AVL Merkle tree. {/* cosmos/cosmos-sdk store/iavl/store.go:64 — tree := iavl.NewMutableTree(...) — explicitly constructs the IAVL tree as the backing store */} + +IAVL gives every read and write of the tree `O(log n)` complexity, meaning the time to read or write a key scales with the height of the tree, not the total number of keys. It also versions state on each block commit, {/* cosmos/cosmos-sdk store/iavl/store.go:134 — Commit() calls st.tree.SaveVersion() to create a new versioned snapshot */} and produces deterministic root hashes that can be used to generate Merkle proofs for light clients. {/* cosmos/cosmos-sdk store/iavl/store.go:402 — getProofFromTree generates Merkle existence/non-existence proofs from the IAVL tree */} + +Each block commit produces a new tree version with a new root hash: + +```text +Block 1 Block 2 Block 3 + + [root h1] [root h2] [root h3] + / \ / \ / \ + [branch1] [branch2] [branch1] [branch2'] [branch1] [branch2''] + / \ / \ / \ / \ / \ / \ + [a] [b] [c] [d] [a] [b] [c] [d'] [a] [b] [c'] [d'] + ↑ ↑ + (updated) (updated) + +// branch1 and branch2 are internal nodes (they store hashes, not data). +// Leaf nodes (a, b, c, d) are actual key-value entries. +// When a leaf changes, only nodes on the path to the root are rewritten (marked '). +// Unmodified subtrees (branch1, a, b) are shared across all three versions. +``` + +Only modified nodes are rewritten, and unchanged nodes are shared across versions. The root hash changes any time any leaf changes. All validators must compute the same root hash. If they disagree, consensus halts. + +Because of this, state transitions must be deterministic, encoding must be deterministic, and transaction ordering must be consistent. + +### Database backend + +The IAVL tree does not store data in memory. It writes versioned nodes to a **database backend** — a key-value store on disk. {/* cosmos/cosmos-sdk store/iavl/store.go:64 — NewMutableTree receives a db dbm.DB parameter that is the backing database */} + +The Cosmos SDK uses [CometBFT's `db` package](https://github.com/cometbft/cometbft-db) to abstract over the database implementation. The default backend is **PebbleDB**, a high-performance LSM-tree store. Other supported backends include **RocksDB** and **memDB** (in-memory, for testing). {/* cosmos/cosmos-sdk server/start.go — openDB selects the backend based on app.toml's db_backend config value */} + +The full storage stack from top to bottom is: + +``` +Module keeper + ↓ +KVStore (namespaced, wrapped with gas/trace) + ↓ +CommitMultiStore (multistore, computes app hash) + ↓ +IAVL tree (versioned Merkle tree) + ↓ +Database backend (PebbleDB on disk) +``` + +The database backend is selected at node startup and configured in `app.toml`. Application code never interacts with it directly; the store layer owns that boundary. + +## Store types in the SDK + +Beyond the base KVStore, the SDK provides several specialized store wrappers. + +- [CommitKVStore](#commitkvstore-persistent-store) +- [CacheMultiStore](#cachemultistore-transaction-isolation) +- [Transient store](#transient-store-ephemeral-per-block) +- [GasKVStore](#gaskvstore) +- [TraceKVStore](#tracekvstore) + +### CommitKVStore (persistent store) + +This is the main persistent store backed by IAVL. {/* cosmos/cosmos-sdk store/iavl/store.go:31 — var _ types.CommitKVStore = (*Store)(nil) */} It persists across blocks, produces versioned commits, {/* cosmos/cosmos-sdk store/iavl/store.go:134 — Commit() calls st.tree.SaveVersion() returning a new version+hash */} and contributes to the app hash. {/* cosmos/cosmos-sdk store/rootmulti/store.go:537 — root Commit() aggregates all substore hashes into CommitID.Hash */} + +### CacheMultiStore (transaction isolation) + +Before executing each transaction, the Cosmos SDK's BaseApp struct creates a cached, copy-on-write view of the multistore. {/* cosmos/cosmos-sdk baseapp/baseapp.go:662 — cacheTxContext() "returns a new context based off of the provided context with a branched multi-store"; calls ms.CacheMultiStore() */} + +All writes during that transaction occur in this cached layer: + +``` +Multistore + ↓ CacheWrap (per transaction) + ↓ Execute tx + → Success → commit changes + → Failure → discard +``` + +- If the transaction succeeds, changes are written to the underlying store. {/* cosmos/cosmos-sdk store/cachemulti/store.go:128 — Write() iterates all substores and calls store.Write() on each, flushing to the parent */} +- If the transaction fails, the cache is discarded and no state changes are committed. {/* cosmos/cosmos-sdk baseapp/baseapp.go:994 — msCache.Write() is only called inside `if err == nil`; on error the CacheMultiStore is abandoned */} + + +This is how transaction atomicity is implemented in the store layer. + +### Transient store (ephemeral per block) + +The SDK also supports **transient stores** which are cleared at the end of each block when `Commit()` is called, making them empty for the next block. {/* cosmos/cosmos-sdk store/transient/store.go:62 — Commit() calls ts.Clear(), discarding all data and returning an empty CommitID */} Transient stores are used for temporary per-block data such as counters or intermediate calculations and they do not affect the app hash. + +### Gas and trace store wrappers + +All store accesses are wrapped with additional behavior by the `GasKVStore` and `TraceKVStore` wrappers. {/* cosmos/cosmos-sdk store/gaskv/store.go:29 — "GStore applies gas tracking to an underlying KVStore"; store/tracekv/store.go:21 — "Store implements the KVStore interface with tracing enabled" */} + +- **GasKVStore**: charges gas for each read and write {/* cosmos/cosmos-sdk store/gaskv/store.go:64 — Get consumes ReadCostFlat + ReadCostPerByte; store/gaskv/store.go:76 — Set consumes WriteCostFlat + WriteCostPerByte */} +- **TraceKVStore**: logs each store operation for debugging {/* cosmos/cosmos-sdk store/tracekv/store.go:21 — "Operations are traced on each core KVStore call and written to the underlying io.Writer" */} + +Gas is consumed at the store layer, meaning every read and write of a KVStore costs gas, and expensive operations naturally cost more. {/* cosmos/cosmos-sdk store/gaskv/store.go:66 — gs.gasMeter.ConsumeGas(gs.gasConfig.ReadCostFlat, ...) on every Get call */} + +{/* todo: link to gas */} + +### Prefix store + +A **prefix store** wraps a KVStore and automatically prepends a fixed byte prefix to every key. {/* cosmos/cosmos-sdk store/prefix/store.go:74 — key() returns cloneAppend(s.prefix, key), prepending the prefix to every key */} This lets keepers scope their reads and writes to a sub-namespace without manually constructing prefixed keys on every call. + +```go +prefixStore := prefix.NewStore(kvStore, types.KeyPrefix("balances")) +prefixStore.Set(key, value) // stored as "balances" + key +``` + +This is how modules avoid key collisions within their own store. + +## Collections API (typed state access) + +In the Cosmos SDK, modules commonly use the collections API to define typed state access. + +Instead of manually constructing byte keys, modules define typed collections such as: + +- `collections.Item[T]` +- `collections.Map[K, V]` +- `collections.Sequence` + +Example: + +```go +// declared in the keeper struct +Counter collections.Item[uint64] + +// read and write in message handlers +count, _ := k.Counter.Get(ctx) +k.Counter.Set(ctx, count+1) +``` + +The Collections API defines the storage schema, handles encoding and decoding, ensures consistent key construction, and makes state access type-safe. {/* cosmos/cosmos-sdk collections/collections.go:82 — Collection interface defines GetName(), GetPrefix(), ValueCodec() and genesis import/export methods */} + +Under the hood, collections still store data in a KVStore. {/* cosmos/cosmos-sdk collections/map.go:20 — Map struct holds sa func(context.Context) store.KVStore as its backing store accessor */} Collections are used to provide a safer abstraction over raw byte keys. See [`collections/collections.go`](https://github.com/cosmos/cosmos-sdk/blob/v0.53.0/collections/collections.go) for the base interface definitions. + +## How modules access state + +Modules typically do not read from or write to the multi-store directly. Instead, each module defines a keeper that manages access to the module’s portion of state. + +{/* todo: link to keeper in modules page */} + +A keeper typically holds: + +- the module’s **store key** (an object-capability used to open the module’s `KVStore` from `Context`), {/* cosmos/cosmos-sdk store/types/store.go:416 — KVStoreKey; "only the pointer value should ever be used - it functions as a capabilities key" */} +- a **Protocol Buffers codec** used to encode and decode values stored as bytes, {/* cosmos/cosmos-sdk codec/proto_codec.go:49 — ProtoCodec implements BinaryMarshaler using gogoproto.Marshal/Unmarshal */} +- **references (interfaces) to other keepers** the module depends on. + +State access typically flows through the keeper: + +```go +MsgServer / QueryServer + ↓ + Keeper + ↓ + KVStore +``` + +The keeper exposes high-level methods that construct keys, encode values, and enforce business logic: + +```go +func (k Keeper) GetBalance(ctx sdk.Context, addr sdk.AccAddress) sdk.Coins +func (k Keeper) SetParams(ctx sdk.Context, params types.Params) +``` + +## Genesis and chain initialization + +Before the first block executes, the chain must start with an initial state called **genesis**, defined in `genesis.json`. {/* cosmos/cosmos-sdk types/module/module.go:491 — ctx.Logger().Info("initializing blockchain state from genesis.json") */} Genesis is the first write to the KVStores — it is how every module's state exists before any transaction runs. + +During `InitChain`, BaseApp calls each module's `InitGenesis` to populate its store: {/* cosmos/cosmos-sdk baseapp/abci.go:124 — app.abciHandlers.InitChainer(finalizeState.Context(), req) invokes the module manager's InitGenesis; types/module/module.go:486 — InitGenesis iterates m.OrderInitGenesis calling each module */} + +``` +genesis.json + ↓ +BaseApp.InitChain + ↓ +Module.InitGenesis + ↓ +KVStores populated +``` + +For information on how modules define their genesis methods (`DefaultGenesis`, `ValidateGenesis`, `InitGenesis`, `ExportGenesis`) {/* cosmos/cosmos-sdk types/module/module.go:77 — DefaultGenesis; types/module/module.go:78 — ValidateGenesis; types/module/module.go:194 — ExportGenesis */} and initialization ordering, {/* cosmos/cosmos-sdk types/module/module.go:357 — SetOrderInitGenesis sets the call order for InitGenesis across modules */} see [modules](./modules.mdx) and the [lifecycle page](./lifecycle.mdx). + +{/* todo: add correct links to modules and lifecycle pages */} + + +## Next steps + +For more information on stores, pruning strategies, and store configuration, see the [store spec](/sdk/v0.53/build/spec/store/store). For the full store interface definitions, see [`store/types/store.go`](https://github.com/cosmos/cosmos-sdk/blob/v0.53.0/store/types/store.go) in the SDK source. + +After learning about state, storage, and initialization, the next section explains how data is serialized and encoded so that every validator interprets state identically. + +{/* todo: link to next page */} \ No newline at end of file