fix(storage): supplementary sled writes + V8 bonding-curve-registry persistence#1936
Open
umwelt wants to merge 6 commits intodevelopmentfrom
Open
fix(storage): supplementary sled writes + V8 bonding-curve-registry persistence#1936umwelt wants to merge 6 commits intodevelopmentfrom
umwelt wants to merge 6 commits intodevelopmentfrom
Conversation
…ck() The function has timestamp 1730419200 (2024-11-01) but the comment said "November 1, 2025". Clarify that this function is test-only and that production genesis is built via GenesisConfig::build_block0() from genesis.toml (2025-11-01T00:00:00Z = 1761955200).
CBE_DECIMALS=8 implies 1 CBE = 10^8 atoms, but the supply constants were stored as whole tokens (100_000_000_000 for 100B CBE). Any wallet or exchange reading decimals() would display 1,000 CBE instead of 100 billion. Fix by aligning with SOV convention — all balances and supply constants are now in atomic units: CBE_TOTAL_SUPPLY = 100_000_000_000 * 10^8 (10^19 atoms) CBE_COMPENSATION_POOL = 40_000_000_000 * 10^8 CBE_OPERATIONAL_TREASURY = 30_000_000_000 * 10^8 CBE_PERFORMANCE_INCENTIVES= 20_000_000_000 * 10^8 CBE_STRATEGIC_RESERVES = 10_000_000_000 * 10^8 Add CBE_ATOMS_PER_TOKEN=100_000_000 constant. Replace all hardcoded vesting amounts in genesis/mod.rs and blockchain.rs with the named constants. Update all tests; fix overflow in percentage assertions by dividing before multiplying.
SOV has no pre-mint at genesis — supply grows exclusively through UBI payouts. The field was parsed from genesis.toml but never consumed anywhere in the GenesisConfig codepath. Remove both the struct and the TOML section to prevent future confusion about whether an initial allocation is possible.
Add --sled-dir option to genesis export-state so it can read directly from the SledStore used by production nodes (which never write .dat files). The Sled path calls load_from_store() which replays all blocks and syncs SOV balances from the token_balances tree — the same code path the node uses on restart, guaranteeing a complete and accurate state snapshot. Also includes a warning that Sled does not support concurrent access: the node must be stopped (or a directory copy used) before running export.
…ed load TokenMint and TokenTransfer transactions with fee!=0 pass signature validation but BlockExecutor rejects them at execution time with "fee must be 0 in Phase 2", causing every proposed block to fail and consensus to stall indefinitely. Two-part fix: 1. verify_and_enqueue_transaction: reject TokenMint/TokenTransfer with fee!=0 at mempool intake so they never enter pending_transactions. 2. load_from_store: call evict_phase2_invalid_transactions on startup to clear any that accumulated before the intake check was in place.
…ersistence
Three production fixes deployed together:
1. No-active-block-transaction halt (heights 3474, 3604)
- Add begin/commit/rollback_supplementary_writes to BlockchainStore trait
- SledStore: supplementary batch opens without height validation, commits
identity/wallet trees without updating LATEST_HEIGHT
- finish_block_processing: when executor is active, open supplementary batch
instead of calling begin_block (which would fail with InvalidBlockHeight)
2. Oracle cbe_usd_price always None after restart
- BlockchainStorageV8: add bonding_curve_registry to persisted state
- FILE_VERSION bumped 7→8; V7 files migrate transparently
- Post-load backfill re-registers CBE curve entry when registry is empty
(handles V7→V8 migration and load_from_store path)
3. Late-finalization: commit quorum for past round ignored
- process_committed_block now takes explicit round parameter
- Late commit votes for H=N R=K arriving while engine is at R=K+1
now correctly apply the block instead of being silently dropped
Also: validate non-empty signatures on system transactions, expand block
limits (payload 1MB→4MB, witness 512KB→2MB), update API handlers.
|
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes several production-impacting persistence and consensus-edge issues across the ZHTP node, including post-executor Sled writes, on-disk storage upgrades for bonding curve registry persistence, and correct handling of late commit quorums.
Changes:
- Add a “supplementary writes” Sled batch path for identity/wallet side-data that must be written after the BlockExecutor has already committed the main block transaction.
- Bump
blockchain.datstorage to V8 to persistbonding_curve_registry, with V7→V8 backfill logic to restore the CBE registry entry on load. - Improve consensus finalization by passing an explicit
roundinto committed-block processing to correctly handle late commit quorums.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| zhtp/src/api/handlers/wallet/mod.rs | Adds amount_human formatting to wallet transaction history output based on token decimals. |
| zhtp/src/api/handlers/blockchain/mod.rs | Improves /blockchain tx display by deriving from/to/amount from typed tx fields. |
| zhtp-cli/src/commands/genesis.rs | Extends genesis export-state to load from a SledStore directory when provided. |
| zhtp-cli/src/argument_parsing.rs | Adds CLI flags for genesis export-state to support --sled-dir alongside legacy --dat-file. |
| lib-consensus/src/engines/consensus_engine/state_machine.rs | Fixes late-finalization handling by parameterizing committed-block processing with round. |
| lib-blockchain/src/transaction/validation.rs | Validates non-empty signatures even for system txs to prevent malformed sigs entering the mempool. |
| lib-blockchain/src/storage/sled_store.rs | Implements begin/commit/rollback_supplementary_writes for post-executor side-data persistence. |
| lib-blockchain/src/storage/mod.rs | Extends the BlockchainStore trait with supplementary write-batch APIs. |
| lib-blockchain/src/resources.rs | Increases default block payload/witness/state-write limits. |
| lib-blockchain/src/genesis/mod.rs | Uses CBE allocation constants in genesis build and removes obsolete sov config. |
| lib-blockchain/src/contracts/tokens/cbe_token.rs | Switches CBE constants to atomic units (10^8 atoms per token) and updates tests accordingly. |
| lib-blockchain/src/blockchain.rs | Adds V8 storage struct + load/save support; adds backfills/evictions and supplementary-write handling in post-processing. |
| lib-blockchain/src/block/core.rs | Clarifies genesis timestamp usage and makes the constant explicitly u64. |
| genesis.toml | Removes [sov] section; keeps CBE config and other genesis parameters. |
You can also share your feedback on Copilot code review. Take the survey.
Comment on lines
386
to
390
| bc.cbe_token | ||
| .create_vesting( | ||
| &operational_key, | ||
| 30_000_000_000, | ||
| CBE_OPERATIONAL_TREASURY, | ||
| 0, |
Comment on lines
+1160
to
+1167
| fn begin_supplementary_writes(&self) -> StorageResult<()> { | ||
| // No transaction must be active | ||
| if self.tx_active.swap(true, Ordering::SeqCst) { | ||
| return Err(StorageError::TransactionAlreadyActive); | ||
| } | ||
| let mut batch_guard = self.tx_batch.lock().unwrap(); | ||
| *batch_guard = Some(PendingBatch::new()); | ||
| Ok(()) |
Comment on lines
+208
to
+210
| /// Raw atomic amount (1 SOV = 100_000_000 atomic units). Use `amount_human` for display. | ||
| amount: u64, | ||
| /// Human-readable amount in SOV (atomic / 100_000_000). Use this for display. |
| .map(|d| hex::encode(d.to)) | ||
| .or_else(|| tx.token_mint_data.as_ref().map(|d| hex::encode(d.to))); | ||
|
|
||
| let divisor = 10u64.pow(token_decimals as u32) as f64; |
Comment on lines
+1205
to
+1209
| .get(&tid) | ||
| .map(|c| c.decimals) | ||
| .unwrap_or(18), | ||
| None => 18, | ||
| } |
Comment on lines
+1180
to
+1186
| // Apply identity and wallet side-data only. | ||
| // Do NOT update LATEST_HEIGHT — the executor already did that. | ||
| self.identities | ||
| .apply_batch(batch.identities) | ||
| .map_err(|e| StorageError::Database(e.to_string()))?; | ||
|
|
||
| self.identity_metadata |
Comment on lines
+26
to
+27
| max_payload_bytes: 4_194_304, // 4 MB | ||
| max_witness_bytes: 2_097_152, // 2 MB (matches fee params block_max_witness_bytes) |
Comment on lines
+101
to
+105
| // Derive from/to/amount from the typed transaction data fields | ||
| let (from, to, amount) = if let Some(ref d) = tx.token_transfer_data { | ||
| (hex::encode(d.from), hex::encode(d.to), d.amount as u64) | ||
| } else if let Some(ref d) = tx.token_mint_data { | ||
| ("system".to_string(), hex::encode(d.to), d.amount as u64) |
Comment on lines
+709
to
+711
| // Test-only genesis timestamp (2024-11-01T00:00:00Z = 1730419200). | ||
| // NOTE: This function is used by unit tests only. Production genesis is built | ||
| // via GenesisConfig::build_block0() which reads the timestamp from genesis.toml |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
No-active-block-transaction halt (hit twice in production at heights 3474 and 3604):
finish_block_processingwas called with no active sled transaction when the executor path was active. Addedbegin/commit/rollback_supplementary_writesto open an identity/wallet batch after the executor has already committed the block — without re-validating block height or touchingLATEST_HEIGHT.Oracle
cbe_usd_price: Noneafter every restart:bonding_curve_registrywas never persisted, soget_cbe_curve_price_atomic()always returnedNoneafter load. Bumped storage to V8 which serializes the registry. V7 files migrate transparently via a post-load backfill that re-registers the CBE genesis curve entry.Late-finalization dropped: commit quorum for round K arriving while engine was at round K+1 (same height) was silently ignored.
process_committed_blocknow takes an explicitroundparameter so the quorum check uses the correct round.Validate non-empty signatures on system transactions to prevent malformed sigs bypassing mempool intake.
Expand block limits: payload 1MB→4MB, witness 512KB→2MB.
Test plan
cbe_usd_price: Some(...)in oracle finalization logsNo active block transactionerror at the next block commit📂 Loaded blockchain storage v8on restart after first save