Drive accepts invalid Identity create transition in mempool (chainlock race condition)

[pshenmic]

Expected Behavior

Drive should throw a error and do not add any invalid platform state transitions in the mempool.

Current Behavior

There is a race condition in identity registration state with chain lock asset lock proof state transitions. When you receive the block's chain lock event (by DAPI or ZMQ), it is not yet fully processed in the platform, and if user tries to create a identity create state transition, drive returns a error:

Error: Asset Lock proof core chain height 1459620 is higher than the current consensus core height 1459619.

This occurs because you receive chainlock event almost in the same time as the platform nodes, and its just not enough time for platform quorum to finish processing, the platform has to produce a new block first that will lock to that new coreChainLockHeight.

Usually, a small timeout solves that issue, however such transactions successfully propogate to the Tenderdash nodes, and even when platform receives a proper chain lock height lock, it fails to execute on rebroadcast, the next tries on DAPI broadcastStateTransition returns an Internal Server Error:

tx%20already%20exists%20in%20cache

Possible Solution

Drive should correctly reject such transaction and do not add that transaction to mempool.

Steps to Reproduce (for bugs)

1. Subscribe to chainlock events
2. Create an assetlock transaction
3. Wait for an chainlock event
4. Create an identity registration state transition with chainlock asset lock proof
5. Try to broadcast it through the network
6. Transaction fails to broadcast because platform is not yet processed and locked to that chainlock height
7. Transaction fails to broadcast on that node, even when platform actually process that chain lock height

Context

Your Environment

• Version used:
• Environment name and version (e.g. Chrome 39, node.js 5.4):
• Operating System and version (desktop, server, or mobile):
• Link to your project:

[lklimek]

Affected transaction: https://testnet.platform-explorer.com/transaction/1748C9998675C431470E02DA6D37224E1F72EA0DE563F268A8EC93EB206634AF

It passes check tx -> gets into proposal -> rejected by drive -> cannot resend the same tx as it's already in the tenderdash cache.

[Claudius-Maginificent]

Did some code-level digging on this. Posting findings — solutioning is left to whoever picks this up.

What's happening

The error originates from ChainAssetLockProof::validate at packages/rs-drive-abci/src/execution/validation/state_transition/common/asset_lock/proof/validate/chain/mod.rs:30-40. It compares the proof's core_chain_locked_height against PlatformState.last_committed_block_info.core_height — i.e. the Platform-side height of the last committed Platform block, not the latest core-side chainlock.

last_committed_block_info is updated post-commit only (packages/rs-drive-abci/src/execution/platform_events/state_transition_processing/block_end/update_state_cache/v0/mod.rs:50). So "current consensus core height" really means "the core height the Platform has already finalized into a committed block."

Why check_tx accepts but proposal rejects

Both check_tx and process_proposal run this check (the same transform_into_action → ChainAssetLockProof::validate path), but they evaluate against different node-local state:

• check_tx: packages/rs-drive-abci/src/execution/validation/state_transition/check_tx_verification/v0/mod.rs calls transform_into_action(... ValidationMode::CheckTx ...) against the local platform.state.load().
• process_proposal: packages/rs-drive-abci/src/abci/handler/process_proposal.rs:236 → run_block_proposal → process_raw_state_transitions_v0 builds PlatformRef from block_platform_state (= previous-block state + in-flight update_core_info, which only updates masternode/quorum lists; it does NOT advance last_committed_block_info). Then runs transform_into_action(... ValidationMode::Validator ...).

In both paths last_committed_core_height reads from the same last_committed_block_info, so single-node behavior is consistent. The visible-asymmetric-behavior the issue describes is most plausibly across nodes:

1. Core emits chainlock for height H. Client + several Platform nodes receive it ~simultaneously.
2. The DAPI node serving broadcastStateTransition has already finalized a Platform block embedding this chainlock (locally last_committed_core_height = H). check_tx here accepts the tx.
3. Tenderdash gossips the tx to other nodes' mempools. On those, the previous Platform block was the most recent commit and last_committed_core_height = H-1.
4. A lagging node becomes the proposer (or runs process_proposal as a validator). Its block_platform_state.last_committed_core_height() == H-1 < H. Validation fires InvalidAssetLockProofCoreChainHeightError (consensus error code 10506, packages/rs-dpp/src/errors/consensus/basic/identity/invalid_asset_lock_proof_core_chain_height_error.rs:13).

The asset-lock validation runs before any fee is owed, so it surfaces as UnpaidConsensusError. process_proposal.rs:287-309 rejects the entire block on any unpaid error (returns ProposalStatus::Reject).

The design assumes last_committed_core_height is monotonic and synchronized across nodes — which only holds after the chainlock-bearing Platform block commits everywhere.

There's also a complementary single-node scenario worth flagging: RecheckTx mode skips this check entirely (chain/mod.rs:30). So a tx that initially passed check_tx stays in the local mempool through subsequent rechecks even if it's no longer valid for the next proposal — only process_proposal re-evaluates with ValidationMode::Validator, where the check fires again.

Why retries fail with "tx already exists in cache"

That string is emitted by Tenderdash itself, not Drive. The flow:

1. process_proposal returns Reject → Tenderdash drops the proposed block but keeps the tx in mempool/cache (fingerprinted by tx hash).
2. The client's retry of the same bytes hits Tenderdash's mempool cache check before reaching Drive's check_tx again, returning the "tx already exists in cache" error.
3. Drive's DAPI layer maps it: packages/rs-dapi/src/services/platform_service/error_mapping.rs:329 → DapiError::AlreadyExists → gRPC ALREADY_EXISTS. SDK-side mitigation lives at packages/rs-sdk/src/internal_cache/mod.rs:215.
4. Drive currently has no mechanism to ask Tenderdash to evict a transient-failure tx. The tx stays cached until it's included in a committed block or expires from the mempool TTL — neither of which happens for a chronically-rejected tx.

So the user-visible flow is: tx admitted to mempool → consistently rejected at proposal across all subsequent rounds → client cannot rebroadcast same bytes (cache hit) → effectively permanent until mempool eviction.

Notable recent context

• c47aed454f (Jan 2024, feat(drive-abci)!: remove unnecessary validation from check tx and re-check #1647 — feat(drive-abci)!: remove unnecessary validation from check tx and re-check) introduced the current ValidationMode split that skips ChainAssetLockProof::validate on RecheckTx.
• fa72ffeea8 (feat(platform)!: identity create replay protection #1819 — identity create replay protection) and 3014b6f524 (feat(platform)!: mitigate issues of asset lock based transitions #1796 — mitigate issues of asset lock based transitions) — adjacent asset-lock validation changes.
• 257d264e21 (feat(dpp): add max_asset_lock_transaction_inputs limit to prevent stuck funds #3491 — add max_asset_lock_transaction_inputs limit) — adjacent structural validation.
• No commits in the last year have touched the chain-height comparison in chain/mod.rs:30-40 itself.

Affected tx (from the comment above): https://testnet.platform-explorer.com/transaction/1748C9998675C431470E02DA6D37224E1F72EA0DE563F268A8EC93EB206634AF — passes check_tx, gets into a proposal, rejected by Drive, cannot resend (Tenderdash cache hit). Matches this analysis.

_{🤖 Co-authored by Claudius the Magnificent AI Agent}