Feat/persistent block archive#4
Merged
Merged
Conversation
redb-backed durable layer that catches blocks past the chain's solidified-head threshold. Same opaque-bytes pattern as CursorStore: the archive doesn't know what payload it carries; the firehose stores pb::Block-encoded bytes there. Schema is single-table u64 → packed [block_id: 32B][payload]. Surface: open / put / put_batch / get / range(start, end, limit) / delete_below(floor) / min_height / max_height / len / is_empty. 13 unit tests including round-trip, persistence-across-reopen, shared-handle-via-clone, retention-pruning, truncation guard. Drops bincode workspace dep + the RUSTSEC-2025-0141 ignore from deny.toml: bincode was declared but never used; the deny ignore existed only on the assumption that lightcycle-store would eventually use it for persistence. The archive uses prost wire bytes via the firehose layer instead. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three pieces wire the archive into the firehose surface: 1. archiver task subscribes to the relayer broadcast and writes every Output::Irreversible to the archive, encoding via the existing encode_block path. Lag-tolerant; the relayer re-emits Irreversible on cold start so any gap recovers. 2. StreamService.with_archive() extends Stream.Blocks backfill: when the consumer's resume height falls below cache.min_height, the archive is walked first and chained into the cache walk. Each archived row is decoded just enough to populate Response.metadata (height, parent_id, time, lib_num); the pb::Block bytes are re-emitted verbatim — no BufferedBlock reconstruction needed. 3. FetchService.with_archive() short-circuits Fetch.Block for archived heights, skipping the upstream RPC entirely. Cache continues to serve the latest blocks; archive serves the older ones; oracle handles cache-miss + non-archived. Backfill rejection paths handle: empty cache without archive, request above cache tip, request below archive floor, mid-walk eviction. 3 new integration tests: - backfill_walks_archive_then_cache_then_live (50→60 archive, 60→63 cache, 63 live) - fetch_block_archive_hit_short_circuits_oracle - backfill_below_archive_floor_returns_failed_precondition Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Operator-facing wiring: - --archive-path PATH (env LIGHTCYCLE_RELAY_ARCHIVE_PATH): when set, opens the redb archive, spawns the archiver task, and threads the archive into Stream.Blocks backfill + Fetch.Block. Unset preserves v0.x in-memory-only behavior. - --archive-retention-blocks N (env LIGHTCYCLE_RELAY_ARCHIVE_RETENTION_BLOCKS): when > 0, retention task ticks every minute, computes floor = current_solidified_head - N, calls archive.delete_below(floor). 0 (default) keeps everything — the right choice for cold-archive deployments. Composite shutdown handle waits on the pump task and aborts the archiver / retention tasks on relay teardown. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hand-rolled tiny HTTP/1.1 server (no new deps). Bound separately from the Prometheus exporter so a misconfigured scrape can't black-hole the readiness signal. Endpoints: - /healthz — always 200 (process is alive iff listener is bound) - /readyz — 200 once the relayer has populated the solidified-head watch with at least one chain-reported height; 503 before then (subsumes "RPC connected" + "first block fetched" + "finality oracle wired" — all three become true together on the first successful tick) Required for kubernetes readiness probes and any orchestrator that gates traffic on health. 6 unit tests including parser correctness, 200-on-/healthz, 503→200 transition on /readyz when head is published, 404 on unknown paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds lightcycle-codec::abi alongside the existing universal TRC-20/721
helpers in events.rs. Operators register Solidity-style event
signatures like:
"Swap(address indexed sender, uint256 amount0In, uint256 amount1In,
uint256 amount0Out, uint256 amount1Out, address indexed to)"
The registry computes topic[0] = keccak256(canonical_signature) and
decodes any matching log into a structured DecodedEvent with
positional-and-named parameter access.
Static-type subset for v0.1: address, bool, uintN/intN (8..=256 bits),
bytesN (1..=32). This covers the long tail of real-world contract
events (DEX swaps, lending accruals, governance votes, etc).
Out of scope, surfaced as typed errors:
- Dynamic types: string, bytes, T[], tuples, structs (parser returns
AbiParseError::UnsupportedType so consumers fail loud)
- Anonymous events (no topic[0] to match against)
Hand-rolled rather than pulling alloy-sol-types: ~300 lines including
parser, registry, and decoder; keeps lightcycle-codec's transitive
deps lean.
18 unit tests including signature parsing across compact/full/messy
forms, width recognition, error paths, decode round-trip on a Uniswap
V2-flavored Swap event with mixed indexed and non-indexed params,
collision/overwrite semantics, and cross-check against the existing
TRC20_TRANSFER_TOPIC0 constant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- health.rs: cargo fmt re-folded a function-call across multiple
lines (was broken by my earlier manual format). Apply the
one-liner the formatter prefers.
- abi.rs: rustdoc -D warnings flagged two intra-doc-link issues:
- "[`crate::events`]" hits the private-intra-doc-links lint
because the events mod is private. Changed to plain text.
- "topic[0]" looked like an unresolved intra-doc-link to `0`.
Wrapped in backticks so rustdoc treats it as code.
Local cargo fmt + RUSTDOCFLAGS=-D warnings cargo doc --workspace
both clean now.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Updates the example invocation to show --archive-path, --archive-retention-blocks, and --health-listen alongside the existing flags. Adds a paragraph describing the healthcheck endpoint behavior and rationale (separate-listener for k8s probe robustness). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Updates the lightcycle-codec section to describe the new abi.rs registry (registration via Solidity-style signature, static-type subset, decode surface) alongside the existing universal TRC-20/721 helpers in events.rs. Updates the lightcycle-firehose Stream.Blocks paragraph to describe the archive-walk path that extends backfill past the in-memory cache window when --archive-path is set. Updates the lightcycle-store section to mark Block cache, Block archive, and Cursor store as landed (was: all three "planned"), with the API and consistency-posture notes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.