feat(interchain-indexer): Improving internal stats module#1650
Merged
feat(interchain-indexer): Improving internal stats module#1650
Conversation
Contributor
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one) (2)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughThis pull request introduces a comprehensive analytics and statistics system for the interchain indexer. It adds a new database schema with six stats tables (stats_assets, stats_asset_tokens, stats_asset_edges, stats_messages, stats_messages_days, and stats_chains) to track cross-chain transfer aggregates, message path statistics, and per-chain user metrics. The changes include SeaORM entity definitions for all stats tables, a new StatsService orchestration layer that coordinates stats projection from finalized messages and transfers, pagination logic for bridged token and chain statistics endpoints, and gRPC service handlers exposing these stats via new API endpoints. Integration points include extending the message buffer and finalization pipeline to trigger stats projection, adding token enrichment hooks to propagate metadata into stats tables, and updating the Avalanche indexer to use StatsService. Configuration options enable stats backfill on startup and periodic background recomputation of per-chain statistics. Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a materialized, incremental statistics pipeline (DB tables + projection/backfill + periodic recomputation) and exposes initial read APIs for interchain analytics (chains catalog, bridged-token stats, chain stats, and message-path views).
Changes:
- Introduces new stats schema (assets/edges/messages/daily rollups/per-chain aggregates) plus incremental projection/backfill and periodic
stats_chainsrecomputation. - Adds new public API endpoints and pagination/sorting logic for stats-driven views (bridged tokens, chain stats, message paths) and a chains catalog endpoint.
- Extends token metadata flow to asynchronously enrich stats tables and propagates fetched token info into stats state.
Reviewed changes
Copilot reviewed 52 out of 53 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| interchain-indexer/justfile | Updates dev run defaults (logging + enable stats backfill on start). |
| interchain-indexer/interchain-indexer-server/tests/avalanche_e2e.rs | Wires StatsService into Avalanche indexer E2E tests. |
| interchain-indexer/interchain-indexer-server/src/settings.rs | Adds stats_backfill_on_start + stats_chains_recalculation_period_secs settings with tests. |
| interchain-indexer/interchain-indexer-server/src/services/stats.rs | Implements new stats gRPC/HTTP handlers (bridged tokens, chain stats, message paths) + parsing helpers. |
| interchain-indexer/interchain-indexer-server/src/services/mod.rs | Registers new chain_info_proto module. |
| interchain-indexer/interchain-indexer-server/src/services/interchain_service.rs | Adds GetChains endpoint and centralizes chain-to-proto conversion. |
| interchain-indexer/interchain-indexer-server/src/services/chain_info_proto.rs | New shared chains::Model → ChainInfo mapping helper. |
| interchain-indexer/interchain-indexer-server/src/server.rs | Wires StatsService, startup backfill, and periodic stats_chains worker; passes stats into indexers and services. |
| interchain-indexer/interchain-indexer-server/src/indexers.rs | Changes indexer spawning to accept Arc<StatsService>. |
| interchain-indexer/interchain-indexer-server/src/config.rs | Removes brittle tests that depended on repo file layout. |
| interchain-indexer/interchain-indexer-server/config/example.toml | Documents new stats-related settings. |
| interchain-indexer/interchain-indexer-proto/swagger/v1/interchain-indexer.swagger.yaml | Adds OpenAPI paths/definitions for new endpoints and pagination DTOs. |
| interchain-indexer/interchain-indexer-proto/proto/v1/stats.proto | Adds stats service RPCs, pagination messages, and sort/order enums. |
| interchain-indexer/interchain-indexer-proto/proto/v1/interchain_indexer.proto | Adds GetChains and message-path request/response messages. |
| interchain-indexer/interchain-indexer-proto/proto/v1/api_config_http.yaml | Adds HTTP mappings for new endpoints. |
| interchain-indexer/interchain-indexer-proto/build.rs | Adds serde defaults for query enum fields and a post-process to dedupe actix-prost duplicate structs. |
| interchain-indexer/interchain-indexer-migration/src/migrations_up/m20260312_175120_add_stats_tables_up.sql | Adds stats tables/types, processing markers, and supporting indexes. |
| interchain-indexer/interchain-indexer-migration/src/migrations_down/m20260312_175120_add_stats_tables_down.sql | Drops stats tables/types/indexes and removes new columns. |
| interchain-indexer/interchain-indexer-migration/src/m20260312_175120_add_stats_tables.rs | Registers SQL-based migration wrapper. |
| interchain-indexer/interchain-indexer-migration/src/lib.rs | Adds the new stats migration to the migrator. |
| interchain-indexer/interchain-indexer-logic/src/token_info/service.rs | Propagates fetched token info into stats tables and adds async enrichment kickoff for stats keys. |
| interchain-indexer/interchain-indexer-logic/src/stats_chains_query.rs | New keyset-paginated query for /stats/chains. |
| interchain-indexer/interchain-indexer-logic/src/stats/service.rs | New orchestration layer for projection/backfill/recompute + read helpers. |
| interchain-indexer/interchain-indexer-logic/src/stats/projection.rs | New transactional batch projection of messages/transfers into stats tables (assets/edges/messages/days). |
| interchain-indexer/interchain-indexer-logic/src/stats/mod.rs | Exposes stats service and types. |
| interchain-indexer/interchain-indexer-logic/src/pagination.rs | Adds stats pagination tokens and sort/order enums. |
| interchain-indexer/interchain-indexer-logic/src/message_buffer/persistence.rs | Adds token-key extraction helper for enrichment. |
| interchain-indexer/interchain-indexer-logic/src/message_buffer/mod.rs | Re-exports finalized token-key helper. |
| interchain-indexer/interchain-indexer-logic/src/message_buffer/maintenance.rs | Runs stats projection in the flush transaction and kicks off token enrichment post-commit. |
| interchain-indexer/interchain-indexer-logic/src/message_buffer/buffer.rs | Refactors buffer to depend on StatsService (and provides constructors for embedders/tests). |
| interchain-indexer/interchain-indexer-logic/src/lib.rs | Registers new stats modules and re-exports pagination/service types. |
| interchain-indexer/interchain-indexer-logic/src/indexer/avalanche/mod.rs | Passes StatsService into buffer and updates blockchain-id resolution to honor process_unknown_chains. |
| interchain-indexer/interchain-indexer-logic/src/indexer/avalanche/consolidation.rs | Initializes stats_processed on new consolidated messages. |
| interchain-indexer/interchain-indexer-logic/src/indexer/avalanche/blockchain_id_resolver.rs | Adds force_add_chain semantics to control whether discovered chains are persisted. |
| interchain-indexer/interchain-indexer-logic/src/chain_info/service.rs | Adds get_all_chains_info() with normalization and DB-backed tests. |
| interchain-indexer/interchain-indexer-logic/src/bridged_tokens_query.rs | New keyset-paginated bridged-token aggregation query + token-link enrichment fetch. |
| interchain-indexer/interchain-indexer-entity/src/codegen/stats_messages_days.rs | Adds SeaORM entity for stats_messages_days. |
| interchain-indexer/interchain-indexer-entity/src/codegen/stats_messages.rs | Adds SeaORM entity for stats_messages. |
| interchain-indexer/interchain-indexer-entity/src/codegen/stats_chains.rs | Adds SeaORM entity for stats_chains. |
| interchain-indexer/interchain-indexer-entity/src/codegen/stats_assets.rs | Adds SeaORM entity for stats_assets. |
| interchain-indexer/interchain-indexer-entity/src/codegen/stats_asset_tokens.rs | Adds SeaORM entity for stats_asset_tokens. |
| interchain-indexer/interchain-indexer-entity/src/codegen/stats_asset_edges.rs | Adds SeaORM entity for stats_asset_edges. |
| interchain-indexer/interchain-indexer-entity/src/codegen/sea_orm_active_enums.rs | Adds EdgeAmountSide active enum. |
| interchain-indexer/interchain-indexer-entity/src/codegen/prelude.rs | Exposes new stats entities in the prelude. |
| interchain-indexer/interchain-indexer-entity/src/codegen/mod.rs | Registers stats entity modules. |
| interchain-indexer/interchain-indexer-entity/src/codegen/crosschain_transfers.rs | Adds stats_processed + stats_asset_id fields and relation to stats_assets. |
| interchain-indexer/interchain-indexer-entity/src/codegen/crosschain_messages.rs | Adds stats_processed field. |
| interchain-indexer/interchain-indexer-entity/src/codegen/chains.rs | Adds relations to stats tables. |
| interchain-indexer/config/avalanche/bridges.json | Sets process_unknown_chains to false for Avalanche config. |
| interchain-indexer/README.md | Documents new stats settings env vars. |
| interchain-indexer/Cargo.toml | Adds the bon dependency and reorders entry. |
| interchain-indexer/.memory-bank/gotchas.md | Documents gotchas for stats edge amount_side semantics and filtering behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
Author
|
@CodeRabbit review |
Contributor
✅ Actions performedReview triggered.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 7
🧹 Nitpick comments (7)
interchain-indexer/interchain-indexer-server/src/settings.rs (1)
135-143: Avoid duplicating the production default in tests.
stats_chains_period_default_for_test()redefines3600, so tests can pass even if production default changes. Reuse the production default function directly to prevent drift.♻️ Suggested tweak
- fn stats_chains_period_default_for_test() -> u64 { - 3600 - } - #[derive(Deserialize)] struct StatsChainsPeriod { - #[serde(default = "stats_chains_period_default_for_test")] + #[serde(default = "super::default_stats_chains_recalculation_period_secs")] stats_chains_recalculation_period_secs: u64, }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@interchain-indexer/interchain-indexer-server/src/settings.rs` around lines 135 - 143, The test-specific function stats_chains_period_default_for_test() redefines the literal 3600 and should be removed; update StatsChainsPeriod to reuse the existing production default function used for stats_chains_recalculation_period_secs (i.e., replace the serde default reference to call the production default function instead of stats_chains_period_default_for_test()), and delete the duplicate stats_chains_period_default_for_test() helper so the test default will automatically follow any production changes.interchain-indexer/interchain-indexer-logic/src/chain_info/service.rs (1)
162-170: Enforce sort order in-service to match the documented contract.
get_all_chains_info()says it returns rows ordered bychains.idascending, but it currently relies on DB behavior only (Line 163). Sorting before return makes this contract stable.🔧 Suggested hardening
pub async fn get_all_chains_info(&self) -> anyhow::Result<Vec<chains::Model>> { let chains = self.db.get_all_chains().await?; let mut out = Vec::with_capacity(chains.len()); for chain in chains { let normalized = normalize_chain(chain); out.push(normalized); } + out.sort_by_key(|c| c.id); Ok(out) }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@interchain-indexer/interchain-indexer-logic/src/chain_info/service.rs` around lines 162 - 170, The get_all_chains_info function currently returns DB rows without enforcing order; update get_all_chains_info to sort the normalized Vec<chains::Model> by chains.id ascending before returning (after calling normalize_chain in the loop or by sorting the original `chains` collection), ensuring the returned ordering matches the documented contract; reference the function name get_all_chains_info, the normalize_chain call, and the db.get_all_chains result when implementing the in-service sort.interchain-indexer/interchain-indexer-logic/src/message_buffer/persistence.rs (1)
1-1: Consolidatestd::collectionsimports.
HashMapis imported on line 1, andHashSetis separately imported on line 12. Per the coding guidelines, use crate-level import grouping.Suggested fix
-use std::collections::HashMap; +use std::collections::{HashMap, HashSet};And remove line 12.
As per coding guidelines: "Use crate-level import grouping with
imports_granularity=Cratesetting".Also applies to: 12-12
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@interchain-indexer/interchain-indexer-logic/src/message_buffer/persistence.rs` at line 1, The file imports from std::collections are split; consolidate them into a single crate-level import by combining HashMap and HashSet into one statement (e.g., use std::collections::{HashMap, HashSet};) and remove the separate HashSet import line; update the top of persistence.rs where HashMap and HashSet are currently imported so both symbols (HashMap, HashSet) are brought in by a single use declaration.interchain-indexer/interchain-indexer-server/src/services/stats.rs (1)
243-254: Consider parallelizing chain info lookups for large result sets.The sequential
awaitcalls forget_chain_infoin the loop could become a bottleneck if the result set is large. For typical message-path queries with a limited number of chain pairs, this is likely acceptable.♻️ Optional: parallelize with futures::future::join_all
use futures::future::join_all; let chain_ids: Vec<_> = rows.iter() .flat_map(|r| [r.src_chain_id, r.dst_chain_id]) .collect::<std::collections::HashSet<_>>() .into_iter() .collect(); let chain_infos: std::collections::HashMap<_, _> = join_all( chain_ids.iter().map(|&id| async move { (id, self.chain_info.get_chain_info(id).await) }) ).await.into_iter().collect(); let items = rows.into_iter().map(|row| { let source = chain_model_to_proto(chain_infos.get(&row.src_chain_id).cloned().flatten()); let destination = chain_model_to_proto(chain_infos.get(&row.dst_chain_id).cloned().flatten()); MessagePathRow { source_chain: Some(source), destination_chain: Some(destination), messages_count: i64_to_u64_nonneg(row.messages_count), } }).collect();🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@interchain-indexer/interchain-indexer-server/src/services/stats.rs` around lines 243 - 254, The loop is performing sequential awaits on self.chain_info.get_chain_info for each row which can be slow for many rows; instead collect unique src/dst chain IDs from rows, concurrently fetch all chain infos (e.g., with futures::future::join_all or FuturesUnordered) into a HashMap keyed by chain id, then map rows into MessagePathRow by looking up chain infos and calling chain_model_to_proto for source and destination; update the items construction to use the pre-fetched map and keep using i64_to_u64_nonneg for messages_count.interchain-indexer/interchain-indexer-logic/src/stats/projection.rs (3)
138-161: Consider increasing the batch size for marking messages as processed.The batch size of
2inrun_in_batches(line 139) seems unusually small. This will generate many individual UPDATE statements for large batches of messages. Consider using a larger batch size (e.g.,500or configurable based onPG_BIND_PARAM_LIMIT / 2) to reduce database round-trips.♻️ Suggested change
let mark: Vec<(i64, i32)> = rows.iter().map(|m| (m.id, m.bridge_id)).collect(); - run_in_batches(&mark, 2, |batch| async { + let batch_size = (crate::bulk::PG_BIND_PARAM_LIMIT / 2).max(1); + run_in_batches(&mark, batch_size, |batch| async { crosschain_messages::Entity::update_many()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@interchain-indexer/interchain-indexer-logic/src/stats/projection.rs` around lines 138 - 161, The batch size passed to run_in_batches when updating StatsProcessed is set to 2 which causes excessive small UPDATEs; change the hardcoded 2 to a larger value or a configurable constant (e.g., 500 or computed from PG_BIND_PARAM_LIMIT/2) so run_in_batches(&mark, <new_batch_size>, |batch| ...) groups many ids per crosschain_messages::Entity::update_many() call and reduces DB round-trips; update any config/constant and tests accordingly.
787-806: Batch size of 1 is inefficient for marking transfers as processed.Similar to the messages case,
run_in_batches(&ids, 1, ...)(line 788) will generate one UPDATE per transfer. Consider increasing to a larger batch size.♻️ Suggested change
for (aid, ids) in by_asset { - run_in_batches(&ids, 1, |batch| async { + let batch_size = crate::bulk::PG_BIND_PARAM_LIMIT.max(1); + run_in_batches(&ids, batch_size, |batch| async { crosschain_transfers::Entity::update_many()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@interchain-indexer/interchain-indexer-logic/src/stats/projection.rs` around lines 787 - 806, The loop currently calls run_in_batches(&ids, 1, ...) which issues one UPDATE per transfer; change the batch size from 1 to a larger value (e.g., 100) or replace the literal with a named constant (e.g., BATCH_SIZE) and use that in run_in_batches(&ids, BATCH_SIZE, ...) inside the same block that calls crosschain_transfers::Entity::update_many(), keeping the same filters (StatsProcessed.eq(0i16)) and column updates so updates are applied in larger groups rather than per-transfer.
765-777: Consider addingon_conflictfor edge inserts to handle race conditions.The
stats_asset_edgesinsert (lines 765-776) does not useon_conflict(). If two concurrent transactions attempt to insert the same edge (samestats_asset_id,src_chain_id,dst_chain_id), one will fail with a unique constraint violation.Since
project_transfers_batchmay run concurrently during backfill or parallel processing, consider using an upsert pattern consistent with thestats_messageshandling.♻️ Suggested change
stats_asset_edges::Entity::insert(stats_asset_edges::ActiveModel { stats_asset_id: Set(stats_asset_id), src_chain_id: Set(src_chain_id), dst_chain_id: Set(dst_chain_id), transfers_count: Set(count), cumulative_amount: Set(cumulative), decimals: Set(working_decimals), amount_side: Set(amount_side), ..Default::default() }) + .on_conflict( + OnConflict::columns([ + stats_asset_edges::Column::StatsAssetId, + stats_asset_edges::Column::SrcChainId, + stats_asset_edges::Column::DstChainId, + ]) + .value( + stats_asset_edges::Column::TransfersCount, + Expr::col((stats_asset_edges::Entity, stats_asset_edges::Column::TransfersCount)).add(count), + ) + .value( + stats_asset_edges::Column::CumulativeAmount, + Expr::col((stats_asset_edges::Entity, stats_asset_edges::Column::CumulativeAmount)).add(cumulative.clone()), + ) + .value(stats_asset_edges::Column::UpdatedAt, Expr::current_timestamp()) + .to_owned(), + ) .exec(tx) .await?;Based on learnings: "Always use
on_conflict()for upserts and batch large inserts when interacting with the database."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@interchain-indexer/interchain-indexer-logic/src/stats/projection.rs` around lines 765 - 777, The insert into stats_asset_edges (inside project_transfers_batch) can race and violate the unique constraint because it lacks on_conflict handling; change the stats_asset_edges::Entity::insert(...) call to an upsert by adding on_conflict() that targets the unique key (stats_asset_id, src_chain_id, dst_chain_id) and performs an update of the updatable columns (transfers_count, cumulative_amount, decimals, amount_side) so concurrent inserts merge instead of erroring; keep using exec(tx).await and mirror the upsert pattern you used for stats_messages to ensure consistent conflict resolution.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@interchain-indexer/interchain-indexer-logic/src/indexer/avalanche/blockchain_id_resolver.rs`:
- Around line 29-39: The in-memory cache used by resolve currently only stores
chain_id, so when force_add_chain requires DB writes those writes can be skipped
on future cache hits; change the cache value to include persistence state (e.g.
store {chain_id, persisted: bool}) or, if easier, on cache hit when
force_add_chain==true re-run the persistence path that updates chains and
avalanche_icm_blockchain_ids until it succeeds; update all code paths in resolve
that read/write the cache and the persistence logic (references: resolve,
force_add_chain, avalanche_icm_blockchain_ids, chains) so that cached entries
reflect whether DB persistence has completed, and ensure warning/partial-failure
paths mark persisted=false so subsequent calls retry.
In `@interchain-indexer/interchain-indexer-logic/src/indexer/avalanche/mod.rs`:
- Around line 835-838: The call sites to BlockchainIdResolver::resolve pass
ctx.process_unknown_chains but the implementation parameter is named
force_add_chain; rename the parameter in the resolve signature and all
implementations/overrides from force_add_chain: bool to process_unknown_chains:
bool (and update any docstrings/comments) so the semantics match the callers (no
caller changes required); ensure the parameter name is updated in the trait/impl
declarations for BlockchainIdResolver::resolve and any related tests or usages.
In `@interchain-indexer/interchain-indexer-logic/src/token_info/service.rs`:
- Around line 272-312: kickoff_token_fetch_for_stats_enrichment currently spawns
an unbounded tokio::spawn per unique token (and additionally uses
in_flight_fetches) which can fan out into thousands of concurrent RPC/DB
operations; change it to limit concurrency by introducing a bounded semaphore or
buffered stream: e.g., create a tokio::sync::Semaphore (or use
futures::stream::iter(uniq).map(|(chain_id,address)| async move { ...
}).buffer_unordered(MAX_CONCURRENCY)) and acquire a permit before calling
fetch_token_info_from_chain_and_persist, releasing the permit after the call;
keep using svc.in_flight_fetches to dedupe but ensure the spawn of
fetch_token_info_from_chain_and_persist only happens after acquiring a permit
(or run the fetches inside the buffered stream) so no more than MAX_CONCURRENCY
concurrent RPC/DB tasks run.
- Around line 235-238: The cache insertion uses or_insert_with which preserves
stale entries, so after a successful enrichment you must replace any existing
cached partial model with the fresh one; in the token_info_cache update inside
kickoff_token_fetch_for_stats_enrichment (the block that currently does
cache.entry(key.clone()).or_insert_with(|| model.clone())), change it to either
call cache.insert(key.clone(), model.clone()) or use entry(...).and_modify(|v|
*v = model.clone()).or_insert_with(|| model.clone()) so the cache is overwritten
with the enriched model rather than keeping stale data.
In `@interchain-indexer/interchain-indexer-proto/proto/v1/stats.proto`:
- Around line 15-16: Fix the typo in the comment above the Pagination message in
interchain-indexer-proto/proto/v1/stats.proto: replace "bridged-t/token stats"
with "bridged-token stats" so the comment reads "Pagination for bridged-token
stats only (`page_token` or explicit cursor fields)." and keep the rest of the
comment unchanged.
In
`@interchain-indexer/interchain-indexer-proto/swagger/v1/interchain-indexer.swagger.yaml`:
- Around line 825-827: The Swagger description contains a propagated typo
"bridged-t/token" — update the original proto comment that generates this
description to the correct term (e.g., "bridged token" or "bridged-tokens") in
the proto file's comment for the Pagination message (the comment that produced
the Swagger description "Pagination for bridged-t/token stats only..."), then
regenerate the OpenAPI/Swagger output so the corrected text replaces
"bridged-t/token" in interchain-indexer.swagger.yaml.
In `@interchain-indexer/interchain-indexer-server/src/server.rs`:
- Around line 123-130: The stats backfill (call to
stats.backfill_stats_until_idle_with_token_enrichment()) currently runs before
the metadata upserts and can write FK-backed rows against missing reference
data; move the entire conditional block that checks
settings.stats_backfill_on_start and calls
backfill_stats_until_idle_with_token_enrichment() so it executes after the
initial reference seeding calls (upsert_chains(), upsert_bridges(),
upsert_bridge_contracts()) have completed; ensure the async call and its await
remain intact and keep the tracing::info message but relocate it below the
upsert_chains/upsert_bridges/upsert_bridge_contracts sequence.
---
Nitpick comments:
In `@interchain-indexer/interchain-indexer-logic/src/chain_info/service.rs`:
- Around line 162-170: The get_all_chains_info function currently returns DB
rows without enforcing order; update get_all_chains_info to sort the normalized
Vec<chains::Model> by chains.id ascending before returning (after calling
normalize_chain in the loop or by sorting the original `chains` collection),
ensuring the returned ordering matches the documented contract; reference the
function name get_all_chains_info, the normalize_chain call, and the
db.get_all_chains result when implementing the in-service sort.
In
`@interchain-indexer/interchain-indexer-logic/src/message_buffer/persistence.rs`:
- Line 1: The file imports from std::collections are split; consolidate them
into a single crate-level import by combining HashMap and HashSet into one
statement (e.g., use std::collections::{HashMap, HashSet};) and remove the
separate HashSet import line; update the top of persistence.rs where HashMap and
HashSet are currently imported so both symbols (HashMap, HashSet) are brought in
by a single use declaration.
In `@interchain-indexer/interchain-indexer-logic/src/stats/projection.rs`:
- Around line 138-161: The batch size passed to run_in_batches when updating
StatsProcessed is set to 2 which causes excessive small UPDATEs; change the
hardcoded 2 to a larger value or a configurable constant (e.g., 500 or computed
from PG_BIND_PARAM_LIMIT/2) so run_in_batches(&mark, <new_batch_size>, |batch|
...) groups many ids per crosschain_messages::Entity::update_many() call and
reduces DB round-trips; update any config/constant and tests accordingly.
- Around line 787-806: The loop currently calls run_in_batches(&ids, 1, ...)
which issues one UPDATE per transfer; change the batch size from 1 to a larger
value (e.g., 100) or replace the literal with a named constant (e.g.,
BATCH_SIZE) and use that in run_in_batches(&ids, BATCH_SIZE, ...) inside the
same block that calls crosschain_transfers::Entity::update_many(), keeping the
same filters (StatsProcessed.eq(0i16)) and column updates so updates are applied
in larger groups rather than per-transfer.
- Around line 765-777: The insert into stats_asset_edges (inside
project_transfers_batch) can race and violate the unique constraint because it
lacks on_conflict handling; change the stats_asset_edges::Entity::insert(...)
call to an upsert by adding on_conflict() that targets the unique key
(stats_asset_id, src_chain_id, dst_chain_id) and performs an update of the
updatable columns (transfers_count, cumulative_amount, decimals, amount_side) so
concurrent inserts merge instead of erroring; keep using exec(tx).await and
mirror the upsert pattern you used for stats_messages to ensure consistent
conflict resolution.
In `@interchain-indexer/interchain-indexer-server/src/services/stats.rs`:
- Around line 243-254: The loop is performing sequential awaits on
self.chain_info.get_chain_info for each row which can be slow for many rows;
instead collect unique src/dst chain IDs from rows, concurrently fetch all chain
infos (e.g., with futures::future::join_all or FuturesUnordered) into a HashMap
keyed by chain id, then map rows into MessagePathRow by looking up chain infos
and calling chain_model_to_proto for source and destination; update the items
construction to use the pre-fetched map and keep using i64_to_u64_nonneg for
messages_count.
In `@interchain-indexer/interchain-indexer-server/src/settings.rs`:
- Around line 135-143: The test-specific function
stats_chains_period_default_for_test() redefines the literal 3600 and should be
removed; update StatsChainsPeriod to reuse the existing production default
function used for stats_chains_recalculation_period_secs (i.e., replace the
serde default reference to call the production default function instead of
stats_chains_period_default_for_test()), and delete the duplicate
stats_chains_period_default_for_test() helper so the test default will
automatically follow any production changes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 08fd22f0-b4c6-4aad-8ee5-1220b83a11f4
📒 Files selected for processing (54)
interchain-indexer/.memory-bank/gotchas.mdinterchain-indexer/Cargo.tomlinterchain-indexer/README.mdinterchain-indexer/config/avalanche/bridges.jsoninterchain-indexer/interchain-indexer-entity/src/codegen/chains.rsinterchain-indexer/interchain-indexer-entity/src/codegen/crosschain_messages.rsinterchain-indexer/interchain-indexer-entity/src/codegen/crosschain_transfers.rsinterchain-indexer/interchain-indexer-entity/src/codegen/mod.rsinterchain-indexer/interchain-indexer-entity/src/codegen/prelude.rsinterchain-indexer/interchain-indexer-entity/src/codegen/sea_orm_active_enums.rsinterchain-indexer/interchain-indexer-entity/src/codegen/stats_asset_edges.rsinterchain-indexer/interchain-indexer-entity/src/codegen/stats_asset_tokens.rsinterchain-indexer/interchain-indexer-entity/src/codegen/stats_assets.rsinterchain-indexer/interchain-indexer-entity/src/codegen/stats_chains.rsinterchain-indexer/interchain-indexer-entity/src/codegen/stats_messages.rsinterchain-indexer/interchain-indexer-entity/src/codegen/stats_messages_days.rsinterchain-indexer/interchain-indexer-logic/src/bridged_tokens_query.rsinterchain-indexer/interchain-indexer-logic/src/chain_info/service.rsinterchain-indexer/interchain-indexer-logic/src/database.rsinterchain-indexer/interchain-indexer-logic/src/indexer/avalanche/blockchain_id_resolver.rsinterchain-indexer/interchain-indexer-logic/src/indexer/avalanche/consolidation.rsinterchain-indexer/interchain-indexer-logic/src/indexer/avalanche/mod.rsinterchain-indexer/interchain-indexer-logic/src/lib.rsinterchain-indexer/interchain-indexer-logic/src/message_buffer/buffer.rsinterchain-indexer/interchain-indexer-logic/src/message_buffer/maintenance.rsinterchain-indexer/interchain-indexer-logic/src/message_buffer/mod.rsinterchain-indexer/interchain-indexer-logic/src/message_buffer/persistence.rsinterchain-indexer/interchain-indexer-logic/src/pagination.rsinterchain-indexer/interchain-indexer-logic/src/stats/mod.rsinterchain-indexer/interchain-indexer-logic/src/stats/projection.rsinterchain-indexer/interchain-indexer-logic/src/stats/service.rsinterchain-indexer/interchain-indexer-logic/src/stats_chains_query.rsinterchain-indexer/interchain-indexer-logic/src/token_info/service.rsinterchain-indexer/interchain-indexer-migration/src/lib.rsinterchain-indexer/interchain-indexer-migration/src/m20260312_175120_add_stats_tables.rsinterchain-indexer/interchain-indexer-migration/src/migrations_down/m20260312_175120_add_stats_tables_down.sqlinterchain-indexer/interchain-indexer-migration/src/migrations_up/m20260312_175120_add_stats_tables_up.sqlinterchain-indexer/interchain-indexer-proto/build.rsinterchain-indexer/interchain-indexer-proto/proto/v1/api_config_http.yamlinterchain-indexer/interchain-indexer-proto/proto/v1/interchain_indexer.protointerchain-indexer/interchain-indexer-proto/proto/v1/stats.protointerchain-indexer/interchain-indexer-proto/swagger/v1/interchain-indexer.swagger.yamlinterchain-indexer/interchain-indexer-server/config/example.tomlinterchain-indexer/interchain-indexer-server/src/config.rsinterchain-indexer/interchain-indexer-server/src/indexers.rsinterchain-indexer/interchain-indexer-server/src/server.rsinterchain-indexer/interchain-indexer-server/src/services/chain_info_proto.rsinterchain-indexer/interchain-indexer-server/src/services/interchain_service.rsinterchain-indexer/interchain-indexer-server/src/services/mod.rsinterchain-indexer/interchain-indexer-server/src/services/stats.rsinterchain-indexer/interchain-indexer-server/src/settings.rsinterchain-indexer/interchain-indexer-server/tests/avalanche_e2e.rsinterchain-indexer/justfileinterchain-indexer/types/package.json
💤 Files with no reviewable changes (1)
- interchain-indexer/interchain-indexer-server/src/config.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch introduces a complete statistics pipeline for interchain analytics and exposes the first read APIs on top of it.
The implementation adds:
In practice, the branch moves statistics from ad hoc runtime computation to materialized database state that can be queried with stable pagination and predictable response time.
Common Implementation Plan
The branch can be understood as one rollout executed in five steps:
Add schema support for analytics data.
stats_assets,stats_asset_tokens,stats_asset_edges,stats_messages,stats_messages_days, andstats_chains.stats_processedcounters tocrosschain_messagesandcrosschain_transfersso projection is incremental and idempotent-friendly.Populate stats tables from indexed data.
stats_messagesandstats_messages_days.Make historical data usable immediately after deploy.
stats_backfill_on_startto project all rows withstats_processed = 0during startup.Add orchestration and refresh flows for read-side APIs.
StatsServiceas the orchestration layer for projection, backfill, enrichment kickoff, and stats queries.stats_chainsrecomputation viastats_chains_recalculation_period_secs.ChainInfoServiceso list/stat endpoints return consistent explorer and route fields.Expose public APIs on top of materialized stats.
stats_messages_days.Introduced API Endpoints
GET /api/v1/interchain/chainsReturns the full list of known chains from the
chainstable.Notes:
id,name,logo,explorer_url,custom_tx_route,custom_address_route, andcustom_token_route;Unknown.GET /api/v1/stats/chain/{chain_id}/bridged-tokensReturns bridged-token aggregates for a chain.
Purpose:
Request behavior:
page_size,last_page, and cursor pagination;TOTAL_TRANSFERS_COUNT,OUTPUT_TRANSFERS_COUNT,INPUT_TRANSFERS_COUNT, orNAME;DESCandASCorder;page_tokenor raw cursor fields depending onapi.use_pagination_token.Response shape:
tokens[]withchain_id,token_address,name,symbol,icon_url, anddecimals.GET /api/v1/stats/chainsReturns known chains together with
unique_transfer_users_count.Request behavior:
page_size,last_page, and cursor pagination;chain_idsfilter as a comma-separated list;ASCandDESCordering byunique_transfer_users_count.Response shape:
chain_id,name,icon_url,explorer_url, andunique_transfer_users_count;stats_chainsrow, in which case the counter is0.GET /api/v1/stats/chain/{chain_id}/messages-paths/sentReturns outgoing message paths for the selected chain.
Request behavior:
chain_idis required;from_dateandto_dateare optional and must useYYYY-MM-DD;counterparty_chain_idsis optional and accepts a comma-separated list ofint64chain ids.Response shape:
source_chain,destination_chain, andmessages_count.Data source:
GET /api/v1/stats/chain/{chain_id}/messages-paths/receivedReturns incoming message paths for the selected chain.
Behavior matches the
sentendpoint, but the selected chain is treated as the destination side.Configuration Changes
New server settings:
stats_backfill_on_start(INTERCHAIN_INDEXER__STATS_BACKFILL_ON_STARTENV)falsetrue, the service projects pending stats rows during startup before indexers begin;stats_chains_recalculation_period_secs(INTERCHAIN_INDEXER__STATS_CHAINS_RECALCULATION_PERIOD_SECSENV)3600(1 hour)stats_chains;0disables the background refresh worker.Notes For Reviewers
stats_messages_daysin addition to cumulativestats_messages.Summary by CodeRabbit
New Features
Documentation
Bug Fixes