fix(data-api): order broad extrinsics/chain-events feeds by observed_at#4955
Conversation
GET /api/v1/extrinsics was returning HTTP 200 with an empty feed (extrinsic_count:0, generated_at:null) for the unfiltered/default view and for broad filters like success=true or call_module=SubtensorModule, while narrow filters worked fine. Root cause: extrinsics/chain_events are TimescaleDB hypertables partitioned on observed_at, but the recent-feed queries ordered by `block_number DESC, extrinsic_index DESC` -- a column with no chunk-exclusion information, forcing a full cross-chunk sort (measured 6+ seconds live, including decompressing 15 historical compressed chunks). That blew past this route's 3000ms statement_timeout, which tryPostgresTier treats as "fall back to D1" -- and D1's extrinsics table was fully dropped in the 2026-07-11 chain-data retirement (#4772), so the fallback silently resolved to an empty-but-200 response instead of erroring. Reordering the recent-feed queries (extrinsics, sudo, governance/ config-changes, chain-events) to `observed_at DESC, block_number DESC, ...` lets ChunkAppend prune straight to the live head chunk -- confirmed live against production Postgres: the same queries dropped from 6041ms to 0.5-75ms. observed_at is the genuine on-chain block timestamp (verified against block #1: 2023-03-20, not an ingestion clock), so this produces the identical ordering as block_number DESC; block_number/extrinsic_index only break ties among extrinsics sharing one block's timestamp. Cursors for the affected routes are now 3-tuples (observed_at, block_number, position) to match; an old 2-part cursor decodes to null (page restarts) rather than crashing. Also fixes scripts/fetch-subnet-hyperparams.py: every hyperparameters row served in production had block_number:0, because `getattr(s.substrate, "block_number", 0)` always fell back to 0 -- SubstrateInterface has no `block_number` attribute (it's the method get_block_number()). Reuses the metagraph snapshot's own per-info `.block` (already fetched in the same run) instead of an extra RPC call per netuid, matching fetch-metagraph-native.py's existing convention. Verified live: 129/129 subnets now report the real chain height.
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
metagraphed-data-api | dcca3d0 | Jul 12 2026, 02:38 PM |
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4955 +/- ##
=======================================
Coverage 97.94% 97.94%
=======================================
Files 163 163
Lines 19418 19420 +2
Branches 7376 7376
=======================================
+ Hits 19018 19020 +2
Misses 53 53
Partials 347 347
🚀 New features to boost your workflow:
|
|
Warning 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 ⏸️ Gittensory review result - manual review recommendedReview updated: 2026-07-12 14:43:23 UTC
⏸️ Suggested Action - Manual Review
Review summary Blockers
Nits — 5 non-blocking
Concerns raised — review before merging
Review context
Contributor next steps
Signal definitions
[BETA] Chat with GittensoryAsk Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.
Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands Visual preview
Click any thumbnail to open the full-size screenshot. Before = production · After = this PR's preview deploy. 🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.
|
…tract change public/metagraph/openapi.json and packages/contract/index.d.ts changed (ChainEventsFeedArtifact.next_cursor's pattern/maxLength widened to a 3-part keyset cursor) -- validate:client-sdk-sync requires the published client version to move in the same PR as a contract change.
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
metagraphed-registry-sync-api | dcca3d0 | Jul 12 2026, 02:38 PM |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
metagraphed-ui | dcca3d0 | Commit Preview URL Branch Preview URL |
Jul 12 2026, 02:39 PM |
Summary
GET /api/v1/extrinsicswas silently returning an empty feed (HTTP 200,extrinsic_count:0,generated_at:null) for the unfiltered/default view and for broad filters (success=true,call_module=SubtensorModule) — confirmed in production during a live-wiring verification sweep. Narrow filters (a specific block,success=false, a specific signer) worked correctly, which pointed at a query-shape problem rather than a data or decode problem.extrinsics/chain_eventsare TimescaleDB hypertables partitioned onobserved_at, but the recent-feed queries ordered byblock_number DESC, extrinsic_index DESC— a column with zero chunk-exclusion value for the planner.EXPLAIN ANALYZEagainst production showed a full cross-chunkParallel Append+Sort, decompressing 15 historical compressed chunks, taking 6041ms — well past this route's 3000msstatement_timeout.tryPostgresTiertreats any non-2xx as "fall back to D1"; D1'sextrinsicstable was fully dropped in the 2026-07-11 chain-data retirement (Fully retire D1 for chain data — delete tables, prune logic, ingest endpoints, and the D1 fallback (gated on #4771) #4772), so the fallback silently resolved to an empty-but-200 response instead of surfacing an error anywhere./api/v1/extrinsics,/api/v1/sudo,/api/v1/governance/config-changes,/api/v1/chain-events) toORDER BY observed_at DESC, block_number DESC, ..., which lets TimescaleDB'sChunkAppendprune straight to the live head chunk. Confirmed live against production Postgres: the identical queries dropped from 6041ms to 0.5–75ms.observed_atis the genuine on-chain block timestamp (verified against block chore(deps): bump astral-sh/setup-uv from d0d8abe699bfb85fec6de9f7adb5ae17292296ff to d0cc045d04ccac9d8b7881df0226f9e82c39688e #1 → 2023-03-20, not an ingestion clock), so ordering by it produces the exact same "most recent first" semantics asblock_number DESC— block production is strictly monotonic with wall-clock time. Cursors for the affected routes are now 3-tuples(observed_at, block_number, position); an old 2-part cursor just decodes tonull(cursor ignored, page restarts) rather than erroring — no live callers depend on cursor stability across this change.scripts/fetch-subnet-hyperparams.py: every hyperparameters row served in production hadblock_number:0becausegetattr(s.substrate, "block_number", 0)silently fell back to the default —SubstrateInterfacehas noblock_numberattribute (it's the methodget_block_number()). Now reuses the metagraph snapshot's own per-info.blockfield (already fetched in the same run, zero extra RPC calls), matchingfetch-metagraph-native.py's existing convention. Verified live: 129/129 subnets now report the real chain height (~8605812) instead of 0.Test plan
npm run lint/npm run format:checkcleannpm test— 256/256 files, 7579/7579 tests green (full suite, twice)npm run validate/validate:contract-drift/validate:schemas/validate:api/validate:openapiall greenEXPLAIN (ANALYZE, BUFFERS, TIMING)(SSH to the indexer box) — before: 6041ms unfiltered scan; after: 36–75ms for unfiltered,success=true, andcall_module=SubtensorModulecasesfetch-subnet-hyperparams.pyend-to-end against productionfinney(bittensor 10.4.0, the CI-pinned version) — 129/129 subnets, all with a real non-zeroblock_numbertests/data-api.test.mjsfor the new 3-tuple cursor shape on all four affected routesChainEventsFeedArtifact.next_cursorpattern/length/description) and regenerated (npm run build→openapi.json,types.d.ts,packages/contract/index.d.ts,schemas/api-components.schema.json)