Skip to content

Track import-HHI Comtrade 429 coverage misses #3979

@koala73

Description

@koala73

Context

PR #3978 fixes the deterministic import-HHI coverage bugs: shared non-standard reporter overrides, smaller Comtrade payload shape, strict seed freshness, and a realistic publish floor. One operational risk remains: correct-code reporters can still miss when Comtrade returns 429s during the monthly seed.

Review evidence: a 106-reporter probe on the two production Comtrade keys hit 46 HTTP 429 responses. Payload shaping reduces oversized responses and timeout pressure, but it does not increase Comtrade key budget. The current mitigation is the seeder retry/backoff plus checkpoint/resume accumulation.

Risk

AE/RU/NO/CH have correct reporter codes and should recover once the seeder completes enough successful fetches. If they remain unmonitored after #3978 deploys and a couple of cron/force-refresh cycles run, the likely cause is still Comtrade rate limiting rather than reporter-code drift.

Acceptance criteria

  • After Fix import-HHI coverage freshness #3978 deployment, verify resilience:recovery:import-hhi:v1 and live resilience scores for AE/RU/NO/CH after force-refresh or two monthly cron attempts.
  • If those reporters remain absent and seed logs show 429s, increase Comtrade key budget and/or widen request spacing before changing scoring logic.
  • Document the operational runbook decision: more COMTRADE_API_KEYS, higher PER_KEY_DELAY_MS, lower concurrency, or reporter cohort splitting.

Non-goals

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions