Tycho Price Solver

Solver-only price ingestion service that leverages Tycho Simulation streams to produce USD spot prices for tokens tracked by Tycho. It runs as an external binary, consumes the hosted Tycho endpoints, and emits periodic JSON reports describing the best routes (≤ max_hops) from each token into configured numeraires. Next milestones cover size probing, on-chain adapters, and persistence into Tycho DB.

Features

• Discovers tokens/components via Tycho RPC (hosted or local) and seeds an in-memory catalog.
• Subscribes to protocol streams (Uniswap v2/v3/v4, Sushi v2, Pancake v2/v3, Ekubo v2, Curve, Balancer, Maverick v2, Rocketpool) using Tycho Simulation.
• Maintains live protocol state in memory, logging spot_price updates for new pools.
• Every refresh_interval_seconds, re-evaluates only dirty tokens (recent pool updates + numeraires) using an incremental route cache, simulates the best paths via Tycho get_amount_out (respecting probe_amounts), and merges the refreshed quotes into a token-centric JSON report (default prices-<chain>.json) including route metadata.
• Supports graceful shutdown (Ctrl+C) and structured tracing via tracing.

Prerequisites

• Rust toolchain (1.70+ recommended).
• Hosted Tycho access (defaults to tycho-beta.propellerheads.xyz with sampletoken auth).
• Optional: EVM RPC URL for future on-chain quoting (placeholder today).

Configuration

All runtime settings live in solver-config.yaml (YAML). A sample config is provided in solver-config.example.yaml:

JSON schema

Generate a JSON Schema describing the configuration accepted by this binary:

tycho-price-solver config schema --output solver-config.schema.json

logging:
  level: info
runtime:
  refresh_interval_seconds: 180
  cache_ttl_seconds: 300
  batch_size: 128
  metrics_addr: 0.0.0.0:9898
  # max_stream_concurrency: 64   # optional; defaults to detected CPU count
  # stream_start_rate_per_sec: 25 # optional; caps new chunked streams per second
  # stream_chunk_size: 25000      # optional; IDs per fallback chunk (stays <2MiB per request)
  # stream_timeout_seconds: 36    # optional; Tycho stream timeout (eth default 36 => 48s for first deltas)
output:
  mode: json        # json | db | both
  normalization:
    anchor: USDC     # symbol or address; output denomination for JSON + DB prices
    strict: false    # when true, missing conversions to anchor fail the pricing loop
  json:
    # path: prices-ethereum.json   # optional global default; per-chain output_file still works
  db:
    # dsn: postgres://USER:PASS@HOST:PORT/tycho_indexer_0
    # max_pool_size: 4
    # statement_batch_size: 200
    # invoke_tvl: false            # run inline component_tvl SQL after upserts
    # timeout_ms: 0                # optional statement timeout (ms) inside the write transaction
chains:
  - name: ethereum
    tycho_url: https://tycho-indexer.example.xyz
    auth_key: auth-key-sample
    rpc_url: https://mainnet.example-rpc
    numeraires: [USDC, USDT, ETH]
    protocols:
      - uniswap_v2
      - sushiswap_v2
      - pancakeswap_v2
      - uniswap_v3
      - pancakeswap_v3
      - uniswap_v4
      - ekubo_v2
      - vm:curve
      - vm:balancer_v2
      - vm:maverick_v2
      - rocketpool
    # min_tvl_native: 10        # optional; omit or set to null to include components without TVL
    token_quality_min: 100
    missing_token_policy: fetch
    max_hops: 3           # number of hops to explore when routing into numeraires
    probe_amounts:        # optional overrides for amount-out probe sizing
      WETH: "1e18"
      USDC: "1000000"
    output_file: prices-ethereum.json

Key fields:

• tycho_url: Tycho indexer endpoint, including scheme and optional port.
• auth_key: API key for Tycho indexer endpoint.
• native_token: Wrapped native token address (e.g. WETH on Ethereum). Used when refreshing component_tvl in native units (output.db.tvl_denomination: native).
• protocols: List of protocol IDs to stream.
• min_tvl_native: Optional TVL floor (native units). Omit or set to null when the indexer has not populated TVL yet; provide a value to require components above that threshold.
• token_quality_min: Minimum Tycho quality score (0–100) to include during token discovery. Set to null to accept every token Tycho knows about (the solver ships with 100 by default).
• missing_token_policy: How to handle pools whose tokens aren’t in the local cache. drop logs and excludes them, fetch (default) pulls token metadata on-demand from Tycho, allow behaves like fetch but still treats misses as optional during readiness checks.
• probe_amounts: Optional per-token overrides (symbol or address) used as input sizes for get_amount_out; defaults to 1 * 10^decimals when unspecified.
• cache_ttl_seconds: Forces a full re-enumeration of routes/prices when the cached results become older than the TTL (safety valve for long-running processes).
• metrics_addr: Address (host:port) for the embedded Prometheus /metrics endpoint. Set to null to disable the exporter.
• output.mode: json (default), db, or both. Controls whether price outputs are written to disk, Postgres, or both.
• output.normalization.anchor: Optional normalization anchor (symbol or address). When set, the solver converts all computed prices into the anchor before emitting JSON/DB output and ranks route candidates using the anchor-denominated value.
• output.normalization.strict: When true, missing conversions to the anchor fail the pricing loop. When false (default), the solver falls back to the next-best convertible route and skips tokens that cannot be converted.
• output.json.path: Optional global JSON path. Chain-level output_file still overrides this (or defaults to prices-<chain>.json when unset).
• output.db.*: DSN and write tuning for in-process DB writes. statement_batch_size chunks the upserts, invoke_tvl runs the inline component_tvl refresh after a successful write, tvl_denomination chooses whether the refreshed TVL is in the normalization anchor or the chain's native token, tvl_require_native_price can hard-fail native refresh when the native conversion price is missing, and timeout_ms sets a transaction-local statement timeout.
• max_stream_concurrency: Upper bound on parallel protocol streams when the solver falls back to ID chunking. Defaults to the machine’s CPU count when omitted.
• stream_start_rate_per_sec: Limits how many new fallback chunk streams may be established per second (defaults to a conservative rate if unset).
• stream_chunk_size: Number of component IDs per fallback chunk (defaults to ~25k so protocol_components responses stay under Tycho’s 2 MiB limit).
• stream_timeout_seconds: Overrides the Tycho stream timeout (seconds). Increase this when you see “First deltas took longer than …s to arrive”.
• output_file: Path of the JSON report (relative or absolute).

Environment Variable Overrides (Secrets-Friendly)

For Kubernetes/Helm deployments, secrets should generally be supplied via environment variables (or mounted secret files) rather than baked into solver-config.yaml.

The solver supports these overrides (precedence: chain-specific env vars → global env vars → config file):

• Tycho auth key:
  • <CHAIN>_TYCHO_AUTH_KEY (e.g. ETHEREUM_TYCHO_AUTH_KEY)
  • <CHAIN>_AUTH_KEY (alias; e.g. ETHEREUM_AUTH_KEY)
  • TYCHO_AUTH_KEY
  • File variants: <CHAIN>_TYCHO_AUTH_KEY_FILE, <CHAIN>_AUTH_KEY_FILE, TYCHO_AUTH_KEY_FILE
• Tycho URL:
  • <CHAIN>_TYCHO_URL (e.g. ETHEREUM_TYCHO_URL)
  • TYCHO_URL
  • File variants: <CHAIN>_TYCHO_URL_FILE, TYCHO_URL_FILE
• Postgres DSN (when output.mode is db or both):
  • TYCHO_PRICE_SOLVER_DB_DSN
  • File variant: TYCHO_PRICE_SOLVER_DB_DSN_FILE

RPC access for VM-backed pools is provided via the standard RPC_URL environment variable. If chains[].rpc_url is set, the solver will set RPC_URL for you at startup; otherwise it expects RPC_URL to already be present in the environment.

Running

From repository root:

cargo run --manifest-path tycho-price-solver/Cargo.toml -- \
  --config tycho-price-solver/solver-config.example.yaml

Runtime behaviour:

1. Discovery – Fetch tokens/components via Tycho RPC and populate in-memory state.
2. Streaming – Start Tycho Simulation stream; log solver::spot entries for new pools and keep protocol states updated (archive RPC required for VM-backed pools).
3. Pricing loop – Every refresh_interval_seconds (default 180s), resolve numeraires, refresh route candidates for dirty tokens (full rebuild triggered by cache_ttl_seconds), re-evaluate their quotes with Tycho Simulation get_amount_out, flag anomalies (e.g., zero quotes, extreme prices), and emit a JSON report that merges refreshed entries with cached prices (fallback still covers direct numeraires when multi-hop routing fails). During bootstrap we tag discovery components that Tycho fails to hydrate (e.g., VM decode errors, unsupported hooks) with stream_decode_failure, track them as filtered, and allow readiness to progress once every configured component has been attempted.

Metrics

• A lightweight HTTP exporter is available at http://<metrics_addr>/metrics (default 0.0.0.0:9898).
• The emitted Prometheus series cover stream readiness (solver_stream_ready), component catch-up progress, discovery/pricing loop durations, pricing token outcomes, unresolved numeraires, and cache sizes.
• Disable the endpoint by setting runtime.metrics_addr to null or an empty string in solver-config.yaml.
• Example scrape config:

  - job_name: tycho-price-solver
    static_configs:
      - targets: ["localhost:9898"]

JSON → SQL Export

• The helper script tycho-price-solver/scripts/json_to_sql.py turns a solver JSON report into INSERT ... ON CONFLICT statements against the Tycho indexer database. Run it from the repo root (so relative imports resolve) using any Python 3.10+ interpreter or uv run.

• The solver now supports in-process DB writes (output.mode: db|both) to avoid the JSON round-trip; the Python helper remains available for manual runs/backfill.

• Note: component_tvl denomination depends on how you refresh it. If token_price.price is anchor-denominated (e.g. USDC) then the raw sum is anchor TVL; setting output.db.tvl_denomination: native converts the refreshed component_tvl into native units (e.g. ETH/WETH on Ethereum) using the configured native_token price.

• Basic invocation

  uv run tycho-price-solver/scripts/json_to_sql.py \
    --input prices-ethereum.json \
    --dsn postgres://USER:PASS@HOST:PORT/tycho_indexer_0 \
    --dry-run

  --dry-run is strongly recommended for the first pass—the script will emit warnings for missing tokens and print the SQL it would have executed before rolling the transaction back.

• Flags overview

  Flag	Meaning
  --input PATH	Path to the solver JSON file (defaults to prices-<chain>.json).
  --dsn DSN	Postgres connection string. For Docker compose the DSN is typically postgres://postgres:<POSTGRES_PASSWORD>@localhost:5431/tycho_indexer_0.
  --chain NAME	Overrides the chain slug; otherwise it is derived from the filename (prices-ethereum.json → ethereum).
  --dry-run	Perform validation and print SQL but do not modify the database.
  --invoke-tvl	After a successful upsert, execute the component_tvl refresh SQL.
  --batch-size N	Chunk size for executemany batches (defaults to 200).
  --log-level LEVEL	One of DEBUG/INFO/WARNING/ERROR/CRITICAL.

• Common issues & troubleshooting

  • connection refused → ensure the Postgres container is running and the DSN host/port matches the published mapping (see docker compose -f tycho-indexer/docker-compose.yaml ps).
  • token_id not found warnings mean the JSON contains addresses not yet present in the token table. Re-run Tycho discovery or accept that those rows will be skipped; the rest of the batch still commits.
  • column "address" does not exist or similar indicates an outdated script. The current version joins token → account and normalises 0x-prefixed addresses before lookup.
  • function calculate_component_tvl() does not exist surfaced on older databases; the script now issues the inline INSERT ... ON CONFLICT SQL Tycho uses internally, so updating to the latest script resolves it.

• Refreshing TVL after import

  • Pass --invoke-tvl when you run the script to trigger the SQL refresh automatically.
  • Or execute the statement manually:

    INSERT INTO component_tvl (protocol_component_id, tvl)
    SELECT bal.protocol_component_id,
           SUM(bal.balance_float * token_price.price / POWER(10.0, token.decimals))
    FROM component_balance AS bal
    JOIN token_price   ON bal.token_id = token_price.token_id
    JOIN token         ON bal.token_id = token.id
    WHERE bal.valid_to = '262142-12-31 23:59:59.999999'
    GROUP BY bal.protocol_component_id
    ON CONFLICT (protocol_component_id) DO UPDATE SET tvl = EXCLUDED.tvl;

  • Verify success with:

    SELECT COUNT(*) FROM component_tvl;
    SELECT protocol_component_id, tvl, modified_ts
    FROM component_tvl ORDER BY modified_ts DESC LIMIT 5;

Endpoint Comparator

• Use tycho-price-solver/scripts/compare_endpoints.py to diff two Tycho indexer RPC endpoints.
• Basic usage:

  uv run tycho-price-solver/scripts/compare_endpoints.py \
    --endpoint-a http://localhost:4242 \
    --token-a readme \
    --endpoint-b https://tycho-beta.propellerheads.xyz \
    --token-b sampletoken \
    --chain ethereum \
    --protocol-system uniswap_v2 \
    --protocol-system sushiswap_v2,pancakeswap_v2,pancakeswap_v3,ekubo_v2,vm:balancer_v2,vm:maverick_v2,rocketpool \
    --tvl-tolerance 0.05 \
    --out diff.json

• The tool checks protocol systems, component IDs, and component TVLs. Adjust tolerances and filters via CLI flags (--help for details; useful knobs include --page-size and --max-pages).
• --protocol-system may be passed multiple times (or as a comma-separated list) to compare subsets such as sushiswap_v2, pancakeswap_v2, pancakeswap_v3, ekubo_v2, vm:balancer_v2, vm:maverick_v2, and rocketpool.
• It first reports component and TVL entry counts per protocol system before diving into detailed diffs. Pass --counts-only to stop after that summary.
• By default the remote endpoint is only queried for component IDs present locally. Use --full-remote to fetch every component from endpoint B as well.
• If a protocol system isn’t enabled on one endpoint (e.g., extractor not running locally) the script logs a warning like “Skipping protocol_system …” and simply omits it from the comparison.

Docker Observability Stack

• A sample Compose bundle lives in observability/docker-compose.yaml; it builds the solver image, and starts Prometheus + Grafana pre-wired against the /metrics endpoint.
• Launch everything with:

  docker compose -f observability/docker-compose.yaml up --build

  (supply a custom solver config by editing tycho-price-solver/solver-config.example.yaml or bind-mounting your own file).
• Prometheus is available at http://localhost:9090, Grafana at http://localhost:3000 (default credentials admin / admin). A starter dashboard titled Tycho Price Solver Overview is provisioned automatically.
• Dashboard highlights: stream readiness, component coverage, pricing rates, loop latency, decode failures grouped by protocol/reason, unresolved numeraires, cache sizes, and discovery duration p95.
• Shut the stack down via docker compose -f observability/docker-compose.yaml down.

Report format (prices-ethereum.json):

{
  "generated_at": "2025-02-12T14:05:00.123Z",
  "entry_count": 3,
  "route_token_count": 2,
  "fallback_token_count": 1,
  "anomaly_count": 1,
  "zero_price_count": 0,
  "entries": [
    {
      "token_address": "0xa0b8…",
      "token_symbol": "USDC",
      "numeraire": "USDT",
      "price": 0.9998,
      "hops": 2,
      "route_symbols": ["USDC", "WETH", "USDT"],
      "route_components": [
        "0x8ad6…",   // USDC/WETH pool
        "0x88b2…"    // WETH/USDT pool
      ],
      "route_kind": "route",
      "flags": []
    }
    // ...
  ]
}

Stop the service with Ctrl+C; both the stream and pricing task will shut down.

Logs & Output

• solver::spot (INFO): new pool spotted + spot price.
• solver::routing (INFO/WARN): numeraires resolution, route enumeration counts, failures.
• solver::pricing (INFO/WARN): JSON report success/failure plus metadata (route tokens, fallback tokens, anomaly counts).
• solver::runtime (INFO/WARN): lifecycle events and shutdown signals.
• solver::stream (INFO): catch-up summaries including filtered_components= for pools we intentionally skipped (missing metadata, decode failures, protocol filters).

Known warnings:

• Ticks exceeded / No liquidity during get_amount_out – indicates the pool lacks data for a quote; entries will show price: null and a missing_quote flag.

Troubleshooting Readiness

• Confirm discovery finished (Discovery phase ready for pricing).
• Look for the latest Stream catch-up progress line: missing_components should trend to zero, while filtered_components lists the pools Tycho couldn’t hydrate (e.g., VM decode failures). Those filtered entries no longer block readiness but still deserve follow-up if the counts grow unexpectedly.
• Once Stream ready for pricing (component catch-up) appears, the pricing loop should be active; if prices are missing afterwards, inspect solver::pricing warnings for route/quote failures.

Roadmap

• Extend protocol adapters (Concentrated liquidity, Balancer VM) with richer probe heuristics and staleness checks once archive RPC access is available.
• Persist prices directly into Tycho DB (token_price) and trigger calculate_component_tvl.
• Enrich observability (metrics exporter, cache hit ratios, anomaly summaries) and add CLI modes for one-shot runs / diffing.

References

• TYCHO_PRICE_INGESTION_SOLVER_ONLY.md
• TYCHO_PRICE_FEED_KNOWLEDGE.md – architecture notes and runtime behaviour.