Bittensor monorepo consolidation: SDK, py-sp-core, website, docs, clones, CI, and release train

[unarbos]

Summary

This PR lands the monorepo consolidation plus the runtime changes that accumulated on mono-bittensor since it diverged from main (~110 commits, spec version now 426). It is large by nature; the review checklist below points at the load-bearing pieces.

What we have done

Monorepo consolidation

• sdk/python/ — the Python SDK (bittensor package) now lives in-repo: intent-based tx API (plan/confirm/execute), generated calls/storage/errors catalogs under _generated/, btcli CLI (wallet, EVM, multisig, extension signer), and a codegen gate (codegen.check --coverage/--names/--drift) that fails CI when the SDK drifts from node metadata.
• py-sp-core/ — new PyO3 crate providing native sp-core key handling to the SDK: sr25519/ed25519 keypairs, SS58, keyfile encryption (NaCl, legacy Fernet, ansible-vault), and PolkadotJS encrypted-JSON import.
• website/ — docs site and website monorepo (Turborepo + Vercel), with tx/query/error reference pages generated from the SDK catalogs; a drift gate fails CI when committed docs pages go stale.
• clones/ — mainnet-clone regression harness: clone mainnet state, sudo-upgrade it to the PR's runtime, then run a curated JS regression suite (npm run test:clone-regressions, 9 tests: balancer, locks/conviction, proxy filters, hotkey swap, emission allocation, issuance trackers) plus the SDK offline suite against the upgraded chain (check-clone-upgrade.yml).

Runtime changes (spec 424 → 426)

• Coldkey pays tx fees for hotkey-origin calls (Charge hotkey-origin fees to the owning coldkey #2570), with a fee_filters.rs list mirrored into the SDK so plan() warns users before signing.
• Reverse index for AssociatedEvmAddress + MAX_ASSOCIATED_UIDS_PER_EVM_ADDRESS cap with a pruning migration (Charge hotkey-origin fees to the owning coldkey #2570 follow-ups).
• Proxy deny-by-default: proxy filter is now an explicit allowlist (Proxy deny by default #2802), missing faucet call added.
• Crowdloan reentrancy fix (Fix crowdloan reentrancy bug #2786), UID lookup fix (Make uidLookup O(1) and bound its cost #2830), start-call error fix (Fix bad start call error name #2811), nonce-reuse avoidance in contract deploy (add finalize after contract deploy #2820), rate-limit cleanup (Remove rate limits for hotkey/coldkey swaps #2727), registry cleanup migration removed (Remove registry pallet cleanup migration #2815).

Org transfer

• All repo URLs, crate metadata, Docker image names (ghcr.io/raofoundation/*), CI workflows, and review tooling moved from opentensor to RaoFoundation (Org migration: update references for RaoFoundation #2834).

CI / release automation

• Release train (release-train.yml): build runtime once → deploy devnet → smoke → deploy testnet → smoke → publish SDK rc to PyPI → propose mainnet upgrade behind an environment gate + triumvirate multisig. Now also publishes py-sp-core manylinux wheels alongside the SDK rc.
• watch-mainnet-release.yml: watches the chain for the executed multisig upgrade, then tags the release, publishes wheels/crates/Docker images, and deploys the prod website. Release SHA now comes from the release-train artifact (pending-release.json) instead of HEAD of main, so docs-only merges after the proposal can't skew the tag.
• Branch mirrors: after each successful deploy, devnet/testnet branches are force-updated to the deployed commit; mainnet branch is updated by the watcher once the multisig executes. These branches always reflect what runs on each chain.
• Fork hardening: every workflow that uses self-hosted runners now depends on a trusted-pr job that only passes for non-fork PRs, so a fork PR can't execute untrusted code on persistent org hardware.
• E2E consolidation: check-bittensor-e2e-tests.yml no longer clones external btcli/bittensor repos at staging; it collects and runs the in-repo SDK e2e suite (-m e2e) against the locally built localnet image.
• New pre-mainnet safety check: verify_finney_evm_association_cap.py scans live Finney state before proposing spec ≥ 426 and fails if any (netuid, evm_address) bucket exceeds the new cap (the migration would prune entries).

SDK / security hardening (final commit)

• Extension bridge auth: the local browser-extension signing bridge now requires a per-session client token (written to ~/.bittensor/extension_bridge.token, mode 0600); unauthorized websocket clients are rejected, and pending requests fail fast when the bridge page disconnects instead of hanging.
• scrypt param validation in py-sp-core encrypted-JSON import (n ≤ 2^18 power of two, r ≤ 8, p = 1) so a malicious keystore JSON can't CPU-DoS the import path; covered by new Rust unit tests including an ed25519 passphrase round-trip.
• Extension signer now rejects ecdsa/ethereum extension accounts explicitly instead of mis-signing them as ed25519.
• Coldkey-fee warning surfaced in Executor.plan() for hotkey-signed calls (recurses through proxy/batch/multisig wrappers).
• Codegen coverage extended to Commitments, AdminUtils (derived from the intent registry), and Contracts pallets.

Verification done locally

• sdk/python: ruff check + ruff format --check clean, 824 unit tests pass (1 skipped), codegen.check --coverage and --names pass.
• py-sp-core: 16 Rust tests pass, including the new scrypt-cap and encrypted-JSON round-trip tests.
• actionlint on all workflows: no errors (only pre-existing shellcheck style infos).

What reviewers should check before merge

Runtime (highest stakes — this ships to mainnet as spec 426)

• Migration for the AssociatedEvmAddress reverse index + cap: verify it's idempotent and try-runtime passes against Finney and testnet state.
• Fee routing for hotkey-origin calls: confirm the call list in runtime/src/fee_filters.rs matches sdk/python/bittensor/fee_filters.py exactly — these are maintained by hand in two places.
• Proxy allowlist (deny-by-default): review the explicit allowlist for calls that should (or should not) be proxyable; this is a behavior change for existing proxies.
• Spec version 426 and no-spec-version-bump gate behavior.

Release automation (can push code to live networks)

• release-train.yml: the mainnet proposal path (environment gate, multisig scripts under .github/scripts/deploy/) — confirm the required reviewers on the mainnet environment are configured in repo settings before this merges.
• Force-pushes to devnet/testnet/mainnet branches: confirm these branch names are not protected in a way that breaks the mirror step, and that nobody bases work on them.
• watch-mainnet-release.yml: artifact lookup by mainnet-upgrade-<spec> name — check retention (artifacts expire; a long gap between proposal and execution could lose pending-release.json).
• PyPI trusted publishing is configured for both bittensor and py-sp-core under the RaoFoundation org.

CI security

• trusted-pr fork gate: sanity-check the if: expression on a real fork PR — jobs that already had their own if: conditions now combine with needs: trusted-pr, and a skipped gate should skip dependents, not pass them.
• Confirm no remaining workflow runs fork-PR code on self-hosted runners (search runs-on: [self-hosted).

SDK / bridge

• Extension bridge token flow: daemon writes the token before the pid file; check the ordering on crash/restart paths, and that older clients (no token) fail with the clear error rather than hanging.
• scrypt caps (n ≤ 2^18, r ≤ 8, p = 1): confirm these accept every keystore format we need to import (PolkadotJS default is n=32768/r=8/p=1; anything nonstandard in the wild will now be rejected).
• Codegen RAW_ONLY additions for Commitments/Contracts: agree these calls should stay raw-only rather than getting intents.

Monorepo mechanics

• The imported histories (sdk/python, website, clones) — confirm we're comfortable with squashed import rather than preserved history.
• Clone-upgrade CI job duration (up to 180 min timeout) and self-hosted runner capacity.
• E2E suite now runs from the in-repo SDK only — confirm dropping the external btcli staging suite from PR CI is intentional and that coverage moved into tests/e2e/.

Not verified locally: full Rust workspace tests, try-runtime, e2e suites, and website build — these run in CI on this PR.

Made with Cursor

[github-actions]

eco-tests changed — indexer review required

This PR modifies files under eco-tests/. and may affect downstream indexing.
cc @evgeny-s — please review manually

Changed files

• eco-tests/Cargo.toml
• eco-tests/src/mock.rs