feat(deploy): wire dnssec_config + doh_config into beta + prod install args#3855
Closed
sea-snake wants to merge 12 commits into
Closed
feat(deploy): wire dnssec_config + doh_config into beta + prod install args#3855sea-snake wants to merge 12 commits into
sea-snake wants to merge 12 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends Internet Identity’s deployment/upgrade tooling and canister configuration to support the email-recovery feature’s DNSSEC trust anchors (dnssec_config) and DoH fallback allowlist (doh_config). It also introduces/updates substantial backend + interface + frontend + test scaffolding for email recovery, including DNSSEC/DKIM/DMARC verification types and fixtures.
Changes:
- Add
dnssec_config+doh_configto init args/persistent state and wire them into beta/prod deployment & upgrade-proposal scripts (including fetching IANA root anchors). - Add stable storage support for email-recovery credentials and a reverse index (hashed email → anchor), plus authz/activity tracking hooks.
- Add frontend UI (manage + recovery flows), Playwright e2e coverage, and test vectors for DKIM/DNSSEC/DMARC verification paths.
Reviewed changes
Copilot reviewed 108 out of 111 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| test_vectors/dnssec/iana-root-anchors-2026-05.json | Captured IANA root anchor fixture used by DNSSEC tests. |
| test_vectors/dkim/synth-rsa-test1._domainkey.test.example.com.txt | DKIM TXT fixture for synthetic RSA test vectors. |
| test_vectors/dkim/synth-rsa-simple-simple.eml | Synthetic DKIM email fixture for canonicalization tests. |
| test_vectors/dkim/synth-rsa-relaxed-simple.eml | Synthetic DKIM email fixture for relaxed/simple case. |
| test_vectors/dkim/synth-rsa-relaxed-relaxed.eml | Synthetic DKIM email fixture for relaxed/relaxed case. |
| test_vectors/dkim/README.md | Documents DKIM fixture set and regeneration approach. |
| src/internet_identity/tests/integration/main.rs | Registers new integration test module for email recovery. |
| src/internet_identity/tests/integration/config/oidc_configs.rs | Formatting-only adjustment in an OIDC config test. |
| src/internet_identity/src/storage/tests.rs | Extends storage migration tests to include email_recovery. |
| src/internet_identity/src/storage/storable/storable_persistent_state.rs | Persists dnssec_config/doh_config across upgrades. |
| src/internet_identity/src/storage/storable/email_recovery_credential.rs | Adds CBOR-storable email recovery credential type. |
| src/internet_identity/src/storage/storable/email_recovery_address_hash.rs | Adds fixed-size stable-map key for address reverse index. |
| src/internet_identity/src/storage/storable/anchor.rs | Adds email_recovery field to stable anchor schema. |
| src/internet_identity/src/storage/storable.rs | Exposes new storable modules for email recovery. |
| src/internet_identity/src/storage/anchor/tests.rs | Updates anchor tests to include email_recovery. |
| src/internet_identity/src/storage/anchor.rs | Adds email_recovery to in-memory anchor model + conversions. |
| src/internet_identity/src/storage.rs | Adds stable reverse index for email recovery address→anchor. |
| src/internet_identity/src/stats/activity_stats/activity_counter/authn_method_counter.rs | Tracks email-recovery authentications in activity stats. |
| src/internet_identity/src/state.rs | Adds dnssec_config + doh_config to PersistentState. |
| src/internet_identity/src/openid/generic.rs | Formatting-only changes in discovery URL validation. |
| src/internet_identity/src/email_recovery/rng.rs | Introduces lazy-seeded heap PRNG for email-recovery nonces. |
| src/internet_identity/src/email_recovery/remove.rs | Implements remove flow for email recovery credentials. |
| src/internet_identity/src/email_recovery/mod.rs | Declares email recovery module surface and constants. |
| src/internet_identity/src/doh/types.rs | Adds DoH fallback internal types/config + provider list. |
| src/internet_identity/src/dnssec/types.rs | Adds canister-side DNSSEC proof bundle types + errors. |
| src/internet_identity/src/dnssec/test_vectors.rs | Adds JSON loaders and includes DNSSEC test fixtures. |
| src/internet_identity/src/dnssec/mod.rs | Adds DNSSEC verifier module entrypoint + exports. |
| src/internet_identity/src/dnssec/canonical.rs | Implements canonicalization helpers for DNSSEC verification. |
| src/internet_identity/src/dmarc/verify.rs | Adds combined DKIM+DMARC verification orchestration + tests. |
| src/internet_identity/src/dmarc/types.rs | Adds DMARC alignment record/outcome types. |
| src/internet_identity/src/dmarc/test_vectors.rs | Adds end-to-end DMARC tests driven by DKIM fixtures. |
| src/internet_identity/src/dmarc/parse.rs | Adds DMARC TXT parser with RFC-position enforcement. |
| src/internet_identity/src/dmarc/mod.rs | Adds DMARC module wiring and exports. |
| src/internet_identity/src/dmarc/alignment.rs | Adds strict/relaxed domain alignment logic + tests. |
| src/internet_identity/src/dkim/types.rs | Adds DKIM verifier result/error/check types. |
| src/internet_identity/src/dkim/mod.rs | Adds DKIM verifier module wiring and exports. |
| src/internet_identity/src/authz_utils.rs | Extends authorization checks to accept email-recovery principals. |
| src/internet_identity/src/anchor_management.rs | Updates bookkeeping to handle email-recovery auth + last_used. |
| src/internet_identity/local_test_arg.did | Adds deterministic test DNSSEC trust anchor to local args. |
| src/internet_identity/internet_identity.did | Extends public Candid with DNSSEC/DoH config + email recovery API/types. |
| src/internet_identity/Cargo.toml | Adds dependencies for DNSSEC verification + DoH fan-out. |
| src/internet_identity_interface/src/internet_identity/types/doh.rs | Adds interface type for DoH configuration. |
| src/internet_identity_interface/src/internet_identity/types/dnssec.rs | Adds interface types for DNSSEC config + proof bundles. |
| src/internet_identity_interface/src/internet_identity/types/attributes.rs | Formatting-only test change. |
| src/internet_identity_interface/src/internet_identity/types/api_v2.rs | Extends IdentityInfo with email_recovery. |
| src/internet_identity_interface/src/internet_identity/types.rs | Re-exports new types and extends init/authz types. |
| src/internet_identity_interface/src/archive/types.rs | Adds archive operations for email-recovery add/remove. |
| src/frontend/tests/e2e-playwright/utils/renderDnssecTestAnchor.mjs | Generates deterministic DNSSEC trust anchor for tests/workflow. |
| src/frontend/tests/e2e-playwright/routes/emailRecovery.spec.ts | Adds e2e coverage for email recovery gating + end-to-end flow. |
| src/frontend/tests/e2e-playwright/fixtures/recoveryPage.ts | Updates recovery-page fixture button label/selector. |
| src/frontend/tests/e2e-playwright/fixtures/index.ts | Registers email-recovery test fixture. |
| src/frontend/src/routes/vc-flow/index/+page.svelte | Keeps init arg shape in sync (adds dnssec/doh fields). |
| src/frontend/src/routes/(new-styling)/recovery/+page.svelte | Adds “Recover with email” path and delegation sign-in bridging. |
| src/frontend/src/routes/(new-styling)/manage/(authenticated)/(access-and-recovery)/recovery/components/InactiveEmailRecovery.svelte | Adds inactive recovery-email card UI. |
| src/frontend/src/routes/(new-styling)/manage/(authenticated)/(access-and-recovery)/recovery/components/ActiveEmailRecovery.svelte | Adds active recovery-email card UI with last-used rendering. |
| src/frontend/src/routes/(new-styling)/manage/(authenticated)/(access-and-recovery)/recovery/+page.svelte | Wires email-recovery card + setup/remove handlers into manage page. |
| src/frontend/src/lib/utils/iiConnection.test.ts | Updates init defaults to include dnssec/doh fields. |
| src/frontend/src/lib/utils/dnssec/rrsig.ts | Adds browser-side RRSIG parser + DNSKEY key-tag computation. |
| src/frontend/src/lib/utils/dnssec/index.ts | Adds FE DNSSEC assembly entrypoints for skeleton + DKIM leaf. |
| src/frontend/src/lib/utils/dnssec/doh.ts | Adds FE DoH wire-format client (resolver queries). |
| src/frontend/src/lib/stores/authentication.store.ts | Adds emailRecovery as an authentication method variant. |
| src/frontend/src/lib/state/featureFlags.ts | Adds EMAIL_RECOVERY feature flag with beta host override. |
| src/frontend/src/lib/components/wizards/setupEmailRecovery/views/SendMagicEmail.svelte | Adds “send magic email” step UI with countdown + clipboard copy. |
| src/frontend/src/lib/components/wizards/setupEmailRecovery/views/FailedView.svelte | Adds failure UI for setup wizard. |
| src/frontend/src/lib/components/wizards/setupEmailRecovery/views/EnterAddress.svelte | Adds address entry UI for setup wizard. |
| src/frontend/src/lib/components/wizards/setupEmailRecovery/views/Done.svelte | Adds success UI for setup wizard. |
| src/frontend/src/lib/components/wizards/setupEmailRecovery/SetupEmailRecoveryWizard.svelte | Implements setup wizard flow incl. polling + optional DKIM leaf submission. |
| src/frontend/src/lib/components/wizards/setupEmailRecovery/index.ts | Exports setup wizard component. |
| src/frontend/src/lib/components/wizards/recoverWithEmail/views/EnterAddressForRecovery.svelte | Adds address entry UI for recovery wizard. |
| src/frontend/src/lib/components/wizards/recoverWithEmail/index.ts | Exports recovery wizard + success payload type. |
| src/canister_tests/src/api/internet_identity.rs | Adds PocketIC API wrappers for email recovery + smtp_request. |
| src/canister_tests/src/api/archive.rs | Updates compat mapping to account for new archive operations. |
| src/archive/archive.did | Extends archive candid with email-recovery operations. |
| scripts/make-upgrade-proposal | Patches generated init args to inject dnssec/doh config on prod. |
| scripts/fetch-iana-root-anchors.bash | Adds script to fetch/review IANA root anchors and render Candid. |
| scripts/deploy-common.bash | Adds flags and wiring to include dnssec/doh init fields on beta deploys. |
| scripts/capture-dnssec-chain.py | Adds utility to capture DNSSEC chains into test-vector JSON. |
| Cargo.toml | Adds workspace dependencies for DNSSEC verifier crypto crates. |
| Cargo.lock | Locks new Rust dependencies (p256, ed25519-dalek, futures, etc.). |
| .github/workflows/canister-tests.yml | Installs canister in CI with generated test DNSSEC trust anchor. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
sea-snake
force-pushed
the
feat/email-recovery-deploy-scripts
branch
3 times, most recently
from
7955539 to
f953cd2
Compare
This was referenced May 6, 2026
Closed
sea-snake
force-pushed
the
feat/email-recovery-deploy-scripts
branch
from
46401af to
c8c1d40
Compare
2 tasks
sea-snake
force-pushed
the
feat/email-recovery-deploy-scripts
branch
from
c8c1d40 to
03673bf
Compare
…l args Email recovery (PRs dfinity#3838-dfinity#3844) needs two new BE init args set on the canister: `dnssec_config.root_anchors` (the IANA root KSKs the DNSSEC verifier trusts) and `doh_config.allowed_domains` (the mailbox-provider domains that can use the DoH-fallback path). This PR plumbs both through the three deploy scripts. What landed: - **`scripts/fetch-iana-root-anchors.bash`** — new module, sourced by the other deploy scripts. `fetch_and_review_iana_root_anchors`: - `curl`s `data.iana.org/root-anchors/root-anchors.xml` (retry + bounded timeout). - Parses every `<KeyDigest>` block with Node, filtering to those whose `validFrom`/`validUntil` window contains "now" — drops retired keys (Kjqmt7v expired 2019) and keeps the active KSKs (Klajeyz key_tag=20326, Kmyv6jo key_tag=38696 today). - Prints a one-line human summary per entry to stderr (id + key_tag + algo + digest_type + digest preview + validity window). - Prompts for confirmation on /dev/tty (gracefully falls back to "accept default" when there's no controlling terminal, so CI / sandboxes don't deadlock). - Writes Candid `vec record { key_tag = ...; algorithm = ...; digest_type = ...; digest = blob "..."; }` to stdout, ready to drop into `dnssec_config.root_anchors`. - **`scripts/deploy-common.bash`** (consumed by `deploy-pr-to-beta` and `deploy-local-to-beta`): - New `DEFAULT_DOH_ALLOWED_DOMAINS` list — gmail, googlemail, outlook, hotmail, live, icloud, me, mac, yahoo, protonmail, proton.me, pm.me. Adding a domain is operator config, not code; override per-deploy with `--doh-domains <a,b,c,...>`. Empty list = DNSSEC-only, no DoH fallback. - New `--skip-email-recovery-init` flag — when set, leave both fields as `opt null` (preserve previous on-chain values). Default is to fetch fresh anchors + use the curated DoH allowlist, since the staging canisters are still in the "needs first-run init" stage. - `build_be_install_arg` now emits `dnssec_config = ...` and `doh_config = ...` alongside the existing trio (BE id, BE_URL, FE_URL). - **`scripts/make-upgrade-proposal`** (production): didc-assist's interactive flow doesn't know how to type a 32-byte SHA-256 digest as Candid blob, so we *post-process* the args.txt it produces: - After didc-assist writes the file (or the trivial `(null)` when the user accepts the no-update default), call `patch_email_recovery_init_fields`. - The patcher uses Node to find/replace the `dnssec_config` and `doh_config` fields in the Candid arg text — or insert them inside the outermost `record { ... }` if didc-assist didn't emit them — with values from the same IANA fetch + DoH allowlist. The user can opt out with `II_SKIP_EMAIL_RECOVERY_INIT=1` for upgrades where the on-chain values are already correct, and override the DoH allowlist via `II_DOH_DOMAINS=...`. - Both staging and production paths share the patcher; encoded Candid round-trips through `didc encode` cleanly in all three edge cases tested locally: existing fields = null, fields absent, args.txt = `(null)`. The reason we don't bundle the IANA anchors into the wasm is that they roll over every ~7 years; the next rollover becomes a one-line change in the next upgrade arg instead of a code change + recompile.
The original landing of these scripts had `dnssec_config` + `doh_config` populated by default on every upgrade, with a `--skip-email-recovery-init` escape hatch. That's the wrong default — most upgrades happen *after* the canister has been initialized once, and on those the values should stay as `opt null` so the canister preserves what's already on chain. Doing a fresh fetch on every upgrade also burns an outbound HTTP request to data.iana.org each time the deploy script runs, which is needless when the operator hasn't asked for it. Flip the polarity: - `UPDATE_EMAIL_RECOVERY_INIT` defaults to `false`. The fields stay `opt null` unless the operator opts in. - The CLI flag is renamed `--skip-email-recovery-init` → `--update-email-recovery-init` and now opts *in* (sets the global to true). - For `make-upgrade-proposal`, the env var is renamed `II_SKIP_EMAIL_RECOVERY_INIT=1` → `II_UPDATE_EMAIL_RECOVERY_INIT=1`, same polarity flip. - Usage text and the inline doc-comments updated to match. `--doh-domains` is unchanged — it's only meaningful alongside `--update-email-recovery-init`, which the help text now spells out. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…DoH' Two consistency fixes from PR dfinity#3855 review: - `scripts/deploy-common.bash`: `--doh-domains` was using `[ -n "$DOH_DOMAINS_ARG" ]` to detect an explicit override, which collapses 'flag not passed' and 'flag passed with empty string' into the same branch. Documented behaviour says the empty list disables the DoH path entirely. Track the explicit flag with a separate `DOH_DOMAINS_ARG_SET` boolean and key the override branch off that, so `--doh-domains ""` now produces `allowed_domains = vec {};`. - `scripts/make-upgrade-proposal`: same fix for the `II_DOH_DOMAINS` env var. Switched the "is overridden" check from `[ -n ... ]` to `[ -n "${II_DOH_DOMAINS+set}" ]` so an empty value counts as override-with-empty-list. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The original list mixed DNSSEC-signed domains in with the unsigned ones, which is the wrong policy: domains that publish DS records already work via the FE's native DNSSEC walk; putting them on the DoH allowlist would let an unsigned-zone misconfiguration (parent DS removed, recursive resolver still happy to resolve) silently downgrade them to the canister's DoH path. Audited 2026-05 with `dig +short DS <domain>` cross-checked against both Google (8.8.8.8) and Cloudflare (1.1.1.1) resolvers. The corrected list keeps only the 23 major consumer mailbox providers that publish *no* DNSSEC; the four that did (live.com, protonmail.com, proton.me, pm.me) were dropped, and the explicit exclusion is documented inline so the list doesn't drift. Removed (DNSSEC-signed): - live.com (kept hotmail.com / outlook.com / msn.com — those don't publish DS) - protonmail.com / proton.me / pm.me Added (no DNSSEC, broaden coverage to other regions / providers): - msn.com (Microsoft) - ymail.com, aol.com (Yahoo / Verizon Media) - zoho.com, fastmail.com, fastmail.fm, hey.com (other Western providers) - yandex.com, yandex.ru, mail.ru (Russia) - qq.com, 163.com, 126.com (China) - naver.com, daum.net (Korea) Both `scripts/deploy-common.bash` (DEFAULT_DOH_ALLOWED_DOMAINS) and `scripts/make-upgrade-proposal` (default_doh_domains_for_prod) now carry exactly the same 23-domain list. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The 23-domain DoH allowlist was spelled out twice — once as a bash array in `deploy-common.bash` (one domain per line) and once as a comma-separated string in `make-upgrade-proposal` — which is two places to keep in sync next time we want to add or remove a provider. Move the list into a tiny shared file `scripts/default-doh-domains.bash` that both scripts source. `make-upgrade-proposal`'s `default_doh_domains_for_prod` now joins the array with `IFS=,` at use time so the patcher still gets the comma-separated form it expects. The `II_DOH_DOMAINS` and `--doh-domains` overrides are unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
live.com's apex publishes DS, but its DKIM TXT lives behind a CNAME chain that lands in unsigned protection.outlook.com. The DNSSEC chain breaks end-to-end at the cross-zone CNAME, so the verifier can't authenticate the resolution and the domain has to fall through to the DoH path. This is a class of failure: "apex signed" doesn't imply "DKIM signed end-to-end". The audit criterion in the file's comment is corrected to reflect that — what matters is whether the DKIM CNAME chain stays in signed territory, not whether the apex publishes DS. The genuine cross-zone-but-still-signed cases (proton.me → proton.ch, tutanota.com → tutanota.de) stay off the list; the DNSSEC verifier's multi-zone bundle handling (see design §7.2) handles them natively.
The off-chain SMTP gateway routes mail at a domain that varies per deploy: id.ai on prod, beta.id.ai on beta. The canister was hardcoding `id.ai` everywhere — recipient dispatch, the validate query, and the user-facing label returned from prepare — so on the beta canister mail to `register@beta.id.ai` reached the canister but failed the recipient match and was silently dropped. Drop the hardcoded constant. Derive the accepted mailbox domains from `related_origins`, which is already a per-deploy arg the deploy scripts wire through (and the same one used for security headers + the FE's `getPrimaryOrigin`). All entries are treated as equal aliases — recipient dispatch and the `smtp_request_validate` query accept `register@<host>` / `recover@<host>` for any host listed in `related_origins`. So a prod deploy with `id.ai` + the `*.icp0.io` aliases accepts mail at all of them; a beta deploy with `beta.id.ai` accepts that one. Drop the `mailbox` field from `EmailRecoveryChallenge` too. The FE already knows which origin the user is on (`window.location.hostname`), so it pairs that with `register` / `recover` to render the label — each tab automatically shows the alias matching the origin the user is on, and the canister never has to single one out as canonical. Empty / unset `related_origins` → no domains accepted; the canister drops every inbound recipient. Real deploys always configure this. Tests: extended `email_recovery::smtp::tests` with `set_related_origins` helper + 15 cases (single host, multi-alias prod, beta-only, unknown user, wrong domain, no-origins-configured); all pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
After the d3c0cc6 change, recipient acceptance in `recipient_matches` reads exclusively from `mailbox_domains()`, which is sourced from `related_origins` with no fallback. The PocketIC fixtures didn't set `related_origins`, so `register@id.ai` envelopes were silently dropped and tests stalled in `Pending` instead of reaching `RegistrationSucceeded`. Set `related_origins: Some(vec!["https://id.ai".into()])` in both init sites so the `id.ai` recipient domain is accepted.
Staging A was previously deployed with the Postbox PoC (dfinity#3760, never merged), which claimed memory IDs 23 and 24 for `SMTP_POSTBOX` (unbounded values) and `PUSH_SUBSCRIPTION` BTreeMaps. Our PR re-uses ID 23 for `lookup_anchor_with_email_recovery` (32-byte fixed key, StorableAnchorNumber value). When `StableBTreeMap::init` runs against the existing memory page, the BTreeMap magic still matches, so it `load`s the postbox-shaped header and tries to read nodes against our K/V config — overflow-page math mismatches and stable-structures traps with `index out of bounds: len 0 but index 187` at `btreemap/node/io.rs:58`. Workaround: in `Storage::from_memory` (post-upgrade only), zero the first bytes of memory IDs 23 and 24. That fails the magic check, so `StableBTreeMap::init` calls `new` and creates a fresh BTreeMap with our layout. Idempotent so it's safe to keep across multiple upgrades, but it's marked TEMPORARY and should be removed once staging A has been upgraded with this code at least once.
Staging A has been upgraded with the wipe code at least once, so the stale `SMTP_POSTBOX` / `PUSH_SUBSCRIPTION` BTreeMap headers at memory IDs 23 and 24 (from dfinity#3760, the Postbox PoC) are gone. Reverts the workaround from 78d33b7 — fresh canisters and upstream production have never had data at those IDs, so no further deploy paths need it.
TEMPORARY (PR dfinity#3855). Adds a thread-local ring buffer and an `er_dbg!` macro, sprinkles trace lines through the email-recovery + DoH flow, and exposes the buffer via a new anonymous query `email_recovery_debug_logs : () -> (vec text)` plus an `email_recovery_debug_clear : () -> ()` companion update. Lost on every upgrade and capped at 2000 lines, so the heap can't grow unbounded. Anonymous because the staging tester needs to read it without holding canister credentials. Remove together with the `email_recovery_debug_log` Rust module and every `crate::er_dbg!` call-site once the staging outlook.com flow is confirmed working end-to-end. Trace points cover: - doh::fetch_txt — entry, allowlist gate, cache decision, per-provider outcall result, quorum result. - doh::prod::outcall — per-provider URL, status, body length, body preview (first 60 ASCII bytes), rejection code on failure. - email_recovery::prepare_common — entry, path picker decision, allowlist outcome, issued nonce. - email_recovery::handle_smtp_request — entry envelope, recipient dispatch (Setup/Recovery/DROP with mailbox_domains for context), nonce extraction, DNSSEC vs DoH path branch. - email_recovery::verify_setup_email_doh — DKIM/DMARC FQDNs, fetch outcomes, DMARC verdict, From-vs-claimed mismatch. - email_recovery::submit_dkim_leaf — entry shape, run_submit verdict.
`semicolon_in_expressions_from_macros` (future-incompatible) trips when `er_dbg!(...)` lands in an expression position — the trailing `;` inside the macro body makes the expansion a statement, so a match arm or closure that uses the macro without its own trailing semicolon errors. Removing the body's `;` keeps the expansion an expression; call-sites that want statement form keep their own trailing `;` outside the invocation, which is the idiomatic shape for `println!`-family macros.
sea-snake
force-pushed
the
feat/email-recovery-deploy-scripts
branch
from
03673bf to
5352b7a
Compare
Contributor
Author
|
Replaced with a new PR from an upstream branch (enables direct collaboration). Same content, new PR number. |
This was referenced May 12, 2026
Merged
sea-snake
added a commit
that referenced
this pull request
May 12, 2026
Two follow-ups from PR #3855 review on the reverse address index: - `Storage::update_email_recovery_lookup` now returns `Result<(), AnchorNumber>` and rejects when the new address is already bound to a *different* anchor (Err carries the existing anchor). Same-anchor rebinds remain idempotent so a user retrying the wizard against their own anchor still works. - A new `StorageError::EmailRecoveryAddressAlreadyBound { existing_anchor }` variant carries that conflict back through `Storage::write`. `email_recovery::smtp::bind_credential` matches it and surfaces the user-facing `EmailRecoveryError::AddressAlreadyRegistered` instead of the InternalCanisterError catch-all the previous `format!("write anchor: {e:?}")` produced. - `StorableEmailRecoveryAddressHash::from_bytes` no longer panics on unexpected input. Switched to the same `slice_to_bounded_32` helper `StorableApplication`'s hash uses, so corrupted stable memory zero-pads / truncates to 32 instead of trapping mid-call. 444 unit tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
sea-snake
added a commit
that referenced
this pull request
May 12, 2026
…DoH' Two consistency fixes from PR #3855 review: - `scripts/deploy-common.bash`: `--doh-domains` was using `[ -n "$DOH_DOMAINS_ARG" ]` to detect an explicit override, which collapses 'flag not passed' and 'flag passed with empty string' into the same branch. Documented behaviour says the empty list disables the DoH path entirely. Track the explicit flag with a separate `DOH_DOMAINS_ARG_SET` boolean and key the override branch off that, so `--doh-domains ""` now produces `allowed_domains = vec {};`. - `scripts/make-upgrade-proposal`: same fix for the `II_DOH_DOMAINS` env var. Switched the "is overridden" check from `[ -n ... ]` to `[ -n "${II_DOH_DOMAINS+set}" ]` so an empty value counts as override-with-empty-list. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
sea-snake
added a commit
that referenced
this pull request
May 12, 2026
TEMPORARY (PR #3855). Adds a thread-local ring buffer and an `er_dbg!` macro, sprinkles trace lines through the email-recovery + DoH flow, and exposes the buffer via a new anonymous query `email_recovery_debug_logs : () -> (vec text)` plus an `email_recovery_debug_clear : () -> ()` companion update. Lost on every upgrade and capped at 2000 lines, so the heap can't grow unbounded. Anonymous because the staging tester needs to read it without holding canister credentials. Remove together with the `email_recovery_debug_log` Rust module and every `crate::er_dbg!` call-site once the staging outlook.com flow is confirmed working end-to-end. Trace points cover: - doh::fetch_txt — entry, allowlist gate, cache decision, per-provider outcall result, quorum result. - doh::prod::outcall — per-provider URL, status, body length, body preview (first 60 ASCII bytes), rejection code on failure. - email_recovery::prepare_common — entry, path picker decision, allowlist outcome, issued nonce. - email_recovery::handle_smtp_request — entry envelope, recipient dispatch (Setup/Recovery/DROP with mailbox_domains for context), nonce extraction, DNSSEC vs DoH path branch. - email_recovery::verify_setup_email_doh — DKIM/DMARC FQDNs, fetch outcomes, DMARC verdict, From-vs-claimed mismatch. - email_recovery::submit_dkim_leaf — entry shape, run_submit verdict.
sea-snake
added a commit
that referenced
this pull request
May 12, 2026
Two follow-ups from PR #3855 review on the reverse address index: - `Storage::update_email_recovery_lookup` now returns `Result<(), AnchorNumber>` and rejects when the new address is already bound to a *different* anchor (Err carries the existing anchor). Same-anchor rebinds remain idempotent so a user retrying the wizard against their own anchor still works. - A new `StorageError::EmailRecoveryAddressAlreadyBound { existing_anchor }` variant carries that conflict back through `Storage::write`. `email_recovery::smtp::bind_credential` matches it and surfaces the user-facing `EmailRecoveryError::AddressAlreadyRegistered` instead of the InternalCanisterError catch-all the previous `format!("write anchor: {e:?}")` produced. - `StorableEmailRecoveryAddressHash::from_bytes` no longer panics on unexpected input. Switched to the same `slice_to_bounded_32` helper `StorableApplication`'s hash uses, so corrupted stable memory zero-pads / truncates to 32 instead of trapping mid-call. 444 unit tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
sea-snake
added a commit
that referenced
this pull request
May 12, 2026
…DoH' Two consistency fixes from PR #3855 review: - `scripts/deploy-common.bash`: `--doh-domains` was using `[ -n "$DOH_DOMAINS_ARG" ]` to detect an explicit override, which collapses 'flag not passed' and 'flag passed with empty string' into the same branch. Documented behaviour says the empty list disables the DoH path entirely. Track the explicit flag with a separate `DOH_DOMAINS_ARG_SET` boolean and key the override branch off that, so `--doh-domains ""` now produces `allowed_domains = vec {};`. - `scripts/make-upgrade-proposal`: same fix for the `II_DOH_DOMAINS` env var. Switched the "is overridden" check from `[ -n ... ]` to `[ -n "${II_DOH_DOMAINS+set}" ]` so an empty value counts as override-with-empty-list. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
sea-snake
added a commit
that referenced
this pull request
May 12, 2026
TEMPORARY (PR #3855). Adds a thread-local ring buffer and an `er_dbg!` macro, sprinkles trace lines through the email-recovery + DoH flow, and exposes the buffer via a new anonymous query `email_recovery_debug_logs : () -> (vec text)` plus an `email_recovery_debug_clear : () -> ()` companion update. Lost on every upgrade and capped at 2000 lines, so the heap can't grow unbounded. Anonymous because the staging tester needs to read it without holding canister credentials. Remove together with the `email_recovery_debug_log` Rust module and every `crate::er_dbg!` call-site once the staging outlook.com flow is confirmed working end-to-end. Trace points cover: - doh::fetch_txt — entry, allowlist gate, cache decision, per-provider outcall result, quorum result. - doh::prod::outcall — per-provider URL, status, body length, body preview (first 60 ASCII bytes), rejection code on failure. - email_recovery::prepare_common — entry, path picker decision, allowlist outcome, issued nonce. - email_recovery::handle_smtp_request — entry envelope, recipient dispatch (Setup/Recovery/DROP with mailbox_domains for context), nonce extraction, DNSSEC vs DoH path branch. - email_recovery::verify_setup_email_doh — DKIM/DMARC FQDNs, fetch outcomes, DMARC verdict, From-vs-claimed mismatch. - email_recovery::submit_dkim_leaf — entry shape, run_submit verdict.
sea-snake
added a commit
that referenced
this pull request
May 12, 2026
…DoH' Two consistency fixes from PR #3855 review: - `scripts/deploy-common.bash`: `--doh-domains` was using `[ -n "$DOH_DOMAINS_ARG" ]` to detect an explicit override, which collapses 'flag not passed' and 'flag passed with empty string' into the same branch. Documented behaviour says the empty list disables the DoH path entirely. Track the explicit flag with a separate `DOH_DOMAINS_ARG_SET` boolean and key the override branch off that, so `--doh-domains ""` now produces `allowed_domains = vec {};`. - `scripts/make-upgrade-proposal`: same fix for the `II_DOH_DOMAINS` env var. Switched the "is overridden" check from `[ -n ... ]` to `[ -n "${II_DOH_DOMAINS+set}" ]` so an empty value counts as override-with-empty-list. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
sea-snake
added a commit
that referenced
this pull request
May 12, 2026
TEMPORARY (PR #3855). Adds a thread-local ring buffer and an `er_dbg!` macro, sprinkles trace lines through the email-recovery + DoH flow, and exposes the buffer via a new anonymous query `email_recovery_debug_logs : () -> (vec text)` plus an `email_recovery_debug_clear : () -> ()` companion update. Lost on every upgrade and capped at 2000 lines, so the heap can't grow unbounded. Anonymous because the staging tester needs to read it without holding canister credentials. Remove together with the `email_recovery_debug_log` Rust module and every `crate::er_dbg!` call-site once the staging outlook.com flow is confirmed working end-to-end. Trace points cover: - doh::fetch_txt — entry, allowlist gate, cache decision, per-provider outcall result, quorum result. - doh::prod::outcall — per-provider URL, status, body length, body preview (first 60 ASCII bytes), rejection code on failure. - email_recovery::prepare_common — entry, path picker decision, allowlist outcome, issued nonce. - email_recovery::handle_smtp_request — entry envelope, recipient dispatch (Setup/Recovery/DROP with mailbox_domains for context), nonce extraction, DNSSEC vs DoH path branch. - email_recovery::verify_setup_email_doh — DKIM/DMARC FQDNs, fetch outcomes, DMARC verdict, From-vs-claimed mismatch. - email_recovery::submit_dkim_leaf — entry shape, run_submit verdict.
pull Bot
pushed a commit
to mikeyhodl/internet-identity
that referenced
this pull request
May 12, 2026
…iring (dfinity#3838) ## Summary First PR in the email-recovery stack (`docs/ongoing/email-recovery.md` §10 Phase 0). Lands a working RFC-4035-compliant DNSSEC verifier for caller-supplied DNS proof bundles, plus the trust-anchor wiring that drives it. PR 2 (DKIM verifier) and PRs 4–9 (storage + recovery methods) build on this. ## What's in this PR ### Verifier core - New `dnssec/` module under `src/internet_identity/src/`: - `types.rs` — `DnsProofBundle`, `SignedRRset`, `DelegationLink`, `Rrsig`, `DnsName`, `DnssecError`, `VerifiedRecord`. - `canonical.rs` — owner-name canonicalization, RR canonical form, RRSIG signed-data construction (RFC 4034 §3.1.8.1, §6.2, §6.3), DS digest input. - `signature.rs` — algorithm dispatch + DS digest matching (SHA-256). - `verify.rs` — four-step algorithm (root anchor match → chain walk → leaf RRSIG → freshness). - Algorithm coverage (RFC 8624 MUST set): - **alg 8** — RSA-SHA256 (RFC 5702): root, com., most legacy zones. - **alg 13** — ECDSA-P256-SHA256 (RFC 6605): most TLDs, Cloudflare, modern zones. - **alg 15** — Ed25519 (RFC 8080): rare in production but mandatory. - Anything else returns `UnsupportedAlgorithm`. ### Wiring - New `DnssecConfig` and `DnssecRootAnchor` types in `internet_identity_interface`, exposed at the top of `InternetIdentityInit` as `dnssec_config: opt opt DnssecConfig` (set/clear semantics matching `analytics_config` and `dummy_auth`). - Trust-anchor list plumbed through `init`/`post_upgrade` into `PersistentState.dnssec_config` (and `StorablePersistentState` for cross-upgrade persistence). - `internet_identity.did` updated. ### Tests 13 unit tests in `dnssec/` covering: - Real cloudflare.com chain verifies end-to-end (exercises alg 8 at root and alg 13 at com → cloudflare.com → leaf). - Empty trust-anchor list rejected with `NoTrustAnchors`. - Wrong trust anchor rejected with `RootAnchorMismatch`. - Flipped byte in root DNSKEY → `RootAnchorMismatch` or `BadSignature`. - Flipped byte in leaf signature → `BadSignature`. - Stale signature (clock advanced past expiration) → `StaleOrFutureSignature`. - Unsupported algorithm (alg 5 / RSA-SHA1) → `UnsupportedAlgorithm(5)`. - Plus canonical-encoding + RFC 3110 RSA key parsing unit tests. ### Test infrastructure - `test_vectors/dnssec/cloudflare-com-2026-05.json` — real DoH-captured chain (root DNSKEY + 2 delegation links + leaf TXT). - `test_vectors/dnssec/iana-root-anchors-2026-05.json` — IANA root KSK trust anchors (Klajeyz/2017 + Kmyv6jo/2024). - `scripts/capture-dnssec-chain.py` — reproducible capture script using dnspython + DoH wire format. Tests use a frozen now from the capture's metadata so freshness checks stay stable indefinitely. ### New deps - `domain` (NLnet Labs, pure Rust) — referenced in docstrings for canonicalisation primitives; signature verification is hand-rolled on top of RustCrypto. - `p256` — ECDSA P-256 verification. - `ed25519-dalek` — Ed25519 verification. All three build cleanly for wasm32-unknown-unknown. ## What's deferred to later PRs in the stack - Captures for additional providers (proton.me, protonmail.com, tutanota.com — gmail.com / icloud.com / outlook.com / fastmail.com don't sign with DNSSEC; this is acknowledged in design doc §7.6). - Synthetic Ed25519 (alg 15) test vector — most production zones are alg 8 or 13; alg 15 is structurally exercised by the dispatch logic but doesn't have a real captured chain in this PR. ## Test plan - [x] `cargo check -p internet_identity --target wasm32-unknown-unknown` — clean (no warnings). - [x] `cargo test -p internet_identity --bin internet_identity` — 238 tests pass (was 227 pre-PR). - [x] `cargo test -p internet_identity_interface --lib` — 42 tests pass. - [x] `cargo clippy -p internet_identity --bin internet_identity --tests -- -D warnings` — clean. - [x] `cargo fmt --check` — clean (modulo a pre-existing diff in attributes.rs unrelated to this PR). - [x] CI on dfinity#3838 — fully green. ## Design doc https://github.com/sea-snake/internet-identity/blob/design/email-recovery/docs/ongoing/email-recovery.md (PR pending review on dfinity/internet-identity) ## Stack This is PR 1 of a 12-PR series. Subsequent PRs: - **PR 2** — mail-auth-backed DKIM verifier, consuming DnsProofBundle from this PR. - **PR 3** — DMARC alignment. - **PRs 4–9** — storage + Candid + behavior for email recovery. - **PRs 10–12** — frontend (DoH walker, Manage UI, recovery wizard). ## PR Stack | # | PR | Description | Status | |---|---|---|---| | 0 | [dfinity#3836](dfinity#3836) | Design doc | Open | | 1 | [dfinity#3838](dfinity#3838) | DNSSEC verifier scaffold | Open | | 2 | [dfinity#3839](dfinity#3839) | DKIM verifier (RFC 6376) | Open | | 3 | [dfinity#3840](dfinity#3840) | DMARC alignment (RFC 7489) | Open | | 4 | [dfinity#3841](dfinity#3841) | DoH fallback | Open | | 5+6 | [dfinity#3842](dfinity#3842) | Setup flow (storage + smtp_request) | Open | | 7 | [dfinity#3843](dfinity#3843) | Recovery flow (delegation) | Open | | 8 | [dfinity#3844](dfinity#3844) | Frontend + feature flag | Open | | 9 | [dfinity#3855](dfinity#3855) | Deploy/upgrade scripts: dnssec_config + doh_config | Open | | 10 | [dfinity#3857](dfinity#3857) | Email-recovery UX overhaul | Open |
sea-snake
added a commit
that referenced
this pull request
May 13, 2026
Two follow-ups from PR #3855 review on the reverse address index: - `Storage::update_email_recovery_lookup` now returns `Result<(), AnchorNumber>` and rejects when the new address is already bound to a *different* anchor (Err carries the existing anchor). Same-anchor rebinds remain idempotent so a user retrying the wizard against their own anchor still works. - A new `StorageError::EmailRecoveryAddressAlreadyBound { existing_anchor }` variant carries that conflict back through `Storage::write`. `email_recovery::smtp::bind_credential` matches it and surfaces the user-facing `EmailRecoveryError::AddressAlreadyRegistered` instead of the InternalCanisterError catch-all the previous `format!("write anchor: {e:?}")` produced. - `StorableEmailRecoveryAddressHash::from_bytes` no longer panics on unexpected input. Switched to the same `slice_to_bounded_32` helper `StorableApplication`'s hash uses, so corrupted stable memory zero-pads / truncates to 32 instead of trapping mid-call. 444 unit tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
sea-snake
added a commit
that referenced
this pull request
May 13, 2026
…DoH' Two consistency fixes from PR #3855 review: - `scripts/deploy-common.bash`: `--doh-domains` was using `[ -n "$DOH_DOMAINS_ARG" ]` to detect an explicit override, which collapses 'flag not passed' and 'flag passed with empty string' into the same branch. Documented behaviour says the empty list disables the DoH path entirely. Track the explicit flag with a separate `DOH_DOMAINS_ARG_SET` boolean and key the override branch off that, so `--doh-domains ""` now produces `allowed_domains = vec {};`. - `scripts/make-upgrade-proposal`: same fix for the `II_DOH_DOMAINS` env var. Switched the "is overridden" check from `[ -n ... ]` to `[ -n "${II_DOH_DOMAINS+set}" ]` so an empty value counts as override-with-empty-list. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
sea-snake
added a commit
that referenced
this pull request
May 13, 2026
TEMPORARY (PR #3855). Adds a thread-local ring buffer and an `er_dbg!` macro, sprinkles trace lines through the email-recovery + DoH flow, and exposes the buffer via a new anonymous query `email_recovery_debug_logs : () -> (vec text)` plus an `email_recovery_debug_clear : () -> ()` companion update. Lost on every upgrade and capped at 2000 lines, so the heap can't grow unbounded. Anonymous because the staging tester needs to read it without holding canister credentials. Remove together with the `email_recovery_debug_log` Rust module and every `crate::er_dbg!` call-site once the staging outlook.com flow is confirmed working end-to-end. Trace points cover: - doh::fetch_txt — entry, allowlist gate, cache decision, per-provider outcall result, quorum result. - doh::prod::outcall — per-provider URL, status, body length, body preview (first 60 ASCII bytes), rejection code on failure. - email_recovery::prepare_common — entry, path picker decision, allowlist outcome, issued nonce. - email_recovery::handle_smtp_request — entry envelope, recipient dispatch (Setup/Recovery/DROP with mailbox_domains for context), nonce extraction, DNSSEC vs DoH path branch. - email_recovery::verify_setup_email_doh — DKIM/DMARC FQDNs, fetch outcomes, DMARC verdict, From-vs-claimed mismatch. - email_recovery::submit_dkim_leaf — entry shape, run_submit verdict.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scripts/fetch-iana-root-anchors.bash— fetches IANA KSK trust anchors from data.iana.org/root-anchors/root-anchors.xml, filters to currently-valid entries, prints a one-line review per anchor, prompts for confirmation, and emits Candidvec record { key_tag, algorithm, digest_type, digest }ready to drop intodnssec_config.root_anchors.scripts/deploy-common.bash(used bydeploy-pr-to-beta+deploy-local-to-beta):build_be_install_argcan now also emitdnssec_configanddoh_configalongside the existing trio. Opt-in behind--update-email-recovery-init— without the flag both fields stayopt null(preserve previous on-chain values), which is what most upgrades want once the canister has been initialized once. Companion--doh-domains <csv>overrides the curated allowlist (empty string = disable DoH entirely).scripts/make-upgrade-proposal(production): didc-assist can't type 32-byte SHA-256 digests as Candid blobs, so we post-process the args.txt it produces to inject the IANA-fetched + DoH-allowlist values. Opt-in viaII_UPDATE_EMAIL_RECOVERY_INIT=1. CompanionII_DOH_DOMAINS=...overrides the allowlist (empty string = disable DoH entirely; the env-var presence check honoursset-but-emptydistinct fromunset).scripts/default-doh-domains.bash, sourced by both deploy-common.bash and make-upgrade-proposal so the two stay in sync. Allowlist criterion is 'DKIM resolution can't be authenticated end-to-end via DNSSEC' — which catches both apex-unsigned domains (Gmail, Outlook, iCloud, Yahoo) AND domains whose apex is signed but whose DKIM CNAMEs into unsigned territory (live.com →protection.outlook.com).The reason we don't bundle the IANA anchors into the wasm is that they roll over every ~7 years; the next rollover becomes a one-line change in the next upgrade arg instead of a code change + recompile.
PR Stack
See the design doc (#3836) for the architecture; the deploy-arg approach is described in §7.5 (root anchors) and §7.6 (DoH allowlist + audit criterion that catches the live.com class).
Test plan
./scripts/deploy-pr-to-beta --helpshows the new--update-email-recovery-initand--doh-domainsflags./scripts/deploy-pr-to-beta -sa -be --dry-run <PR>(default) prints anicp installcommand withdnssec_configanddoh_configabsent — both fields stayopt nullto preserve previous on-chain values./scripts/deploy-pr-to-beta -sa -be --dry-run --update-email-recovery-init <PR>populates both fields with IANA-fetched anchors + the curated DoH list, and the install arg encodes viadidc encode -t '(opt InternetIdentityInit)'./scripts/deploy-pr-to-beta -sa -be --dry-run --update-email-recovery-init --doh-domains gmail.com,outlook.com <PR>produces an install arg with only those two domains in the allowlist./scripts/deploy-pr-to-beta -sa -be --dry-run --update-email-recovery-init --doh-domains '' <PR>produces an install arg with an empty allowlist (DoH disabled, DNSSEC-only)./scripts/deploy-local-to-beta -sa -be --dry-runworks analogouslymake-upgrade-proposal(withoutII_UPDATE_EMAIL_RECOVERY_INIT) leaves the args.txt unchanged after didc-assistII_UPDATE_EMAIL_RECOVERY_INIT=1 make-upgrade-proposalpopulatesdnssec_config+doh_configfrom IANA + the curated allowlist, and the encoded*.binfiles validateII_UPDATE_EMAIL_RECOVERY_INIT=1 II_DOH_DOMAINS='' make-upgrade-proposalproduces an install arg with an emptyallowed_domainsvec🤖 Generated with Claude Code