feat(doh): DoH fallback for DKIM/DMARC TXT records on unsigned domains

[sea-snake]

Summary

PR 4 of the email-recovery stack. Adds the DoH (DNS-over-HTTPS) fallback path for fetching DKIM/DMARC TXT records when a domain isn't DNSSEC-signed (Gmail, Outlook, iCloud, etc.). Stacks on #3878 (DMARC alignment).

• Allowlist-gated outcalls. DohConfig.allowed_domains (deploy/upgrade arg, persisted in PersistentState). Domains not on the list return DomainNotAllowed without touching the network.
• 3-of-5 quorum across 4 jurisdictions. Cloudflare 🇺🇸 + Google 🇺🇸 + Quad9 🇨🇭 + CIRA 🇨🇦 + IIJ 🇯🇵. Strict majority; no single jurisdiction can reach quorum on its own (asserted in unit tests).
• In-flight dedup. Hand-rolled oneshot-style Waker primitive collapses concurrent fetches for the same FQDN to one outcall fan-out.
• IC-correct outcall path. http_request_with_closure with a transform query function that reduces the wire DNS response to its TXT RDATA (drops TTL/transaction-ID drift across replicas, so consensus succeeds). max_response_bytes = 4 KiB, cycle budget mirrors the OIDC discovery outcall.
• Cache hygiene. Stale-pending TTL (PENDING_STALE_AFTER_SECS = 120) recovers from the IC-specific hazard where a trapped post-yield continuation leaves an InFlight entry stuck forever (state across .await is committed, not transactional). Lazy eviction on lookup + opportunistic sweep on publish bound the value-cache size to {unique FQDNs queried in max_cache_age_secs}.

Public API: pub async fn fetch_txt(name, registered_domain) -> Result<Vec<u8>, DohError>. Allowlist match is case-insensitive on the registered domain. Steady-state outcall load for a continuous Gmail flow at default 1h TTL: ~10 outcalls/hr per domain (2 fan-outs × 5 providers), independent of email volume.

PR 8 onwards will wire this into the DKIM/DMARC verifiers from PRs 2/3 alongside the DNSSEC chain from PR 1.

Test plan

• 43 new doh unit tests: parser (round-trip, malformed, NXDOMAIN), bucketing (3-of-5/2-of-5 split-bucket/all-fail/all-malformed), dedup wakeups, stale-pending eviction wakes orphan waiters with AllProvidersFailed, late-publisher no-op after stale takeover, lazy eviction on lookup, sweep on publish, end-to-end fetch_txt with NotConfigured / DomainNotAllowed / case-insensitive allowlist / cache hit-then-refetch / TTL cap / quorum-failure-not-cached.
• Full unit test suite green (408 tests).
• cargo check --target wasm32-unknown-unknown clean.
• cargo clippy --tests -- -D warnings clean.
• cargo fmt --check clean.
• Smoke test on local replica: fetch_txt against a real allowlisted domain, verify provider fan-out + transform + consensus end-to-end (deferred to integration testing once gateway plumbing lands in PR 8).
• Verify IIJ DoH endpoint URL (public.dns.iij.jp) with a real outcall before deploy.

PR Stack

#	PR	Description	Status
0	#3836	Design doc	Open
1	#3838	DNSSEC verifier scaffold	Open
2	#3877	DKIM verifier (RFC 6376)	Open
3	#3878	DMARC alignment (RFC 7489)	Open
4	#3879	DoH fallback	Open
5+6	#3880	Setup flow (storage + smtp_request)	Open
7	#3881	Recovery flow (delegation)	Open
8	#3882	Frontend + feature flag	Open
9	#3883	Deploy/upgrade scripts: dnssec_config + doh_config	Open
10	#3884	Email-recovery UX overhaul	Open

[aterga]

PR Overview (pragmatic lens)
This is a clean, well-scoped addition: a DNS-over-HTTPS (DoH) fallback path for fetching DKIM/DMARC TXT records on domains that lack DNSSEC (the vast majority of consumer email providers). It is gated, resilient, IC-native, and low-overhead. The design choices (allowlist, 3-of-5 cross-jurisdiction quorum, transform-based deterministic outcalls, in-flight deduplication, and cache-with-stale-pending hygiene) directly address the real risks of outcalls on the Internet Computer.

43 new unit tests + clean cargo check/clippy/fmt + explicit test plan for local-replica smoke test in PR 8 make this low-risk to merge once the small items below are addressed. No breaking changes; everything is additive and configuration-gated.

Blockers (must-fix before merge)

None identified. The security posture is already stronger than many production outcall patterns in the IC ecosystem.

Major Security Gaps

None.

• Outcall surface is strictly allowlist-gated on the registered domain (case-insensitive) before any network activity → DomainNotAllowed is returned with zero cycles spent.
• Quorum (3-of-5 across Cloudflare 🇺🇸, Google 🇺🇸, Quad9 🇨🇭, CIRA 🇨🇦, IIJ 🇯🇵) plus the unit-test assertion that “no single jurisdiction can reach quorum on its own” eliminates single-provider or single-country poisoning.
• Response canonicalization via http_request_with_closure + transform query function (drops TTL/transaction-ID variance) guarantees deterministic consensus.
• Cache hygiene explicitly mitigates the IC-specific “trapped continuation after yield” hazard (state is committed, not rolled back).
• Parsing/validation is defensive (invalid DNS names rejected, label-anchored suffix match between queried name and registered domain).

This is materially safer than a naïve http_request to arbitrary DoH endpoints.

Easy-to-Change Improvements (quick wins, <30 min each)

1. Expose the provider list and DoH URLs as constants with comments (in doh/mod.rs or types.rs).

   • Action: Add a const PROVIDERS: &[(&str, &str)] (name + URL) and a short comment explaining the jurisdiction distribution and why two US providers are acceptable given the quorum rule.
   • Why: Reviewers/deployers will want to know exactly which endpoints are called; makes future rotation trivial.

2. Make PENDING_STALE_AFTER_SECS (120 s) a named const at the top of cache.rs with a one-line comment referencing the IC yield hazard.

   • Also add a matching integration-test case that forces a continuation trap (or mocks it) to prove eviction works.

3. Document the exact steady-state outcall cost in the PR description / module docs.

   • You already have it internally (“~10 outcalls/hr per domain at 1 h TTL”). Surface it as a comment on fetch_txt so operators know the cycle budget impact before enabling email recovery at scale.

4. Add one explicit test for the allowlist edge case where the registered domain matches but the queried name does not suffix-match (should be rejected before outcall).

   • The defensive validation is mentioned; just make the test visible so future refactors can’t regress it.

5. Tiny Cargo.toml hygiene (optional but nice): the new deps (futures alloc-only, url, identity_jose) are all justified by the stack, but add a one-line comment next to each explaining why it is pulled in by this PR (avoids “why is this here?” questions later).

Strategic Recommendations

1. Rate-limit / back-pressure story (post-merge, before PR 8 wiring).
   Even though per-domain load is tiny, a popular anchor tenant using Gmail/Outlook could still generate bursts. Consider a simple global token-bucket (or just a last_outcall timestamp + cooldown) in a follow-up PR so one misbehaving recovery flow cannot starve the rest of the canister.

2. Observability hooks.
   Expose a DohMetrics struct (or just increment counters via ic_cdk::api::call::call_perform counters if available) for:

   • cache hit/miss
   • quorum success/failure (and which providers disagreed)
   • allowlist denials
     This will be gold when email recovery goes live.

3. Configuration defaults & upgrade safety.
   The DohConfig fields are optional in both upgrade arg and persistent state — perfect. In the next config PR (or in main.rs init), consider a sensible default allowlist (vec!["gmail.com".to_string(), "outlook.com".to_string(), ...]) or at least document the empty-list behaviour clearly so mainnet upgrades don’t silently disable the fallback.

4. Long-term: consider a hybrid DNSSEC + DoH path (already planned in the stack).
   Once DNSSEC path (PR 1) is wired, the DoH path becomes the “unsigned” branch. Document the decision matrix (DNSSEC → DoH → fail) in one place so the full email-recovery verification story is easy to audit.

5. Testing strategy for the full stack.
   When PR 8 wires this in, add at least one end-to-end test that uses a real Gmail/Outlook test account (with known DKIM/DMARC records) against the local replica. The current unit-test coverage is excellent for the DoH layer; the integration surface is where subtle bugs hide.

Overall verdict: Ship after the five easy items above (mostly documentation + one test). This is a high-quality, defensively-written piece of infrastructure that materially unblocks email recovery for the most common real-world domains. Great work.