From 9d2939f7fb420e749c2f63637d7d7b0f56848058 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Mon, 16 Mar 2026 22:32:33 -0700 Subject: [PATCH 1/4] Add Server-Side Cookie (SSC) product requirements document --- docs/internal/ssc-prd.md | 634 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 634 insertions(+) create mode 100644 docs/internal/ssc-prd.md diff --git a/docs/internal/ssc-prd.md b/docs/internal/ssc-prd.md new file mode 100644 index 00000000..a33fb3d6 --- /dev/null +++ b/docs/internal/ssc-prd.md @@ -0,0 +1,634 @@ +# Product Requirements: Server-Side Cookie (SSC) + +**Status:** Draft +**Author:** Trusted Server Product +**Last updated:** 2026-03-12 + +--- + +## Table of Contents + +1. [Overview](#1-overview) +2. [Problem Statement](#2-problem-statement) +3. [Goals and Non-Goals](#3-goals-and-non-goals) +4. [Target Customers](#4-target-customers) +5. [TS Lite Deployment Mode](#5-ts-lite-deployment-mode) +6. [SSC Identity and Cookie Structure](#6-ssc-identity-and-cookie-structure) +7. [Consent Lifecycle](#7-consent-lifecycle) +8. [KV Store Identity Graph](#8-kv-store-identity-graph) +9. [Pixel Sync Endpoint](#9-pixel-sync-endpoint) +10. [S2S Batch Sync API](#10-s2s-batch-sync-api) +11. [Bidstream Decoration](#11-bidstream-decoration) +12. [Configuration](#12-configuration) +13. [Documentation Updates](#13-documentation-updates) +14. [Open Questions](#14-open-questions) +15. [Success Metrics](#15-success-metrics) + +--- + +## 1. Overview + +Server-Side Cookie (SSC) is a stable, privacy-respecting user identity mechanism built into Trusted Server. It replaces the existing SyntheticID system with a cleaner signal (IP address + publisher salt only), a consent-aware lifecycle, a server-side identity graph backed by Fastly KV Store, and a standalone "TS Lite" deployment mode that allows SSPs, DSPs, identity providers, and publishers to adopt SSC without deploying the full Trusted Server feature set. + +SSC runs at a publisher-controlled first-party subdomain (e.g., `ssc.publisher.com`), sets a cookie scoped to the publisher's apex domain, and optionally orchestrates real-time bidding or decorates outbound ad requests with resolved identity signals from configured partners. + +--- + +## 2. Problem Statement + +### 2.1 SyntheticID signal degradation + +The current SyntheticID uses User-Agent, Accept-Language, Accept-Encoding, and IP address as HMAC inputs. Each of these signals is eroding: + +- **User-Agent reduction**: Chrome's UA freeze has eliminated OS version and minor browser version. The UA string no longer meaningfully differentiates users. +- **Accept-Language homogenization**: Browser defaults increasingly converge, reducing entropy. +- **IPv6 privacy extensions**: Modern operating systems rotate the interface ID portion of IPv6 addresses on a per-session or daily basis, causing SyntheticID mismatches for returning users. + +The result is degrading match rates and false new-user rates on browsers where these signals change. + +### 2.2 No consent enforcement + +SyntheticID is created unconditionally. There is no mechanism to check TCF (EU/UK) or GPP (US) consent before creating the ID. This is a compliance gap that must be closed before SSC can be offered as a product to regulated publishers. + +### 2.3 Adoption blocked by full TS requirement + +SSPs, DSPs, and identity providers want the identity and sync capabilities of Trusted Server without the JS injection pipeline, HTML processing, proxy routing, and auction orchestration that full TS requires. There is no lightweight deployment path today, which blocks a large class of potential adopters. + +--- + +## 3. Goals and Non-Goals + +### Goals + +- Replace SyntheticID's unstable browser signal inputs with IP address + publisher salt only +- Enforce TCF and GPP consent before creating or maintaining the SSC +- Implement real-time consent withdrawal: delete cookie and KV entry when consent is revoked +- Build a server-side identity graph in Fastly KV Store that accumulates resolved partner IDs over time +- Provide two KV write paths: real-time pixel sync redirects and S2S batch push from partners +- Expose two bidstream integration modes: header decoration (`/identify`) and full auction orchestration (`/auction`) +- Enable a "TS Lite" deployment mode via runtime TOML feature flags so SSC can run without the full TS feature surface + +### Non-Goals + +- Replacing the publisher's consent management platform (CMP): SSC reads and enforces consent signals; it does not generate them +- Building a data management platform (DMP): SSC stores resolved partner IDs as a sync spine, not audience segments +- Backward compatibility with SyntheticID: SSC uses a different cookie name, header name, and ID generation method. No migration path is provided +- Real-time user matching across unrelated domains (cross-site tracking) +- Data deletion framework: out of scope for this PRD; flagged for a follow-on document + +--- + +## 4. Target Customers + +| Customer type | Deployment mode | Primary value | +| -------------------- | ------------------------------ | ------------------------------------------------------------------- | +| Publisher (full TS) | Full TS + SSC enabled | Consent-aware first-party ID, bidstream enrichment, identity graph | +| Publisher (SSC only) | TS Lite at `ssc.publisher.com` | First-party cookie at apex domain, identity sync | +| SSP | TS Lite | Pixel sync endpoint to build match table against SSC hash | +| DSP | TS Lite | S2S batch API to push/receive ID mappings, enriched bid requests | +| Identity provider | TS Lite | Register as a partner, sync resolved IDs into the KV identity graph | + +--- + +## 5. TS Lite Deployment Mode + +### 5.1 Concept + +TS Lite is a runtime configuration of the existing Trusted Server binary. It is not a separate binary or separate codebase. A publisher (or SSP/DSP deploying on behalf of a publisher) creates a Fastly service pointing to a subdomain — typically `ssc.publisher.com` — and deploys the standard TS WASM binary with a `trusted-server.toml` that disables all routes except SSC-related functionality. + +### 5.2 Route surface in TS Lite + +| Route | Full TS | TS Lite | +| -------------------------------------- | -------- | ----------------------- | +| `GET /static/tsjs=` | Enabled | Disabled | +| `POST /auction` | Enabled | Optional (configurable) | +| `GET /first-party/proxy` | Enabled | Disabled | +| `GET /first-party/click` | Enabled | Disabled | +| `POST /first-party/sign` | Enabled | Disabled | +| `GET /first-party/proxy-rebuild` | Enabled | Disabled | +| HTML injection pipeline | Enabled | Disabled | +| GTM integration | Enabled | Disabled | +| `GET /sync` | Disabled | **Enabled** | +| `GET /identify` | Disabled | **Enabled** | +| `POST /api/v1/sync` | Disabled | **Enabled** | +| `GET /.well-known/trusted-server.json` | Enabled | Enabled | + +When a disabled route is requested, TS returns `404` with the header `X-ts-error: feature-disabled`. + +### 5.3 Cookie domain and subdomain setup + +The publisher points a subdomain of their choosing (e.g., `ssc`) via DNS CNAME to their Fastly service. They configure `publisher.domain = "publisher.com"` in `trusted-server.toml`. Trusted Server derives `cookie_domain = ".publisher.com"` from this setting and sets the SSC cookie with that domain attribute. + +This gives the cookie read access across all subdomains of `publisher.com` — including `www.publisher.com` — without requiring a separate verification step. The publisher's control over their DNS and Fastly service implicitly proves TLD+1 ownership, following the same trust model as the existing `publisher.cookie_domain` setting. + +**Constraint:** A publisher cannot configure a cookie domain outside their declared `publisher.domain`. Attempting to set `cookie_domain = ".otherdomain.com"` is rejected at startup validation. + +### 5.4 Safari and browser compatibility + +The SSC is set as an HTTP `Set-Cookie` response header (not via JavaScript). For server-set cookies on first-party publisher domains that are not classified as cross-site trackers by Safari's ITP, the effective maximum lifetime is 1 year — the same as the configured `Max-Age`. Since `ssc.publisher.com` is a publisher-owned domain, it is unlikely to be classified as a tracker. + +The ITP interaction for users who arrive exclusively via third-party sync pixel redirects (where `ssc.publisher.com` may be seen as a cross-site recipient) will be monitored post-launch. A cookie refresh strategy — re-issuing `Set-Cookie` on every same-site organic request — is deferred pending production data. + +--- + +## 6. SSC Identity and Cookie Structure + +### 6.1 ID generation + +The SSC is generated by HMAC-SHA256 of a fixed input set, using a publisher-specific secret key. + +**Inputs (IP address + salt only):** + +| Input | Value | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------- | +| IP address | IPv4 as-is; IPv6 summarized to /64 prefix (first 4 hextets) — discards rotating interface ID. On dual-stack, IPv6 is preferred. | +| Secret key | Publisher-specific salt, configured in `trusted-server.toml` | + +**Removed from SyntheticID:** + +- `User-Agent` +- `Accept-Language` +- `Accept-Encoding` +- Handlebars template (input is now fixed, not configurable) + +**Output format (unchanged from SyntheticID):** + +``` +{64-character hex HMAC-SHA256}.{6-character random alphanumeric suffix} +``` + +The 64-character prefix is the stable, deterministic portion used as the KV store key. The 6-character suffix is random, regenerated each time a fresh SSC is created. Once an SSC is set in a cookie, the full value (prefix + suffix) is preserved on subsequent requests. + +**IPv6 /64 prefix rationale:** The first 64 bits of an IPv6 address identify the network prefix assigned by the ISP or home router. The remaining 64 bits (the interface ID) are rotated by privacy extensions on most modern operating systems. Using only the /64 prefix produces a stable hash for returning users while discarding the rotating portion that would cause false new-user signals. + +### 6.2 Cookie attributes + +| Attribute | Value | +| --------- | ----------------------------------------------------------------------------------------- | +| Name | `ts-ssc` | +| Domain | `.publisher.com` (derived from `publisher.domain` in TOML) | +| Path | `/` | +| Secure | Yes | +| SameSite | `Lax` | +| Max-Age | `31536000` (1 year) | +| HttpOnly | No — JavaScript on `www.publisher.com` may need to read the value for ad stack decoration | + +### 6.3 Response header + +The SSC value is also set as a response header for server-side consumers: + +``` +X-ts-ssc: +``` + +This header is internal to Trusted Server and is stripped before proxying requests to downstream backends, consistent with how other `X-ts-*` headers are handled. + +### 6.4 Retrieval priority + +On each request, Trusted Server looks for an existing SSC in this order: + +1. `X-ts-ssc` request header (set by TS on a prior response, forwarded by the publisher's infrastructure) +2. `ts-ssc` cookie +3. Generate fresh SSC (subject to consent check — see Section 7) + +### 6.5 No backward compatibility with SyntheticID + +SSC uses a different cookie name (`ts-ssc` vs `synthetic_id`), a different header name (`X-ts-ssc` vs `x-synthetic-id`), and a different ID generation method. No fallback to reading the `synthetic_id` cookie is provided. SyntheticID code remains in full TS and continues to function; SSC is a parallel system. + +--- + +## 7. Consent Lifecycle + +Consent enforcement is a core requirement of SSC. The system must not create or maintain an SSC for users who have not given consent, and must actively revoke the SSC when consent is withdrawn. + +### 7.1 Consent signal sources and precedence + +When evaluating consent on a given request, Trusted Server checks signals in the following order. The first signal found wins: + +1. **`X-consent-advertising` request header** — set by the Didomi integration (or another CMP proxy) in a prior server-side decode. This is the freshest signal and takes precedence over browser-stored values. +2. **`euconsent-v2` cookie** — the TCF v2 consent string stored by the publisher's CMP. +3. **`gpp` cookie** — the IAB Global Privacy Platform string for US state-level consent. +4. **Default: no consent** — if no signal is found, do not create the SSC (fail safe). + +### 7.2 Pre-creation consent check + +Before creating a new SSC, Trusted Server evaluates the user's region (via Fastly's `x-geo-country` header) and applies the appropriate consent rule: + +| Region | Required signal | Rule | +| ---------------------------------------------------------------------------------------------------- | --------------- | ----------------------------------------------------------------------------------------------- | +| EU member states | TCF string | Create SSC only if `purposeConsents[1]` (store and/or access information on a device) is `true` | +| United Kingdom | TCF string | Same as EU | +| US states with privacy laws (CA, CO, CT, VA, TX, OR, MT, DE, NH, NJ, TN, IN, IA, KY, NE, MD, MN, RI) | GPP string | Create SSC unless user has opted out of sale or sharing of personal data | +| Rest of world | None required | Create SSC on first visit | + +### 7.3 Consent withdrawal (real-time enforcement) + +On every request, Trusted Server decodes the consent signal (a microsecond in-memory operation with no I/O). If consent is not present or has been revoked: + +**If `ts-ssc` cookie is present:** + +1. Delete the cookie by issuing `Set-Cookie: ts-ssc=; Max-Age=0; Domain=.publisher.com; Path=/; Secure; SameSite=Lax` +2. Delete the KV identity graph entry: `kv_store.delete(ssc_hash)` — this operation takes approximately 25ms and runs in the request path + +**If no `ts-ssc` cookie is present:** + +- Do nothing + +**If consent is present:** + +- Proceed with normal SSC create-or-refresh flow + +**Known tradeoff:** The KV delete adds approximately 25ms of latency to the first request after consent withdrawal. This is an intentional product decision — real-time consent enforcement is a differentiating capability of Trusted Server, and the latency cost is acceptable. + +### 7.4 Future: Data deletion framework + +A formal data deletion endpoint (`POST /api/v1/delete-user`) that allows authenticated partners to trigger deletion of a user's KV entry and cookie is out of scope for this PRD. It is flagged as a follow-on requirement. + +--- + +## 8. KV Store Identity Graph + +### 8.1 Purpose + +The Fastly KV Store serves as a persistent identity graph keyed on the SSC hash. It accumulates resolved partner IDs over time through two write paths: real-time pixel sync redirects and S2S batch pushes from partners. This graph is read at auction time to populate `user.eids` in outbound OpenRTB requests. + +### 8.2 Schema + +**KV key:** The 64-character hex hash portion of the SSC (without the `.suffix`). The hash is stable across sessions for the same user+network+key combination and is safe to use as a long-lived identifier. + +**KV value (JSON body, max ~5KB):** + +```json +{ + "v": 1, + "created": 1741824000, + "last_seen": 1741910400, + "consent": { + "tcf": "CP...", + "gpp": "DBA...", + "ok": true, + "updated": 1741910400 + }, + "geo": { + "country": "US", + "region": "CA" + }, + "ids": { + "ssp_x": { "uid": "abc123", "synced": 1741824000 }, + "liveramp": { "uid": "LR_xyz", "synced": 1741890000 } + } +} +``` + +**KV metadata (max 2048 bytes, readable without streaming body):** + +```json +{ "ok": true, "country": "US", "v": 1 } +``` + +The metadata field is used for consent withdrawal checks. When consent status must be evaluated for a user with an existing SSC, Trusted Server reads metadata only — not the full body — keeping the hot-path latency minimal. + +### 8.3 TTL + +KV entries are created or refreshed with a `time_to_live_sec=31536000` parameter (1 year), matching the cookie `Max-Age`. Fastly's TTL mechanism is eventual garbage collection — entries may persist up to 24 hours past expiry before being removed. This is acceptable for identity data; SSC does not use KV TTL for security-critical expiration. + +### 8.4 Conflict resolution and atomic updates + +When two write paths (pixel sync and S2S batch) attempt to update the same KV entry concurrently, Trusted Server uses Fastly's generation markers to perform atomic read-modify-write: + +1. Read the current KV entry; capture the `generation` header +2. Merge the new partner ID into the `ids` map in memory +3. Write back with `if-generation-match: ` +4. On 412 (Precondition Failed), retry from step 1 (up to 3 retries) + +Within a successful write, conflicts between two different partners updating the same SSC key are resolved by last-write-wins per partner namespace. Partner IDs are keyed by partner ID in the `ids` map; different partners never overwrite each other's entries. + +### 8.5 KV store names + +Two KV stores are required: + +| Store | TOML key | Contents | +| ---------------- | --------------- | ---------------------------------- | +| Identity graph | `ssc_store` | SSC hash → identity graph JSON | +| Partner registry | `partner_store` | Partner ID → config + API key hash | + +The existing `counter_store` and `opid_store` settings (currently defined but unused in `settings.rs`) can be deprecated in a follow-on cleanup. + +--- + +## 9. Pixel Sync Endpoint + +### 9.1 Purpose + +The pixel sync endpoint allows SSPs and DSPs to synchronize their user IDs with the SSC hash via a browser-side redirect. When a partner's sync pixel fires, the user's browser is redirected through `ssc.publisher.com/sync`, Trusted Server reads the existing `ts-ssc` cookie, and writes the partner's user ID into the KV identity graph. + +This is the primary real-time write path for building the identity graph from existing cookie sync infrastructure. + +### 9.2 Endpoint + +``` +GET /sync +``` + +### 9.3 Parameters + +| Parameter | Required | Description | +| --------- | -------- | ------------------------------------------------------------------------------------------------- | +| `partner` | Yes | Partner ID, must match a registered partner in `partner_store` KV | +| `uid` | Yes | Partner's user ID for this user | +| `return` | Yes | Callback URL to redirect to after sync (must match partner's `allowed_return_domains`) | +| `consent` | No | TCF or GPP string from the partner's context, used if no consent signal is present on the request | + +### 9.4 Flow + +1. Read the `ts-ssc` cookie. If absent, redirect to `return` URL immediately without writing to KV. Do not create a new SSC during a sync — a sync redirect is not an organic user visit and must not be used to bootstrap identity. +2. Look up the partner record in `partner_store` KV using the `partner` parameter. Return `400` if the partner is not found. +3. Validate the `return` URL against the partner's `allowed_return_domains`. Return `400` if the domain is not on the allowlist. +4. Evaluate consent for this user (from KV metadata or decode from request cookies). If consent is not present, redirect to `return` without writing KV. +5. If consent is valid, perform an atomic read-modify-write to update `ids[partner_id]` in the KV identity graph (with generation marker — see Section 8.4). +6. Redirect to the `return` URL with `ts_synced=1` appended as a query parameter. + +### 9.5 Security + +- The `return` URL must match an allowlisted domain configured per partner. Open redirects are not permitted. +- Partners control when to fire their sync pixel; no HMAC signature is required on the inbound sync request. +- Anti-stuffing rate limit: a maximum of `sync_rate_limit` sync writes per SSC hash per hour per partner (configurable per partner in `partner_store`, default 100). + +### 9.6 User stories + +**As an SSP**, I want to fire a sync pixel when I see a user so that I can associate my user ID with the SSC hash and receive enriched bid requests when the publisher calls Trusted Server for auction. + +**Acceptance criteria:** + +- [ ] `GET /sync?partner=ssp_x&uid=abc&return=https://sync.ssp.com/ack` returns a redirect to the `return` URL within 50ms (excluding KV write time) +- [ ] KV entry for the SSC hash contains `ids.ssp_x.uid = "abc"` after a successful sync +- [ ] Sync is a no-op (redirect only, no KV write) if no `ts-ssc` cookie is present +- [ ] Sync is a no-op if the user has not given consent +- [ ] `return` URL domains not in partner's `allowed_return_domains` receive a `400` response +- [ ] Rate limit is enforced: more than `sync_rate_limit` writes per hour per SSC hash per partner are rejected with `429` + +--- + +## 10. S2S Batch Sync API + +### 10.1 Purpose + +The S2S batch sync API allows partners to push ID mappings to Trusted Server in bulk via an authenticated REST endpoint. This write path handles large-scale partner-initiated syncs, back-fills for users whose browser-side pixel sync has not fired, and DSP-side match data that originates from non-browser contexts. + +### 10.2 Endpoint + +``` +POST /api/v1/sync +``` + +### 10.3 Authentication + +Partners authenticate using a Bearer token. The token is validated against a bcrypt hash stored in the partner's record in `partner_store` KV. This requires one KV lookup per API call but allows API key rotation without redeploying the binary. + +``` +Authorization: Bearer +``` + +Partner provisioning (writing a partner record into `partner_store`) is performed as a manual admin operation. An automated provisioning endpoint is deferred to a follow-on. + +### 10.4 Request + +``` +POST /api/v1/sync +Content-Type: application/json +Authorization: Bearer + +{ + "mappings": [ + { + "ssc_hash": "<64-character hex hash>", + "partner_uid": "abc123", + "timestamp": 1741824000 + }, + ... + ] +} +``` + +Maximum batch size per request: 1000 mappings (subject to revision based on KV write throughput testing). + +### 10.5 Response + +```json +{ + "accepted": 998, + "rejected": 2, + "errors": [ + { "index": 45, "reason": "ssc_hash_not_found" }, + { "index": 72, "reason": "consent_withdrawn" } + ] +} +``` + +HTTP status `207 Multi-Status` when any mappings are rejected; `200 OK` when all are accepted. + +### 10.6 Consent enforcement + +Before writing a mapping, Trusted Server checks the KV metadata for the given SSC hash. Mappings for users with `consent.ok = false` are rejected with reason `consent_withdrawn`. Partners must not submit mappings for users who have withdrawn consent; this enforcement is a safeguard, not the primary compliance mechanism. + +### 10.7 Conflict resolution + +- If the KV entry does not exist for a given `ssc_hash`, the mapping is rejected with reason `ssc_hash_not_found`. The S2S API does not create new KV entries — only the SSC creation flow (from organic browser visits) can create entries. +- If the partner has an existing entry for the same `ssc_hash` and the request's `timestamp` is older than the stored `synced` timestamp, the mapping is skipped (no error, counted as accepted). +- Otherwise, atomic read-modify-write with generation markers (see Section 8.4). + +### 10.8 User stories + +**As a DSP**, I want to push my user ID mappings to Trusted Server in bulk so that the publisher's auction requests are enriched with my resolved ID and I can bid on users I recognize. + +**Acceptance criteria:** + +- [ ] `POST /api/v1/sync` with a valid Bearer token and a batch of up to 1000 mappings returns a response within 5 seconds +- [ ] Accepted mappings are written to the corresponding KV identity graph entries within 1 second +- [ ] Mappings for unknown `ssc_hash` values are rejected with `ssc_hash_not_found` +- [ ] Mappings for users with withdrawn consent are rejected with `consent_withdrawn` +- [ ] Invalid or expired Bearer tokens receive `401 Unauthorized` +- [ ] Requests exceeding 1000 mappings receive `400 Bad Request` +- [ ] Rate limiting by API key is enforced + +--- + +## 11. Bidstream Decoration + +### 11.1 Two integration modes + +Trusted Server exposes two modes for injecting SSC identity into the bidstream. Publishers choose the mode that fits their existing ad stack. + +### 11.2 Mode A: Header decoration (`/identify`) + +For publishers whose existing ad server handles auction calls, Trusted Server provides an identification-only endpoint that returns the SSC value and resolved identity signals as response headers. The publisher's ad server reads these headers and injects them into its own OpenRTB bid requests. + +**Endpoint:** `GET /identify` + +**Response:** `204 No Content` with the following headers: + +| Header | Value | +| ------------------- | ------------------------------------------------------------- | +| `X-ts-ssc` | `` | +| `X-ts-eids` | Base64-encoded JSON array of OpenRTB 2.6 `user.eids` objects | +| `X-ts-` | Resolved UID per partner (e.g., `X-ts-uid2`, `X-ts-liveramp`) | + +**If consent is not present:** + +``` +HTTP 204 No Content +X-ts-ssc-consent: denied +``` + +No identity headers are returned. The publisher's ad server must handle this case — typically by omitting `user.eids` from the bid request. + +### 11.3 Mode B: Full auction orchestration (`/auction`) + +For publishers using Trusted Server as their auction endpoint, SSC identity is injected directly into outbound OpenRTB requests to Prebid Server. This is an extension of the existing `/auction` endpoint behavior. + +**Changes from current behavior:** + +- `user.id` is set to the full SSC value (`hash.suffix`) +- `user.eids` is populated from the KV identity graph for this user (see OpenRTB structure below) +- `user.consent` is set to the decoded TCF string (currently always `null`) +- SSP-specific `ext.eids`: when calling a specific PBS adapter, only that SSP's resolved ID is included in the adapter-level `ext.eids`. All configured identity providers are included at the top-level `user.eids`. + +### 11.4 OpenRTB 2.6 `user.eids` structure + +```json +{ + "user": { + "id": "a1b2c3...AbC123", + "consent": "CP...", + "eids": [ + { + "source": "liveramp.com", + "uids": [{ "id": "LR_xyz", "atype": 3 }] + }, + { + "source": "id5-sync.com", + "uids": [{ "id": "ID5-abc", "atype": 3 }] + }, + { + "source": "uidapi.com", + "uids": [{ "id": "A4A...", "atype": 3 }] + } + ] + } +} +``` + +`atype` values follow the OpenRTB 2.6 specification: `1` = cookie/device, `2` = hashed email, `3` = partner-defined. All SSC-derived IDs use `atype: 3`. + +### 11.5 Partner taxonomy + +Each partner registered in `partner_store` declares: + +- `source_domain`: the OpenRTB `source` value for their EID (e.g., `"liveramp.com"`) +- `openrtb_atype`: integer (typically `3`) +- `bidstream_enabled`: boolean — whether this partner's UID should appear in `user.eids` on auction requests + +### 11.6 User stories + +**As a publisher using Mode A**, I want to call `/identify` from my ad server so that I can enrich my own auction requests with SSC identity signals without changing my auction infrastructure. + +**Acceptance criteria:** + +- [ ] `GET /identify` returns `204` with `X-ts-ssc` and `X-ts-eids` headers within 30ms (KV read + response) +- [ ] If consent is denied, response contains `X-ts-ssc-consent: denied` and no identity headers +- [ ] `X-ts-eids` is a valid base64-encoded OpenRTB 2.6 `user.eids` array +- [ ] Individual `X-ts-` headers are present for each partner with `bidstream_enabled: true` and a resolved UID + +**As a publisher using Mode B**, I want Trusted Server to include resolved partner IDs in every auction request so that SSPs receive enriched bid requests without additional publisher-side configuration. + +**Acceptance criteria:** + +- [ ] Outbound OpenRTB request to PBS contains `user.id` equal to the SSC value +- [ ] `user.eids` contains one entry per partner with `bidstream_enabled: true` and a resolved UID in the KV graph +- [ ] `user.consent` contains the decoded TCF string when available +- [ ] Partners without a resolved UID for this user are omitted from `user.eids` (no empty entries) + +--- + +## 12. Configuration + +### 12.1 New `[ssc]` section in `trusted-server.toml` + +```toml +[ssc] +enabled = true +ssc_store = "ssc_identity_store" # Fastly KV store: SSC hash → identity graph +partner_store = "ssc_partners" # Fastly KV store: partner ID → config + API key hash +secret_key = "" + +# Partner configs live in partner_store KV, not in TOML. +# Use the admin tooling to provision new partners. +# This allows key rotation without redeploying the binary. +``` + +### 12.2 New `[features]` section + +```toml +[features] +# Full TS defaults: all true +# TS Lite defaults: set the following to false +auction = true +js_injection = true +html_processing = true +proxy_routes = true +request_signing = true +ssc = true +``` + +### 12.3 Partner record schema (in `partner_store` KV) + +KV key: the partner ID string (e.g., `"ssp_x"`) + +```json +{ + "name": "Example SSP", + "key_hash": "$2b$12$...", + "source_domain": "example-ssp.com", + "openrtb_atype": 3, + "bidstream_enabled": true, + "allowed_return_domains": ["sync.example-ssp.com"], + "sync_rate_limit": 100 +} +``` + +--- + +## 13. Documentation Updates + +The following documentation changes are required alongside the SSC feature: + +- **Rename SyntheticID → Server-Side Cookie** across the entire `docs/` GitHub Pages site. The underlying concept is the same but the product name changes. +- **New integration guides**, one per customer type: + - Publisher (TS Lite): setting up `ssc.publisher.com`, configuring `trusted-server.toml`, DNS CNAME setup + - SSP: pixel sync integration guide, sync pixel URL format, callback handling + - DSP: S2S batch API reference, authentication, conflict resolution behavior + - Identity Provider: registering as a partner, `source_domain` and `openrtb_atype` configuration, sync patterns +- **API reference** for the three new endpoints: `GET /sync`, `GET /identify`, `POST /api/v1/sync` +- **Consent enforcement guide**: how TCF and GPP signals are read, precedence rules, what happens on withdrawal + +--- + +## 14. Open Questions + +| # | Question | Owner | Target resolution | +| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | -------------------------- | +| 1 | Partner provisioning flow: should partner records be written manually by a TS admin, or via a `/admin/partners/register` endpoint using the existing admin auth pattern? The latter is more scalable but requires additional implementation. | Product | Before engineering kickoff | +| 2 | Should TS Lite expose a `GET /health` endpoint so partners can programmatically verify their service is running and their partner config is active in KV? | Product | Before engineering kickoff | + +--- + +## 15. Success Metrics + +| Metric | Target | Measurement method | +| -------------------------------- | ---------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | +| SSC match rate (returning users) | >90% within 30 days | Fastly real-time logs: ratio of requests with existing `ts-ssc` cookie vs. new SSC generations | +| Consent enforcement accuracy | 0 SSCs created for opted-out EU/UK users | Log audit: verify no `ts-ssc` `Set-Cookie` in responses where consent signal is absent | +| KV sync latency (pixel sync) | p99 <75ms end-to-end | Fastly log timing on `/sync` endpoint | +| S2S batch API throughput | >500 mappings/sec sustained | Load test prior to partner onboarding | +| Identity graph fill rate | >50% of SSC hashes with at least 1 resolved partner ID within 60 days of partner go-live | KV scan sample | +| TS Lite adoption | First non-publisher customer (SSP or DSP) live within 90 days of launch | Customer record | From bb1a2e24f22a489402e0c8797b671fbe86dd3ec8 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Wed, 18 Mar 2026 12:36:07 -0700 Subject: [PATCH 2/4] Updates to the SSC PRD document --- docs/internal/ssc-prd.md | 788 +++++++++++++++++++++++++++------------ 1 file changed, 555 insertions(+), 233 deletions(-) diff --git a/docs/internal/ssc-prd.md b/docs/internal/ssc-prd.md index a33fb3d6..3269204b 100644 --- a/docs/internal/ssc-prd.md +++ b/docs/internal/ssc-prd.md @@ -1,4 +1,4 @@ -# Product Requirements: Server-Side Cookie (SSC) +# Product Requirements: Edge Cookie (EC) **Status:** Draft **Author:** Trusted Server Product @@ -13,24 +13,27 @@ 3. [Goals and Non-Goals](#3-goals-and-non-goals) 4. [Target Customers](#4-target-customers) 5. [TS Lite Deployment Mode](#5-ts-lite-deployment-mode) -6. [SSC Identity and Cookie Structure](#6-ssc-identity-and-cookie-structure) +6. [EC Identity and Cookie Structure](#6-ec-identity-and-cookie-structure) 7. [Consent Lifecycle](#7-consent-lifecycle) 8. [KV Store Identity Graph](#8-kv-store-identity-graph) 9. [Pixel Sync Endpoint](#9-pixel-sync-endpoint) 10. [S2S Batch Sync API](#10-s2s-batch-sync-api) -11. [Bidstream Decoration](#11-bidstream-decoration) -12. [Configuration](#12-configuration) -13. [Documentation Updates](#13-documentation-updates) -14. [Open Questions](#14-open-questions) -15. [Success Metrics](#15-success-metrics) +11. [S2S Pull Sync (TS-Initiated)](#11-s2s-pull-sync-ts-initiated) +12. [Bidstream Decoration](#12-bidstream-decoration) +13. [Configuration](#13-configuration) +14. [Documentation Updates](#14-documentation-updates) +15. [Open Questions](#15-open-questions) +16. [Success Metrics](#16-success-metrics) --- ## 1. Overview -Server-Side Cookie (SSC) is a stable, privacy-respecting user identity mechanism built into Trusted Server. It replaces the existing SyntheticID system with a cleaner signal (IP address + publisher salt only), a consent-aware lifecycle, a server-side identity graph backed by Fastly KV Store, and a standalone "TS Lite" deployment mode that allows SSPs, DSPs, identity providers, and publishers to adopt SSC without deploying the full Trusted Server feature set. +Edge Cookie (EC) is a stable, privacy-respecting user identity mechanism built into Trusted Server. It replaces the existing SyntheticID system with a cleaner signal (IP address + publisher passphrase only), a consent-aware lifecycle, and a server-side identity graph backed by Fastly KV Store that accumulates resolved partner IDs over time. -SSC runs at a publisher-controlled first-party subdomain (e.g., `ssc.publisher.com`), sets a cookie scoped to the publisher's apex domain, and optionally orchestrates real-time bidding or decorates outbound ad requests with resolved identity signals from configured partners. +The EC hash is derived from the user's IP address and a publisher-chosen passphrase. A publisher's passphrase is consistent across all their own domains, producing the same EC hash for the same user everywhere they operate. Publishers may also share their passphrase with trusted partners to form an **identity-federated consortium** — members sharing a passphrase produce the same EC hash for the same user, enabling cross-property identity resolution by mutual agreement. Publishers using different passphrases produce unrelated hashes with no cross-property linkage. + +EC sets a cookie on the publisher's apex domain (e.g., `ec.publisher.com` sets `ts-ec` on `.publisher.com`) and optionally orchestrates real-time bidding or decorates outbound ad requests with resolved identity signals from configured partners. --- @@ -48,11 +51,11 @@ The result is degrading match rates and false new-user rates on browsers where t ### 2.2 No consent enforcement -SyntheticID is created unconditionally. There is no mechanism to check TCF (EU/UK) or GPP (US) consent before creating the ID. This is a compliance gap that must be closed before SSC can be offered as a product to regulated publishers. +SyntheticID is created unconditionally. There is no mechanism to check TCF (EU/UK) or GPP (US) consent before creating the ID. This is a compliance gap that must be closed before EC can be offered as a product to regulated publishers. -### 2.3 Adoption blocked by full TS requirement +### 2.3 Publishers need a reliable, deterministic signal that can be explicitly shared -SSPs, DSPs, and identity providers want the identity and sync capabilities of Trusted Server without the JS injection pipeline, HTML processing, proxy routing, and auction orchestration that full TS requires. There is no lightweight deployment path today, which blocks a large class of potential adopters. +Today, regular cookies don't suffice for publisher and partner needs. Additionally, only having these identifiers in the 1st party domain's cookie have created slow, undesirable behaviour in the form of cookie syncs. --- @@ -61,63 +64,68 @@ SSPs, DSPs, and identity providers want the identity and sync capabilities of Tr ### Goals - Replace SyntheticID's unstable browser signal inputs with IP address + publisher salt only -- Enforce TCF and GPP consent before creating or maintaining the SSC +- Enforce TCF and GPP consent before creating or maintaining the EC - Implement real-time consent withdrawal: delete cookie and KV entry when consent is revoked - Build a server-side identity graph in Fastly KV Store that accumulates resolved partner IDs over time -- Provide two KV write paths: real-time pixel sync redirects and S2S batch push from partners +- Provide three KV write paths: real-time pixel sync redirects, S2S batch push from partners, and TS-initiated S2S pull from partner resolution endpoints - Expose two bidstream integration modes: header decoration (`/identify`) and full auction orchestration (`/auction`) -- Enable a "TS Lite" deployment mode via runtime TOML feature flags so SSC can run without the full TS feature surface +- Expose a publisher-authenticated `/admin/partners/register` endpoint for partner provisioning without direct KV access ### Non-Goals -- Replacing the publisher's consent management platform (CMP): SSC reads and enforces consent signals; it does not generate them -- Building a data management platform (DMP): SSC stores resolved partner IDs as a sync spine, not audience segments -- Backward compatibility with SyntheticID: SSC uses a different cookie name, header name, and ID generation method. No migration path is provided +- Replacing the publisher's consent management platform (CMP): EC reads and enforces consent signals; it does not generate them +- Building a data management platform (DMP): EC stores resolved partner IDs as a sync spine, not audience segments +- Backward compatibility with SyntheticID: EC uses a different cookie name, header name, and ID generation method. No migration path is provided - Real-time user matching across unrelated domains (cross-site tracking) - Data deletion framework: out of scope for this PRD; flagged for a follow-on document +- **TS Lite deployment mode** (runtime feature flags to run EC without the full TS feature surface): requirements are captured in Section 5 but are deferred to a follow-on iteration. The current iteration targets publishers running full Trusted Server. --- ## 4. Target Customers -| Customer type | Deployment mode | Primary value | -| -------------------- | ------------------------------ | ------------------------------------------------------------------- | -| Publisher (full TS) | Full TS + SSC enabled | Consent-aware first-party ID, bidstream enrichment, identity graph | -| Publisher (SSC only) | TS Lite at `ssc.publisher.com` | First-party cookie at apex domain, identity sync | -| SSP | TS Lite | Pixel sync endpoint to build match table against SSC hash | -| DSP | TS Lite | S2S batch API to push/receive ID mappings, enriched bid requests | -| Identity provider | TS Lite | Register as a partner, sync resolved IDs into the KV identity graph | +**This iteration** targets publishers running the full Trusted Server stack. SSP, DSP, and identity provider customers interact with EC via the sync and bidstream endpoints but do not require a separate TS deployment. + +| Customer type | Deployment mode | Primary value | In scope | +|---|---|---|---| +| Publisher (full TS) | Full TS + EC enabled | Consent-aware first-party ID, bidstream enrichment, identity graph | **Yes** | +| SSP | Partner — integrates via pixel sync and/or S2S pull | Build match table against EC hash; receive enriched bid requests | **Yes** (as partner) | +| DSP | Partner — integrates via S2S batch and/or S2S pull | Push/receive ID mappings; enriched bid requests | **Yes** (as partner) | +| Identity provider | Partner — integrates via S2S batch | Sync resolved IDs into the KV identity graph | **Yes** (as partner) | +| Publisher (EC only) | TS Lite at `ec.publisher.com` | First-party cookie at apex domain without full TS | Deferred (see Section 5) | --- -## 5. TS Lite Deployment Mode +## 5. TS Lite Deployment Mode (Deferred - out of scope) + +> **This section is out of scope for the current iteration.** Requirements are captured here for planning purposes and will be promoted to an active PRD in a follow-on phase. The current iteration delivers EC, the KV identity graph, all three sync mechanisms, and bidstream decoration — all within the existing full Trusted Server deployment model. No feature flags or route-disabling infrastructure will be built now. ### 5.1 Concept -TS Lite is a runtime configuration of the existing Trusted Server binary. It is not a separate binary or separate codebase. A publisher (or SSP/DSP deploying on behalf of a publisher) creates a Fastly service pointing to a subdomain — typically `ssc.publisher.com` — and deploys the standard TS WASM binary with a `trusted-server.toml` that disables all routes except SSC-related functionality. +TS Lite is a runtime configuration of the existing Trusted Server binary. It is not a separate binary or separate codebase. A publisher (or SSP/DSP deploying on behalf of a publisher) creates a Fastly service pointing to a subdomain — typically `ec.publisher.com` — and deploys the standard TS WASM binary with a `trusted-server.toml` that disables all routes except EC-related functionality. ### 5.2 Route surface in TS Lite -| Route | Full TS | TS Lite | -| -------------------------------------- | -------- | ----------------------- | -| `GET /static/tsjs=` | Enabled | Disabled | -| `POST /auction` | Enabled | Optional (configurable) | -| `GET /first-party/proxy` | Enabled | Disabled | -| `GET /first-party/click` | Enabled | Disabled | -| `POST /first-party/sign` | Enabled | Disabled | -| `GET /first-party/proxy-rebuild` | Enabled | Disabled | -| HTML injection pipeline | Enabled | Disabled | -| GTM integration | Enabled | Disabled | -| `GET /sync` | Disabled | **Enabled** | -| `GET /identify` | Disabled | **Enabled** | -| `POST /api/v1/sync` | Disabled | **Enabled** | -| `GET /.well-known/trusted-server.json` | Enabled | Enabled | +| Route | Full TS | TS Lite | +|---|---|---| +| `GET /static/tsjs=` | Enabled | Disabled | +| `POST /auction` | Enabled | Optional (configurable) | +| `GET /first-party/proxy` | Enabled | Disabled | +| `GET /first-party/click` | Enabled | Disabled | +| `POST /first-party/sign` | Enabled | Disabled | +| `GET /first-party/proxy-rebuild` | Enabled | Disabled | +| HTML injection pipeline | Enabled | Disabled | +| GTM integration | Enabled | Disabled | +| `GET /sync` | Disabled | **Enabled** | +| `GET /identify` | Disabled | **Enabled** | +| `POST /api/v1/sync` | Disabled | **Enabled** | +| `GET /.well-known/trusted-server.json` | Enabled | Enabled | When a disabled route is requested, TS returns `404` with the header `X-ts-error: feature-disabled`. ### 5.3 Cookie domain and subdomain setup -The publisher points a subdomain of their choosing (e.g., `ssc`) via DNS CNAME to their Fastly service. They configure `publisher.domain = "publisher.com"` in `trusted-server.toml`. Trusted Server derives `cookie_domain = ".publisher.com"` from this setting and sets the SSC cookie with that domain attribute. +The publisher points a subdomain of their choosing (e.g., `ec`) via DNS CNAME to their Fastly service. They configure `publisher.domain = "publisher.com"` in `trusted-server.toml`. Trusted Server derives `cookie_domain = ".publisher.com"` from this setting and sets the EC cookie with that domain attribute. This gives the cookie read access across all subdomains of `publisher.com` — including `www.publisher.com` — without requiring a separate verification step. The publisher's control over their DNS and Fastly service implicitly proves TLD+1 ownership, following the same trust model as the existing `publisher.cookie_domain` setting. @@ -125,124 +133,137 @@ This gives the cookie read access across all subdomains of `publisher.com` — i ### 5.4 Safari and browser compatibility -The SSC is set as an HTTP `Set-Cookie` response header (not via JavaScript). For server-set cookies on first-party publisher domains that are not classified as cross-site trackers by Safari's ITP, the effective maximum lifetime is 1 year — the same as the configured `Max-Age`. Since `ssc.publisher.com` is a publisher-owned domain, it is unlikely to be classified as a tracker. +The EC is set as an HTTP `Set-Cookie` response header (not via JavaScript). For server-set cookies on first-party publisher domains that are not classified as cross-site trackers by Safari's ITP, the effective maximum lifetime is 1 year — the same as the configured `Max-Age`. Since `ec.publisher.com` is a publisher-owned domain, it is unlikely to be classified as a tracker. -The ITP interaction for users who arrive exclusively via third-party sync pixel redirects (where `ssc.publisher.com` may be seen as a cross-site recipient) will be monitored post-launch. A cookie refresh strategy — re-issuing `Set-Cookie` on every same-site organic request — is deferred pending production data. +The ITP interaction for users who arrive exclusively via third-party sync pixel redirects (where `ec.publisher.com` may be seen as a cross-site recipient) will be monitored post-launch. A cookie refresh strategy — re-issuing `Set-Cookie` on every same-site organic request — is deferred pending production data. --- -## 6. SSC Identity and Cookie Structure +## 6. EC Identity and Cookie Structure ### 6.1 ID generation -The SSC is generated by HMAC-SHA256 of a fixed input set, using a publisher-specific secret key. +The EC is generated by HMAC-SHA256 of a fixed input set, using a publisher-specific secret key. **Inputs (IP address + salt only):** -| Input | Value | -| ---------- | ------------------------------------------------------------------------------------------------------------------------------- | +| Input | Value | +|---|---| | IP address | IPv4 as-is; IPv6 summarized to /64 prefix (first 4 hextets) — discards rotating interface ID. On dual-stack, IPv6 is preferred. | -| Secret key | Publisher-specific salt, configured in `trusted-server.toml` | +| Secret key | Publisher-chosen passphrase, configured in `trusted-server.toml`. Consistent across all of the publisher's own domains. Publishers who share the same passphrase with other publishers form an identity-federated consortium — the same user produces the same EC hash across all consortium members. Publishers using different passphrases produce unrelated hashes with no cross-property linkage. | **Removed from SyntheticID:** - - `User-Agent` - `Accept-Language` - `Accept-Encoding` - Handlebars template (input is now fixed, not configurable) **Output format (unchanged from SyntheticID):** - ``` {64-character hex HMAC-SHA256}.{6-character random alphanumeric suffix} ``` -The 64-character prefix is the stable, deterministic portion used as the KV store key. The 6-character suffix is random, regenerated each time a fresh SSC is created. Once an SSC is set in a cookie, the full value (prefix + suffix) is preserved on subsequent requests. +The 64-character prefix is the stable, deterministic portion used as the KV store key. The 6-character suffix is random, regenerated each time a fresh EC is created. Once an EC is set in a cookie, the full value (prefix + suffix) is preserved on subsequent requests. **IPv6 /64 prefix rationale:** The first 64 bits of an IPv6 address identify the network prefix assigned by the ISP or home router. The remaining 64 bits (the interface ID) are rotated by privacy extensions on most modern operating systems. Using only the /64 prefix produces a stable hash for returning users while discarding the rotating portion that would cause false new-user signals. ### 6.2 Cookie attributes -| Attribute | Value | -| --------- | ----------------------------------------------------------------------------------------- | -| Name | `ts-ssc` | -| Domain | `.publisher.com` (derived from `publisher.domain` in TOML) | -| Path | `/` | -| Secure | Yes | -| SameSite | `Lax` | -| Max-Age | `31536000` (1 year) | -| HttpOnly | No — JavaScript on `www.publisher.com` may need to read the value for ad stack decoration | +| Attribute | Value | +|---|---| +| Name | `ts-ec` | +| Domain | `.publisher.com` (derived from `publisher.domain` in TOML) | +| Path | `/` | +| Secure | Yes | +| SameSite | `Lax` | +| Max-Age | `31536000` (1 year) | +| HttpOnly | No — JavaScript on `www.publisher.com` may need to read the value for ad stack decoration | ### 6.3 Response header -The SSC value is also set as a response header for server-side consumers: +The EC value is also set as a response header for server-side consumers: ``` -X-ts-ssc: +X-ts-ec: ``` This header is internal to Trusted Server and is stripped before proxying requests to downstream backends, consistent with how other `X-ts-*` headers are handled. ### 6.4 Retrieval priority -On each request, Trusted Server looks for an existing SSC in this order: +On each request, Trusted Server looks for an existing EC in this order: -1. `X-ts-ssc` request header (set by TS on a prior response, forwarded by the publisher's infrastructure) -2. `ts-ssc` cookie -3. Generate fresh SSC (subject to consent check — see Section 7) +1. `X-ts-ec` request header (set by TS on a prior response, forwarded by the publisher's infrastructure) +2. `ts-ec` cookie +3. Generate fresh EC (subject to consent check — see Section 7) ### 6.5 No backward compatibility with SyntheticID -SSC uses a different cookie name (`ts-ssc` vs `synthetic_id`), a different header name (`X-ts-ssc` vs `x-synthetic-id`), and a different ID generation method. No fallback to reading the `synthetic_id` cookie is provided. SyntheticID code remains in full TS and continues to function; SSC is a parallel system. +EC uses a different cookie name (`ts-ec` vs `synthetic_id`), a different header name (`X-ts-ec` vs `x-synthetic-id`), and a different ID generation method. No fallback to reading the `synthetic_id` cookie is provided. SyntheticID code remains in full TS and continues to function; EC is a parallel system. --- ## 7. Consent Lifecycle -Consent enforcement is a core requirement of SSC. The system must not create or maintain an SSC for users who have not given consent, and must actively revoke the SSC when consent is withdrawn. +Consent enforcement is a core requirement of EC. The system must not create or maintain an EC for users who have not given consent, and must actively revoke the EC when consent is withdrawn. ### 7.1 Consent signal sources and precedence -When evaluating consent on a given request, Trusted Server checks signals in the following order. The first signal found wins: +Section 7.1 describes **how** consent signals are read. Section 7.2 describes **whether** a signal is required at all for a given region. These two sections work in sequence: TS first determines the region (7.2), then — only if that region requires a consent signal — reads and evaluates the signal using the precedence order below. + +When a consent signal is required for the user's region, Trusted Server checks sources in the following order. The first signal found wins: 1. **`X-consent-advertising` request header** — set by the Didomi integration (or another CMP proxy) in a prior server-side decode. This is the freshest signal and takes precedence over browser-stored values. 2. **`euconsent-v2` cookie** — the TCF v2 consent string stored by the publisher's CMP. 3. **`gpp` cookie** — the IAB Global Privacy Platform string for US state-level consent. -4. **Default: no consent** — if no signal is found, do not create the SSC (fail safe). +4. **Default: no consent** — if the region requires a signal and none is found, do not create the EC (fail safe). This step does not apply to regions where no signal is required — a user in a rest-of-world region with no consent cookies present is not subject to this fail-safe. ### 7.2 Pre-creation consent check -Before creating a new SSC, Trusted Server evaluates the user's region (via Fastly's `x-geo-country` header) and applies the appropriate consent rule: +Before creating a new EC, Trusted Server first evaluates the user's region (via Fastly's `x-geo-country` header) to determine whether a consent signal is required. If the region requires a signal, TS reads it using the precedence order in Section 7.1; if no signal is found, creation is blocked (the fail-safe in step 4 applies). If the region does not require a signal, TS creates the EC unconditionally. -| Region | Required signal | Rule | -| ---------------------------------------------------------------------------------------------------- | --------------- | ----------------------------------------------------------------------------------------------- | -| EU member states | TCF string | Create SSC only if `purposeConsents[1]` (store and/or access information on a device) is `true` | -| United Kingdom | TCF string | Same as EU | -| US states with privacy laws (CA, CO, CT, VA, TX, OR, MT, DE, NH, NJ, TN, IN, IA, KY, NE, MD, MN, RI) | GPP string | Create SSC unless user has opted out of sale or sharing of personal data | -| Rest of world | None required | Create SSC on first visit | +| Region | Required signal | Rule | +|---|---|---| +| EU member states | TCF string | Create EC only if `purposeConsents[1]` (store and/or access information on a device) is `true`. If no TCF signal is found, do not create EC (7.1 step 4 applies). | +| United Kingdom | TCF string | Same as EU | +| US states with privacy laws (CA, CO, CT, VA, TX, OR, MT, DE, NH, NJ, TN, IN, IA, KY, NE, MD, MN, RI) | GPP string | Create EC unless user has opted out of sale or sharing of personal data. If no GPP signal is found, do not create EC (7.1 step 4 applies). | +| Rest of world | None required | Create EC on first visit regardless of whether any consent signal is present. Section 7.1 step 4 does not apply. | ### 7.3 Consent withdrawal (real-time enforcement) On every request, Trusted Server decodes the consent signal (a microsecond in-memory operation with no I/O). If consent is not present or has been revoked: -**If `ts-ssc` cookie is present:** - -1. Delete the cookie by issuing `Set-Cookie: ts-ssc=; Max-Age=0; Domain=.publisher.com; Path=/; Secure; SameSite=Lax` -2. Delete the KV identity graph entry: `kv_store.delete(ssc_hash)` — this operation takes approximately 25ms and runs in the request path - -**If no `ts-ssc` cookie is present:** +**If `ts-ec` cookie is present:** +1. Delete the cookie by issuing `Set-Cookie: ts-ec=; Max-Age=0; Domain=.publisher.com; Path=/; Secure; SameSite=Lax` +2. Delete the KV identity graph entry: `kv_store.delete(ec_hash)` — this operation takes approximately 25ms and runs in the request path +**If no `ts-ec` cookie is present:** - Do nothing **If consent is present:** - -- Proceed with normal SSC create-or-refresh flow +- Proceed with normal EC create-or-refresh flow **Known tradeoff:** The KV delete adds approximately 25ms of latency to the first request after consent withdrawal. This is an intentional product decision — real-time consent enforcement is a differentiating capability of Trusted Server, and the latency cost is acceptable. -### 7.4 Future: Data deletion framework +### 7.4 Data deletion framework + +Trusted Server implements the [IAB Data Subject Rights — Data Deletion Request Framework](https://github.com/InteractiveAdvertisingBureau/Data-Subject-Rights/blob/main/Data%20Deletion%20Request%20Framework.md) as its mechanism for honoring data deletion requests from users and partners. This is the authoritative answer for partners and regulators asking "how do I delete a user?" — there is no separate interim process. + +**TS role in the framework:** Trusted Server acts as the **1st party** (it has the direct user relationship via the publisher's domain). It both receives deletion requests and initiates them downstream to registered partners who hold the same user's data. + +**How it works:** -A formal data deletion endpoint (`POST /api/v1/delete-user`) that allows authenticated partners to trigger deletion of a user's KV entry and cookie is out of scope for this PRD. It is flagged as a follow-on requirement. +1. TS publishes a `dsrdelete.json` discovery file at `ec.publisher.com/.well-known/dsrdelete.json` listing its deletion endpoint, supported identifier types (EC hash), and public key. +2. A deletion request arrives as an HTTP `POST` containing a signed `rqJWT` (wrapping an `idJWT` identifying the user by EC hash). +3. TS verifies the JWT signatures, looks up the EC hash in the KV identity graph, deletes the KV entry and issues `Set-Cookie: ts-ec=; Max-Age=0` to expire the cookie. +4. TS returns a signed `acJWT` with result code `0` (success) or the appropriate error code. +5. TS propagates the deletion request to all registered partners in `partner_store` who have a resolved UID for this user, using their declared deletion endpoints. + +**Identifier type:** The EC hash (64-character hex prefix, without `.suffix`) is the stable identifier registered in `dsrdelete.json`. The `.suffix` portion is not used for deletion matching — the hash is sufficient to locate the KV entry. + +**Interim answer for partners during onboarding (before TS's deletion endpoint ships):** Publishers can manually delete a KV entry by EC hash via the Fastly KV management API or console. The EC cookie expires naturally within 1 year. A formal `POST` endpoint implementing the full JWT protocol above is required before any regulated publisher goes live. + +**Implementation status:** The `dsrdelete.json` discovery file and the JWT-based deletion endpoint are a follow-on engineering deliverable, to be completed before regulated publisher onboarding. --- @@ -250,11 +271,11 @@ A formal data deletion endpoint (`POST /api/v1/delete-user`) that allows authent ### 8.1 Purpose -The Fastly KV Store serves as a persistent identity graph keyed on the SSC hash. It accumulates resolved partner IDs over time through two write paths: real-time pixel sync redirects and S2S batch pushes from partners. This graph is read at auction time to populate `user.eids` in outbound OpenRTB requests. +The Fastly KV Store serves as a persistent identity graph keyed on the EC hash. It accumulates resolved partner IDs over time through two write paths: real-time pixel sync redirects and S2S batch pushes from partners. This graph is read at auction time to populate `user.eids` in outbound OpenRTB requests. ### 8.2 Schema -**KV key:** The 64-character hex hash portion of the SSC (without the `.suffix`). The hash is stable across sessions for the same user+network+key combination and is safe to use as a long-lived identifier. +**KV key:** The 64-character hex hash portion of the EC (without the `.suffix`). The hash is stable across sessions for the same user+network+key combination and is safe to use as a long-lived identifier. **KV value (JSON body, max ~5KB):** @@ -286,41 +307,143 @@ The Fastly KV Store serves as a persistent identity graph keyed on the SSC hash. { "ok": true, "country": "US", "v": 1 } ``` -The metadata field is used for consent withdrawal checks. When consent status must be evaluated for a user with an existing SSC, Trusted Server reads metadata only — not the full body — keeping the hot-path latency minimal. +The metadata field is used for consent withdrawal checks. When consent status must be evaluated for a user with an existing EC, Trusted Server reads metadata only — not the full body — keeping the hot-path latency minimal. ### 8.3 TTL -KV entries are created or refreshed with a `time_to_live_sec=31536000` parameter (1 year), matching the cookie `Max-Age`. Fastly's TTL mechanism is eventual garbage collection — entries may persist up to 24 hours past expiry before being removed. This is acceptable for identity data; SSC does not use KV TTL for security-critical expiration. +KV entries are created or refreshed with a `time_to_live_sec=31536000` parameter (1 year), matching the cookie `Max-Age`. Fastly's TTL mechanism is eventual garbage collection — entries may persist up to 24 hours past expiry before being removed. This is acceptable for identity data; EC does not use KV TTL for security-critical expiration. -### 8.4 Conflict resolution and atomic updates +### 8.4 Conflict resolution -When two write paths (pixel sync and S2S batch) attempt to update the same KV entry concurrently, Trusted Server uses Fastly's generation markers to perform atomic read-modify-write: - -1. Read the current KV entry; capture the `generation` header -2. Merge the new partner ID into the `ids` map in memory -3. Write back with `if-generation-match: ` -4. On 412 (Precondition Failed), retry from step 1 (up to 3 retries) - -Within a successful write, conflicts between two different partners updating the same SSC key are resolved by last-write-wins per partner namespace. Partner IDs are keyed by partner ID in the `ids` map; different partners never overwrite each other's entries. +Concurrent writes from different partners to the same KV entry must not overwrite each other's data. Each partner's ID is stored under its own namespace in the `ids` map — a write for `ssp_x` must never clobber an existing entry for `liveramp`. Implementation must guarantee this isolation under concurrent write conditions. ### 8.5 KV store names Two KV stores are required: -| Store | TOML key | Contents | -| ---------------- | --------------- | ---------------------------------- | -| Identity graph | `ssc_store` | SSC hash → identity graph JSON | +| Store | TOML key | Contents | +|---|---|---| +| Identity graph | `ec_store` | EC hash → identity graph JSON | | Partner registry | `partner_store` | Partner ID → config + API key hash | The existing `counter_store` and `opid_store` settings (currently defined but unused in `settings.rs`) can be deprecated in a follow-on cleanup. +### 8.6 KV Store degraded behavior + +The EC cookie is deterministic (derived from IP + publisher salt) and lives in the browser. It does not depend on KV Store availability. KV Store holds identity enrichment only — resolved partner UIDs accumulated over time. The degraded behavior policy follows from this: **EC always works; enrichment degrades gracefully.** + +| Operation | KV unavailable or error | Rationale | +|---|---|---| +| EC cookie creation | Set the cookie. Skip the KV entry creation silently. Log the failure at `warn` level. | The cookie is the identity anchor — it does not require KV. The KV entry will be created on the next request once KV recovers. | +| EC cookie refresh (existing user) | Refresh the cookie. Skip the KV `last_seen` update silently. Log at `warn`. | Same as above — the cookie continues working. Stale `last_seen` is acceptable. | +| `/sync` KV write | Redirect to `return` with `ts_synced=0&ts_reason=write_failed`. | The browser redirect must not be blocked by KV availability. This case is already specified in Section 9.4. | +| `/identify` KV read | Return `200` with `ec` hash (from cookie) and `degraded: true`. Set `uids: {}` and `eids: []`. | The EC hash is still valid and useful for attribution and analytics. Empty uids signal that enrichment is unavailable, not that the user has no synced partners. `degraded: true` lets callers distinguish transient KV failure from a genuinely unenriched user. | +| S2S batch write (`/api/v1/sync`) | Return `207` with all mappings rejected, `reason: "kv_unavailable"`. | The request was valid; the failure is infrastructure. Partners should retry the batch. | +| S2S pull sync write (async) | Discard the resolved uid. Log at `warn`. Retry will occur on the next qualifying request per the `pull_sync_ttl_sec` window. | Async path — no user-facing impact. | +| Consent withdrawal KV delete | Expire the cookie immediately. Log the KV delete failure at `error` level. Retry the KV delete on the next request for this user. | Cookie deletion is the primary enforcement mechanism. KV delete failure must not block or delay the cookie expiry. | + +**`degraded: true` in `/identify` responses** + +When a KV read fails, the `/identify` response includes `"degraded": true` in the JSON body alongside an empty `uids` and `eids`. The `ec` field is still populated from the cookie. Callers should proceed with identity-only targeting (EC hash) and omit partner UID parameters from downstream requests. + +```json +{ + "ec": "a1b2c3...AbC123", + "consent": "ok", + "degraded": true, + "uids": {}, + "eids": [] +} +``` + +### 8.7 Buyer confidence in KV entries (Deferred - out of scope) + +#### Problem + +Code attestation (reproducible WASM builds + published binary hashes) proves that the TS binary running on Fastly's infrastructure matches the open-source repository. It does not, however, prove that the *data* inside `ec_store` was written by that attested binary. A malicious or compromised operator could write arbitrary identity mappings directly into the KV store — bypassing all code paths — and buyers would have no way to detect it. + +#### Solution: JOSE-signed KV entry bodies + +Every identity graph entry written to `ec_store` by the TS WASM binary is signed using JSON Web Signatures (JWS, RFC 7515) before storage. The signing key is generated at binary load time and is bound to the running instance; the corresponding public key is published alongside the binary hash in the attestation record. + +At read time, the TS binary verifies the JWS signature before consuming any fields from the entry. An entry that fails signature verification is treated as absent, the request proceeds as if the KV key does not exist, and the failure is logged at `error` level. + +**What a valid signature proves:** + +- The entry was written by a TS binary instance whose signing key corresponds to a published, attested binary hash. +- The entry body has not been modified since it was written. +- A buyer who trusts the attested binary can transitively trust any entry that carries a valid signature. + +**What it does not prove:** + +- That the *input data* (e.g., a partner-supplied UID) was accurate at the time of write. Signal accuracy remains the partner's responsibility. +- Anything about entries written before this feature was deployed. A migration pass will resign existing entries or treat them as unsigned (degraded) until they are refreshed by a normal TS write. + +#### Attestation record endpoint + +The signing public key is published as a namespaced field inside the existing `/.well-known/trusted-server.json` discovery document — the same endpoint partners already fetch for request signing key distribution. No new endpoint is required. + +``` +GET /.well-known/trusted-server.json +``` + +Response (application/json): + +```json +{ + "version": "1.0", + "jwks": { + "keys": [ + { "kty": "OKP", "crv": "Ed25519", "kid": "ts-2026-A", "use": "sig", "x": "..." } + ] + }, + "attestation": { + "binary_hash": "", + "alg": "ES256", + "jwk": { "kty": "EC", "crv": "P-256", "x": "...", "y": "..." }, + "expires_at": "2026-06-18T00:00:00Z" + } +} +``` + +The `jwks` field is unchanged — it continues to serve request signing keys on its existing rotation schedule. The `attestation` object is a separate namespace and does not affect existing consumers of this endpoint. + +| Field | Description | +|---|---| +| `attestation.binary_hash` | SHA-256 hex of the deployed WASM binary. Cross-referenced with Fastly's signed deployment manifest in the reproducible builds PRD. | +| `attestation.alg` | JWS algorithm used for all KV entry signatures. Fixed at `ES256` (ECDSA P-256). | +| `attestation.jwk` | Public key in JWK format (RFC 7517). Buyers use this to verify signatures in KV-derived `user.eids`. Distinct from the `jwks` request-signing keys. | +| `attestation.expires_at` | UTC timestamp after which the attestation key should be considered untrustworthy. Buyers must re-fetch before this time. | + +**Key TTL:** 90 days. The attestation key rotates on each new TS deployment. The previous key's `expires_at` is set 7 days after rotation to allow in-flight impressions to drain. + +**Key storage:** The signing private key lives in the Fastly Secret Store under `ec_signing_key`. It is provisioned at deploy time and never exposed in responses or logs. + +**Caching:** `trusted-server.json` should be served with `Cache-Control: max-age=3600` to ensure buyers pick up a rotated attestation key within one hour of a new deployment. This is shorter than the JWKS key rotation window and is safe for both key types. + +> **Future:** When the reproducible builds PRD ships, the `attestation` object may be graduated to a dedicated `/.well-known/ts-attestation.json` endpoint if the data (multiple binary hashes, Fastly co-signatures) outgrows the shared document. The field names will remain compatible. + +#### Buyer-facing verification flow + +1. Publisher includes `site.ext.ts_discovery` pointing to `/.well-known/trusted-server.json` in the bid request. +2. Buyer fetches `trusted-server.json` and caches it until `attestation.expires_at`. +3. Buyer independently verifies `attestation.binary_hash` against Fastly's signed deployment manifest (see separate PRD). +4. Buyer verifies the JWS signature on each `user.eids` entry against `attestation.jwk`. +5. Buyer trusts KV-derived signals only for entries with a valid signature from a non-expired attestation key. + +#### Relationship to reproducible builds PRD + +JOSE-signed KV entries close the *data integrity* gap that code attestation leaves open. Reproducible builds and published binary hashes address the *code integrity* layer — proving that the deployed binary matches the audited source. These are complementary controls that together form a complete trust chain for buyers. + +The reproducible builds feature has broader scope than the identity graph (it applies to all TS behaviour, not just KV writes) and will be specified in a dedicated PRD. The `attestation.binary_hash` field in `trusted-server.json` anticipates that PRD — buyers can record it today, and the reproducible builds PRD will define the process for independently verifying it against Fastly's signed deployment manifest. + --- ## 9. Pixel Sync Endpoint ### 9.1 Purpose -The pixel sync endpoint allows SSPs and DSPs to synchronize their user IDs with the SSC hash via a browser-side redirect. When a partner's sync pixel fires, the user's browser is redirected through `ssc.publisher.com/sync`, Trusted Server reads the existing `ts-ssc` cookie, and writes the partner's user ID into the KV identity graph. +The pixel sync endpoint allows SSPs and DSPs to synchronize their user IDs with the EC hash via a browser-side redirect. When a partner's sync pixel fires, the user's browser is redirected through `ec.publisher.com/sync`, Trusted Server reads the existing `ts-ec` cookie, and writes the partner's user ID into the KV identity graph. This is the primary real-time write path for building the identity graph from existing cookie sync infrastructure. @@ -332,40 +455,51 @@ GET /sync ### 9.3 Parameters -| Parameter | Required | Description | -| --------- | -------- | ------------------------------------------------------------------------------------------------- | -| `partner` | Yes | Partner ID, must match a registered partner in `partner_store` KV | -| `uid` | Yes | Partner's user ID for this user | -| `return` | Yes | Callback URL to redirect to after sync (must match partner's `allowed_return_domains`) | -| `consent` | No | TCF or GPP string from the partner's context, used if no consent signal is present on the request | +| Parameter | Required | Description | +|---|---|---| +| `partner` | Yes | Partner ID, must match a registered partner in `partner_store` KV | +| `uid` | Yes | Partner's user ID for this user | +| `return` | Yes | Callback URL to redirect to after sync (must match partner's `allowed_return_domains`) | +| `consent` | No | TCF or GPP string from the partner's context, used if no consent signal is present on the request | ### 9.4 Flow -1. Read the `ts-ssc` cookie. If absent, redirect to `return` URL immediately without writing to KV. Do not create a new SSC during a sync — a sync redirect is not an organic user visit and must not be used to bootstrap identity. +1. Read the `ts-ec` cookie. If absent, redirect to `return` URL with `ts_synced=0` appended. Do not create a new EC during a sync — a sync redirect is not an organic user visit and must not be used to bootstrap identity. 2. Look up the partner record in `partner_store` KV using the `partner` parameter. Return `400` if the partner is not found. 3. Validate the `return` URL against the partner's `allowed_return_domains`. Return `400` if the domain is not on the allowlist. -4. Evaluate consent for this user (from KV metadata or decode from request cookies). If consent is not present, redirect to `return` without writing KV. -5. If consent is valid, perform an atomic read-modify-write to update `ids[partner_id]` in the KV identity graph (with generation marker — see Section 8.4). -6. Redirect to the `return` URL with `ts_synced=1` appended as a query parameter. +4. Evaluate consent for this user by decoding from request cookies (or the optional `consent` query parameter if no cookie signal is present). If consent is absent or invalid, redirect to `return` with `ts_synced=0&ts_reason=no_consent`. No KV write is performed. +5. Perform an atomic read-modify-write to update `ids[partner_id]` in the KV identity graph (with generation marker — see Section 8.4). If the write fails after all retries, redirect to `return` with `ts_synced=0&ts_reason=write_failed`. +6. On successful KV write, redirect to `return` with `ts_synced=1` appended as a query parameter. + +**`ts_synced` values:** + +| Value | Meaning | +|---|---| +| `ts_synced=1` | KV write succeeded — partner uid is now in the identity graph | +| `ts_synced=0&ts_reason=no_ec` | No EC cookie present — user has not established an EC on this publisher | +| `ts_synced=0&ts_reason=no_consent` | Consent absent or invalid — write suppressed | +| `ts_synced=0&ts_reason=write_failed` | KV write failed after retries — partner should retry on a future pixel fire | + +Partners should treat `ts_synced=0` as a signal that the mapping was not stored. The `ts_reason` parameter is informational; partners should not gate their own behavior on specific reason values. ### 9.5 Security -- The `return` URL must match an allowlisted domain configured per partner. Open redirects are not permitted. +- The `return` URL is validated against the partner's `allowed_return_domains` using **exact hostname match** — `sync.example-ssp.com` does not match `a.sync.example-ssp.com`. Suffix or wildcard matching is not supported. This prevents subdomain takeover abuse where an attacker controlling an abandoned subdomain of a legitimate partner could exploit TS as an open redirect. Partners needing multiple callback hostnames must register each one explicitly in `allowed_return_domains`. Open redirects are not permitted. - Partners control when to fire their sync pixel; no HMAC signature is required on the inbound sync request. -- Anti-stuffing rate limit: a maximum of `sync_rate_limit` sync writes per SSC hash per hour per partner (configurable per partner in `partner_store`, default 100). +- Anti-stuffing rate limit: a maximum of `sync_rate_limit` sync writes per EC hash per hour per partner (configurable per partner in `partner_store`, default 100). ### 9.6 User stories -**As an SSP**, I want to fire a sync pixel when I see a user so that I can associate my user ID with the SSC hash and receive enriched bid requests when the publisher calls Trusted Server for auction. +**As an SSP**, I want to fire a sync pixel when I see a user so that I can associate my user ID with the EC hash and receive enriched bid requests when the publisher calls Trusted Server for auction. **Acceptance criteria:** - - [ ] `GET /sync?partner=ssp_x&uid=abc&return=https://sync.ssp.com/ack` returns a redirect to the `return` URL within 50ms (excluding KV write time) -- [ ] KV entry for the SSC hash contains `ids.ssp_x.uid = "abc"` after a successful sync -- [ ] Sync is a no-op (redirect only, no KV write) if no `ts-ssc` cookie is present -- [ ] Sync is a no-op if the user has not given consent -- [ ] `return` URL domains not in partner's `allowed_return_domains` receive a `400` response -- [ ] Rate limit is enforced: more than `sync_rate_limit` writes per hour per SSC hash per partner are rejected with `429` +- [ ] KV entry for the EC hash contains `ids.ssp_x.uid = "abc"` after a successful sync; response redirects to `return` with `ts_synced=1` +- [ ] If no `ts-ec` cookie is present, redirects to `return` with `ts_synced=0&ts_reason=no_ec`; no KV write performed +- [ ] If consent is absent or invalid, redirects to `return` with `ts_synced=0&ts_reason=no_consent`; no KV write performed +- [ ] If KV write fails after all retries, redirects to `return` with `ts_synced=0&ts_reason=write_failed` +- [ ] `return` URL domains not in partner's `allowed_return_domains` receive a `400` response (no redirect) +- [ ] Rate limit is enforced: more than `sync_rate_limit` writes per hour per EC hash per partner are rejected with `429` --- @@ -383,13 +517,7 @@ POST /api/v1/sync ### 10.3 Authentication -Partners authenticate using a Bearer token. The token is validated against a bcrypt hash stored in the partner's record in `partner_store` KV. This requires one KV lookup per API call but allows API key rotation without redeploying the binary. - -``` -Authorization: Bearer -``` - -Partner provisioning (writing a partner record into `partner_store`) is performed as a manual admin operation. An automated provisioning endpoint is deferred to a follow-on. +Partners authenticate with a rotatable API key. Key rotation must not require redeploying the binary. Partner provisioning is handled via the `/admin/partners/register` endpoint (see Section 15, Open Questions). ### 10.4 Request @@ -401,7 +529,7 @@ Authorization: Bearer { "mappings": [ { - "ssc_hash": "<64-character hex hash>", + "ec_hash": "<64-character hex hash>", "partner_uid": "abc123", "timestamp": 1741824000 }, @@ -419,22 +547,32 @@ Maximum batch size per request: 1000 mappings (subject to revision based on KV w "accepted": 998, "rejected": 2, "errors": [ - { "index": 45, "reason": "ssc_hash_not_found" }, + { "index": 45, "reason": "ec_hash_not_found" }, { "index": 72, "reason": "consent_withdrawn" } ] } ``` -HTTP status `207 Multi-Status` when any mappings are rejected; `200 OK` when all are accepted. +**HTTP status rules:** + +| Condition | Status | +|---|---| +| All mappings accepted | `200 OK` | +| Some mappings accepted, some rejected | `207 Multi-Status` | +| Auth valid, batch valid, but **all** mappings rejected | `207 Multi-Status` with `accepted: 0` | +| Auth invalid | `401 Unauthorized` (no body processing) | +| Batch exceeds 1000 mappings or malformed JSON | `400 Bad Request` (no body processing) | + +A `207` with `accepted: 0` signals "your request was received and processed correctly, but none of the submitted EC hashes were found or eligible." This is distinct from an auth or protocol error. Partners should treat this as a data signal — either the EC hashes are stale/unknown, or consent has been withdrawn for all submitted users — and should not retry the same batch without investigating the underlying cause. ### 10.6 Consent enforcement -Before writing a mapping, Trusted Server checks the KV metadata for the given SSC hash. Mappings for users with `consent.ok = false` are rejected with reason `consent_withdrawn`. Partners must not submit mappings for users who have withdrawn consent; this enforcement is a safeguard, not the primary compliance mechanism. +Before writing a mapping, Trusted Server checks the KV metadata for the given EC hash. Mappings for users with `consent.ok = false` are rejected with reason `consent_withdrawn`. Partners must not submit mappings for users who have withdrawn consent; this enforcement is a safeguard, not the primary compliance mechanism. ### 10.7 Conflict resolution -- If the KV entry does not exist for a given `ssc_hash`, the mapping is rejected with reason `ssc_hash_not_found`. The S2S API does not create new KV entries — only the SSC creation flow (from organic browser visits) can create entries. -- If the partner has an existing entry for the same `ssc_hash` and the request's `timestamp` is older than the stored `synced` timestamp, the mapping is skipped (no error, counted as accepted). +- If the KV entry does not exist for a given `ec_hash`, the mapping is rejected with reason `ec_hash_not_found`. The S2S API does not create new KV entries — only the EC creation flow (from organic browser visits) can create entries. +- If the partner has an existing entry for the same `ec_hash` and the request's `timestamp` is older than the stored `synced` timestamp, the mapping is skipped (no error, counted as accepted). - Otherwise, atomic read-modify-write with generation markers (see Section 8.4). ### 10.8 User stories @@ -442,10 +580,9 @@ Before writing a mapping, Trusted Server checks the KV metadata for the given SS **As a DSP**, I want to push my user ID mappings to Trusted Server in bulk so that the publisher's auction requests are enriched with my resolved ID and I can bid on users I recognize. **Acceptance criteria:** - - [ ] `POST /api/v1/sync` with a valid Bearer token and a batch of up to 1000 mappings returns a response within 5 seconds - [ ] Accepted mappings are written to the corresponding KV identity graph entries within 1 second -- [ ] Mappings for unknown `ssc_hash` values are rejected with `ssc_hash_not_found` +- [ ] Mappings for unknown `ec_hash` values are rejected with `ec_hash_not_found` - [ ] Mappings for users with withdrawn consent are rejected with `consent_withdrawn` - [ ] Invalid or expired Bearer tokens receive `401 Unauthorized` - [ ] Requests exceeding 1000 mappings receive `400 Bad Request` @@ -453,47 +590,258 @@ Before writing a mapping, Trusted Server checks the KV metadata for the given SS --- -## 11. Bidstream Decoration +## 11. S2S Pull Sync (TS-Initiated) + +### 11.1 Purpose + +The pixel sync endpoint (Section 9) requires the user's browser to initiate a redirect, which can be blocked by ad blockers or ITP. The S2S batch API (Section 10) requires the partner to proactively push mappings. Neither path helps when the publisher wants to opportunistically ask a partner "do you know this user?" without waiting for a pixel to fire. + +S2S pull sync inverts the S2S batch model: Trusted Server calls the partner's resolution endpoint directly, server-to-server, and writes the returned uid into the KV identity graph. No browser pixel is involved. The HTTP return path is the response body — no redirect required. + +**What the partner resolves against** + +The partner's resolution endpoint receives the EC hash and IP address. The partner must look these up against their own **server-side user database** — not a browser cookie. Common sources partners use: + +- **IP-based user graph**: major SSPs and DSPs maintain server-side mappings of IP → their own uid, built from bid stream traffic and direct visits. If a user has hit any page on which this partner runs, they may have an IP mapping. +- **Prior bid stream observation**: once the EC hash begins appearing in outbound bid requests (Mode B), partners who have bid on those requests can build their own reverse map of EC hash → their uid. Subsequent pull calls can then be resolved against this map. +- **Authenticated / hashed-email graph**: for partners with deterministic identity (UID2, RampID), they may resolve from email-hash mappings independently of IP. -### 11.1 Two integration modes +**Implication:** pull sync only returns a uid for users the partner already knows by some server-side signal. If the partner has never seen this user by any channel, they return null and the call is a no-op. This is not a general solution for new users — it is a reliable, pixel-free path for users the partner already knows. -Trusted Server exposes two modes for injecting SSC identity into the bidstream. Publishers choose the mode that fits their existing ad stack. +**What it solves and what it doesn't:** -### 11.2 Mode A: Header decoration (`/identify`) +| User scenario | Pixel sync | S2S batch | S2S pull | +|---|---|---|---| +| New user, Chrome, 3p cookies available | Works (bootstraps KV) | Not applicable | No server-side mapping yet — no-op | +| Returning user after prior pixel sync | Redundant (already in KV) | Works | Works (partner has IP or bid-stream mapping) | +| Safari user, partner has IP-based mapping | Blocked / unreliable | Works if partner knows EC hash | Works — partner resolves from their IP graph | +| User unknown to partner by any signal | No uid to sync | No uid to push | No uid to return — no-op | +| Authenticated user with hashed email | Works | Works | Works | -For publishers whose existing ad server handles auction calls, Trusted Server provides an identification-only endpoint that returns the SSC value and resolved identity signals as response headers. The publisher's ad server reads these headers and injects them into its own OpenRTB bid requests. +S2S pull does not solve the cold-start problem for users the partner has never seen. It degrades gracefully to a no-op in those cases. + +### 11.2 When TS initiates a pull + +Trusted Server initiates a pull sync for a given partner when all of the following are true on an incoming request: + +1. A valid `ts-ec` cookie is present (user has an established EC) +2. Consent is valid for this user +3. The partner has `pull_sync_enabled: true` in their `partner_store` record +4. The KV identity graph for this EC hash has no entry for this partner, **or** the existing entry's `synced` timestamp is older than `pull_sync_ttl_sec` (configurable per partner, default 86400 — 1 day) + +### 11.3 Execution model + +Pull sync calls are dispatched **asynchronously after the response is sent** using Fastly's `send_async` / background task model. They do not add latency to the user-facing request. + +A maximum of `pull_sync_concurrency` partner calls are dispatched per request (configurable globally, default 3). If more partners qualify, they are queued and dispatched on subsequent requests for the same user. + +### 11.4 Partner resolution endpoint + +Each partner exposes a resolution endpoint declared in their `partner_store` record as `pull_sync_url`. Trusted Server calls it with a `GET` request: + +``` +GET ?ec_hash=<64-char-hex>&ip= +Authorization: Bearer +``` + +`ts_pull_token` is a per-partner token provisioned during partner registration, used so the partner can authenticate inbound requests from Trusted Server. It is stored in `partner_store` KV in plaintext (outbound credential, not inbound). + +**Expected response (`200 OK`):** + +```json +{ "uid": "abc123" } +``` + +**If the partner does not recognize the user:** + +```json +{ "uid": null } +``` + +or `404 Not Found`. Both are treated as a no-op — no KV write. + +Any response other than `200` with a valid body is treated as a transient failure. Trusted Server does not retry on failure; the next qualifying request for this user will trigger a new attempt. + +### 11.5 KV write + +On a successful resolution (`uid` is non-null), Trusted Server performs the same atomic read-modify-write used by the pixel sync path (Section 8.4): read the existing KV entry with a generation marker, merge `ids[partner_id].uid`, write back with `if-generation-match`. + +The `synced` timestamp is set to the current Unix timestamp, which resets the `pull_sync_ttl_sec` clock. + +### 11.6 Partner configuration additions + +The following fields are added to the partner record schema (Section 13.3): + +```json +{ + "pull_sync_enabled": true, + "pull_sync_url": "https://api.example-ssp.com/ts/resolve", + "pull_sync_ttl_sec": 86400, + "ts_pull_token": "" +} +``` + +### 11.7 Security + +- The `pull_sync_url` domain must be on an allowlist declared in the partner record. Trusted Server will not call arbitrary URLs. +- Pull sync calls are one-way data flows: TS sends only the EC hash and IP. No other user data (consent string, geo, other partner IDs) is included in the pull request. +- Rate limiting: a maximum of `pull_sync_rate_limit` pull calls per EC hash per partner per hour (configurable per partner, default 10). This prevents the pull mechanism from being used as a polling channel. + +### 11.8 User stories + +**As an SSP**, I want Trusted Server to call my resolution endpoint when it sees a user I might know, so that my uid is available for bidstream decoration without requiring the publisher to include a sync pixel in their page. + +**Acceptance criteria:** +- [ ] When a request arrives with a valid `ts-ec` cookie and a partner with `pull_sync_enabled: true` has no KV entry (or a stale entry), a pull call is dispatched asynchronously after the response is sent +- [ ] A successful pull response with a non-null `uid` results in a KV write within 1 second +- [ ] A `null` or `404` response results in no KV write and no error logged above `DEBUG` level +- [ ] Pull calls are not initiated during the pixel sync flow (no double-write) +- [ ] Rate limit is enforced: more than `pull_sync_rate_limit` pull calls per EC hash per partner per hour are suppressed +- [ ] Pull calls do not add measurable latency to the user-facing response (async dispatch) + +--- + +## 12. Bidstream Decoration + +### 12.1 Two integration modes + +Trusted Server exposes two modes for injecting EC identity into the bidstream. Publishers choose the mode that fits their existing ad stack. + +### 12.2 Mode A: Identity resolution (`/identify`) + +Trusted Server exposes `/identify` as a standalone identity resolution endpoint for callers that need EC identity and resolved partner UIDs outside of TS's own auction orchestration. TS builds the OpenRTB request in Mode B — `/identify` is not part of that path. It serves three distinct use cases: + +**Use case 1 — Attribution and analytics** +Any server-side or browser-side system that needs to tag an event, impression, or conversion with the user's EC hash. Examples: analytics pipelines, attribution platforms, reporting dashboards. + +**Use case 2 — Publisher ad server outbid context** +After TS's auction completes and winners are delivered to the publisher's ad server endpoint, the publisher's ad server may need EC identity and resolved partner UIDs to evaluate whether to accept the programmatic winner or outbid with a direct-sold placement. For this use case, TS includes the EC identity in the winner notification payload directly (see Section 12.3) — a separate `/identify` call is only needed if the publisher's ad server receives the winner through a path that does not carry TS headers. + +**Use case 3 — Client-side wrappers for non-TS SSPs** +Some SSPs run client-side header bidding wrappers (e.g., Amazon TAM, certain Index Exchange configurations) that do not participate in TS's server-side auction orchestration. A Prebid.js module or custom wrapper script calls `/identify` from the browser to obtain the EC hash and resolved partner UIDs, then injects those values into bid requests sent to those SSPs. This ensures non-TS demand sources bid with the same identity enrichment as TS-orchestrated bids, enabling a fair comparison at winner selection. + +> **Prerequisite for use case 3:** For a non-TS SSP to receive a useful UID from `/identify`, that SSP must already be a registered partner in `partner_store` and must have a resolved uid in the KV identity graph for this user (via pixel sync, S2S batch, or S2S pull). Without a prior sync, `/identify` returns no uid for that partner. **Endpoint:** `GET /identify` -**Response:** `204 No Content` with the following headers: +**When to call:** Once per auction event — not per-pageview. For use case 3, call before sending bid requests to non-TS SSPs. -| Header | Value | -| ------------------- | ------------------------------------------------------------- | -| `X-ts-ssc` | `` | -| `X-ts-eids` | Base64-encoded JSON array of OpenRTB 2.6 `user.eids` objects | -| `X-ts-` | Resolved UID per partner (e.g., `X-ts-uid2`, `X-ts-liveramp`) | +#### Call patterns + +**Pattern 1 — Browser-direct (recommended for use cases 1 and 3)** + +A script on the publisher's page calls `/identify` via `fetch()`. Because `ec.publisher.com` is same-site with the publisher's domain, the browser sends the `ts-ec` cookie and consent cookies automatically. No forwarding required. + +```js +const identity = await fetch('https://ec.publisher.com/identify') + .then(r => r.json()); + +// GAM key-value targeting +googletag.pubads().setTargeting('ts_ec', identity.ec); +googletag.pubads().setTargeting('ts_uid2', identity.uids.uid2); + +// Prebid.js userIds injection +pbjs.setConfig({ userSync: { userIds: [{ name: 'uid2', value: { id: identity.uids.uid2 } }] } }); +``` -**If consent is not present:** +**Pattern 2 — Origin-server proxy (for use case 2 when TS winner headers are unavailable)** +A server-side caller must forward the following from the original browser request: + +| Header to forward | Required | +|---|---| +| `Cookie: ts-ec=` or `X-ts-ec: ` | Yes — without this, TS cannot identify the user | +| `Cookie: euconsent-v2=` or `Cookie: gpp=` | Yes — without this, TS returns `consent: denied` and no identity data | +| `X-consent-advertising: ` | Optional — takes precedence over cookie consent if present | + +#### Cookie and consent handling + +`/identify` follows the EC retrieval priority from Section 6.4. It does not generate a new EC — if no EC is present, the response body contains `consent: denied` and empty identity fields. Consent is evaluated per Section 7.1. `/identify` never sets or modifies cookies. + +#### Response + +**`200 OK` — identity resolved** + +EC is present and consent is valid. Identity values are returned as a JSON body. Callers use these values to construct URL parameters for GAM, SSP bid requests, analytics events, or any other downstream system. + +```json +{ + "ec": "a1b2c3...AbC123", + "consent": "ok", + "degraded": false, + "uids": { + "uid2": "A4A...", + "liveramp": "LR_xyz", + "id5": "ID5-abc" + }, + "eids": [ + { "source": "uidapi.com", "uids": [{ "id": "A4A...", "atype": 3 }] }, + { "source": "liveramp.com", "uids": [{ "id": "LR_xyz", "atype": 3 }] } + ] +} ``` -HTTP 204 No Content -X-ts-ssc-consent: denied + +`uids` contains one key per partner with `bidstream_enabled: true` and a resolved UID in the KV graph. Partners with no resolved UID for this user are omitted. + +**`200 OK` — KV unavailable (degraded)** + +EC is present and consent is valid, but the KV read failed. The EC hash is returned; `uids` and `eids` are empty. `degraded: true` distinguishes this from a user who simply has no synced partners yet — callers should proceed with EC-only targeting and may retry on the next auction. + +```json +{ + "ec": "a1b2c3...AbC123", + "consent": "ok", + "degraded": true, + "uids": {}, + "eids": [] +} ``` -No identity headers are returned. The publisher's ad server must handle this case — typically by omitting `user.eids` from the bid request. +**`403 Forbidden` — consent denied** + +EC is present but the user has not given consent (or consent has been withdrawn). Callers must omit identity parameters from all downstream requests. The status code alone is sufficient to detect this case — body parsing is not required. + +```json +{ "consent": "denied" } +``` -### 11.3 Mode B: Full auction orchestration (`/auction`) +**`204 No Content` — no EC present** -For publishers using Trusted Server as their auction endpoint, SSC identity is injected directly into outbound OpenRTB requests to Prebid Server. This is an extension of the existing `/auction` endpoint behavior. +No `ts-ec` cookie and no `X-ts-ec` header was found on the request. The user has not yet established an EC on this publisher. No body is returned. Callers should proceed without identity enrichment. -**Changes from current behavior:** +#### Response headers (supplementary) -- `user.id` is set to the full SSC value (`hash.suffix`) +In addition to the JSON body, TS sets the following response headers for server-to-server callers, logging, and future use. These are not the primary integration contract — callers should read the JSON body. + +| Header | Value | +|---|---| +| `X-ts-ec` | `` or absent if no EC | +| `X-ts-eids` | Base64-encoded JSON array of OpenRTB 2.6 `user.eids` objects | +| `X-ts-` | Resolved UID per partner (e.g., `X-ts-uid2`, `X-ts-liveramp`) | +| `X-ts-ec-consent` | `ok` or `denied` | + +### 12.3 Mode B: Full auction orchestration (`/auction`) + +Trusted Server owns the full auction path in Mode B. TS builds the OpenRTB request, injects EC identity and resolved partner UIDs, sends it to Prebid Server, receives bids, selects winners, and delivers the winner set to the publisher's ad server endpoint. The publisher's ad server does not build the OpenRTB request — it receives auction winners from TS and either accepts the programmatic winner or outbids it with a direct-sold placement. + +**EC injection into the outbound OpenRTB request (changes from current behavior):** + +- `user.id` is set to the full EC value (`hash.suffix`) - `user.eids` is populated from the KV identity graph for this user (see OpenRTB structure below) - `user.consent` is set to the decoded TCF string (currently always `null`) - SSP-specific `ext.eids`: when calling a specific PBS adapter, only that SSP's resolved ID is included in the adapter-level `ext.eids`. All configured identity providers are included at the top-level `user.eids`. -### 11.4 OpenRTB 2.6 `user.eids` structure +**EC context in winner notification to publisher's ad server:** + +When TS delivers auction winners to the publisher's ad server endpoint, the response includes EC identity so the publisher's ad server has full context for its outbid decision without needing to call `/identify` separately: + +| Header | Value | +|---|---| +| `X-ts-ec` | `` | +| `X-ts-eids` | Base64-encoded JSON array of OpenRTB 2.6 `user.eids` objects | +| `X-ts-ec-consent` | `ok` or `denied` | + +### 12.4 OpenRTB 2.6 `user.eids` structure ```json { @@ -518,9 +866,9 @@ For publishers using Trusted Server as their auction endpoint, SSC identity is i } ``` -`atype` values follow the OpenRTB 2.6 specification: `1` = cookie/device, `2` = hashed email, `3` = partner-defined. All SSC-derived IDs use `atype: 3`. +`atype` values follow the OpenRTB 2.6 specification: `1` = cookie/device, `2` = hashed email, `3` = partner-defined. All EC-derived IDs use `atype: 3`. -### 11.5 Partner taxonomy +### 12.5 Partner taxonomy Each partner registered in `partner_store` declares: @@ -528,107 +876,81 @@ Each partner registered in `partner_store` declares: - `openrtb_atype`: integer (typically `3`) - `bidstream_enabled`: boolean — whether this partner's UID should appear in `user.eids` on auction requests -### 11.6 User stories +### 12.6 User stories -**As a publisher using Mode A**, I want to call `/identify` from my ad server so that I can enrich my own auction requests with SSC identity signals without changing my auction infrastructure. +**As a publisher using Mode A for analytics/attribution**, I want to call `/identify` from a browser script so that I can tag events and impressions with the user's EC hash and resolved partner UIDs using URL parameters. **Acceptance criteria:** +- [ ] `GET /identify` returns `200` with a valid JSON body within 30ms when EC is present and consent is valid +- [ ] `uids` object contains one key per partner with `bidstream_enabled: true` and a resolved UID; partners with no resolved UID are omitted +- [ ] If consent is denied, response is `403 Forbidden` with body `{"consent": "denied"}` +- [ ] If no EC is present, response is `204 No Content` with no body +- [ ] Response headers `X-ts-ec`, `X-ts-eids`, `X-ts-`, and `X-ts-ec-consent` are present on `200` responses as supplementary signals -- [ ] `GET /identify` returns `204` with `X-ts-ssc` and `X-ts-eids` headers within 30ms (KV read + response) -- [ ] If consent is denied, response contains `X-ts-ssc-consent: denied` and no identity headers -- [ ] `X-ts-eids` is a valid base64-encoded OpenRTB 2.6 `user.eids` array -- [ ] Individual `X-ts-` headers are present for each partner with `bidstream_enabled: true` and a resolved UID - -**As a publisher using Mode B**, I want Trusted Server to include resolved partner IDs in every auction request so that SSPs receive enriched bid requests without additional publisher-side configuration. +**As a publisher using a client-side wrapper for non-TS SSPs**, I want to call `/identify` from my Prebid.js configuration so that SSPs outside TS's auction receive the same identity enrichment as TS-orchestrated bids, enabling a fair winner comparison. **Acceptance criteria:** +- [ ] `GET /identify` called from the browser returns resolved UIDs for all registered partners with a KV entry for this user +- [ ] A partner with no KV entry for this user is omitted from `uids` — no empty or null entries +- [ ] Response is available within 30ms so it does not block Prebid.js auction timeout + +**As a publisher using Mode B**, I want Trusted Server to build and send enriched OpenRTB requests to Prebid Server and deliver winners to my ad server with full EC context, so my ad server can make outbid decisions without additional identity lookups. -- [ ] Outbound OpenRTB request to PBS contains `user.id` equal to the SSC value +**Acceptance criteria:** +- [ ] Outbound OpenRTB request to PBS contains `user.id` equal to the EC value - [ ] `user.eids` contains one entry per partner with `bidstream_enabled: true` and a resolved UID in the KV graph - [ ] `user.consent` contains the decoded TCF string when available - [ ] Partners without a resolved UID for this user are omitted from `user.eids` (no empty entries) +- [ ] Winner notification to publisher's ad server includes `X-ts-ec`, `X-ts-eids`, and `X-ts-ec-consent` headers --- -## 12. Configuration - -### 12.1 New `[ssc]` section in `trusted-server.toml` - -```toml -[ssc] -enabled = true -ssc_store = "ssc_identity_store" # Fastly KV store: SSC hash → identity graph -partner_store = "ssc_partners" # Fastly KV store: partner ID → config + API key hash -secret_key = "" +## 13. Configuration -# Partner configs live in partner_store KV, not in TOML. -# Use the admin tooling to provision new partners. -# This allows key rotation without redeploying the binary. -``` - -### 12.2 New `[features]` section - -```toml -[features] -# Full TS defaults: all true -# TS Lite defaults: set the following to false -auction = true -js_injection = true -html_processing = true -proxy_routes = true -request_signing = true -ssc = true -``` +The following capabilities must be configurable without redeploying the binary: -### 12.3 Partner record schema (in `partner_store` KV) +- **EC enable/disable** — EC can be turned on or off per deployment +- **Publisher passphrase** — the HMAC key used for EC hash generation; same value across all of the publisher's domains; shared with trusted partners to form an identity-federated consortium +- **Identity graph store** — the KV store backing the EC hash → identity graph +- **Partner registry store** — the KV store backing partner configuration and API key validation +- **Partner records** — each partner's allowed sync domains, bidstream settings, pull sync configuration, and API credentials; managed via `/admin/partners/register` without redeployment -KV key: the partner ID string (e.g., `"ssp_x"`) - -```json -{ - "name": "Example SSP", - "key_hash": "$2b$12$...", - "source_domain": "example-ssp.com", - "openrtb_atype": 3, - "bidstream_enabled": true, - "allowed_return_domains": ["sync.example-ssp.com"], - "sync_rate_limit": 100 -} -``` +The exact configuration format (TOML keys, KV schema, JSON field names) is an engineering decision and will be documented in the technical design doc. --- -## 13. Documentation Updates +## 14. Documentation Updates -The following documentation changes are required alongside the SSC feature: +The following documentation changes are required alongside the EC feature: -- **Rename SyntheticID → Server-Side Cookie** across the entire `docs/` GitHub Pages site. The underlying concept is the same but the product name changes. +- **Rename SyntheticID → Edge Cookie** across the entire `docs/` GitHub Pages site. The underlying concept is the same but the product name changes. - **New integration guides**, one per customer type: - - Publisher (TS Lite): setting up `ssc.publisher.com`, configuring `trusted-server.toml`, DNS CNAME setup - - SSP: pixel sync integration guide, sync pixel URL format, callback handling - - DSP: S2S batch API reference, authentication, conflict resolution behavior + - Publisher (full TS): enabling EC in `trusted-server.toml`, partner onboarding via `/admin/partners/register` + - SSP: pixel sync integration guide, sync pixel URL format, callback handling, optional pull resolution endpoint + - DSP: S2S batch API reference, authentication, conflict resolution behavior, optional pull resolution endpoint - Identity Provider: registering as a partner, `source_domain` and `openrtb_atype` configuration, sync patterns -- **API reference** for the three new endpoints: `GET /sync`, `GET /identify`, `POST /api/v1/sync` +- **API reference** for the four new endpoints: `GET /sync`, `GET /identify`, `POST /api/v1/sync`, and the partner-side pull resolution contract +- **Pull sync integration guide**: partner requirements for exposing a resolution endpoint, authentication, expected response shape, rate limit behavior - **Consent enforcement guide**: how TCF and GPP signals are read, precedence rules, what happens on withdrawal --- -## 14. Open Questions +## 15. Open Questions -| # | Question | Owner | Target resolution | -| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | -------------------------- | -| 1 | Partner provisioning flow: should partner records be written manually by a TS admin, or via a `/admin/partners/register` endpoint using the existing admin auth pattern? The latter is more scalable but requires additional implementation. | Product | Before engineering kickoff | -| 2 | Should TS Lite expose a `GET /health` endpoint so partners can programmatically verify their service is running and their partner config is active in KV? | Product | Before engineering kickoff | +| # | Question | Owner | Status | +|---|---|---|---| +| 1 | Partner provisioning: TS will expose a `/admin/partners/register` endpoint authenticated at the publisher level (bearer token issued per publisher Fastly service), so publishers can onboard SSP/DSP partners without touching KV directly. Engineering to define the exact auth mechanism. | Engineering | **Resolved** — `/admin/partners/register` endpoint, publisher-authenticated | +| 2 | Should TS Lite expose a `GET /health` endpoint so partners can programmatically verify their service is running and their partner config is active in KV? | Product | **N/A** — TS Lite deferred (see Section 5) | --- -## 15. Success Metrics +## 16. Success Metrics -| Metric | Target | Measurement method | -| -------------------------------- | ---------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | -| SSC match rate (returning users) | >90% within 30 days | Fastly real-time logs: ratio of requests with existing `ts-ssc` cookie vs. new SSC generations | -| Consent enforcement accuracy | 0 SSCs created for opted-out EU/UK users | Log audit: verify no `ts-ssc` `Set-Cookie` in responses where consent signal is absent | -| KV sync latency (pixel sync) | p99 <75ms end-to-end | Fastly log timing on `/sync` endpoint | -| S2S batch API throughput | >500 mappings/sec sustained | Load test prior to partner onboarding | -| Identity graph fill rate | >50% of SSC hashes with at least 1 resolved partner ID within 60 days of partner go-live | KV scan sample | -| TS Lite adoption | First non-publisher customer (SSP or DSP) live within 90 days of launch | Customer record | +| Metric | Target | Measurement method | +|---|---|---| +| EC match rate (returning users) | >90% within 30 days | Fastly real-time logs: ratio of requests with existing `ts-ec` cookie vs. new EC generations | +| Consent enforcement accuracy | 0 ECs created for opted-out EU/UK users | Log audit: verify no `ts-ec` `Set-Cookie` in responses where consent signal is absent | +| KV sync latency (pixel sync) | p99 <75ms end-to-end | Fastly log timing on `/sync` endpoint | +| S2S batch API throughput | >500 mappings/sec sustained | Load test prior to partner onboarding | +| S2S pull sync resolution rate | >30% of pull calls return a non-null uid within 60 days of first partner go-live | Fastly log: pull call outcomes per partner | +| Identity graph fill rate | >50% of EC hashes with at least 1 resolved partner ID within 60 days of partner go-live | KV scan sample | From 9529683ed45f8e04246edfa6cc3d3ff86e7b24a3 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Wed, 18 Mar 2026 19:26:51 -0700 Subject: [PATCH 3/4] Fixed formatting --- docs/internal/ssc-prd.md | 256 +++++++++++++++++++++------------------ 1 file changed, 138 insertions(+), 118 deletions(-) diff --git a/docs/internal/ssc-prd.md b/docs/internal/ssc-prd.md index 3269204b..7f88ef15 100644 --- a/docs/internal/ssc-prd.md +++ b/docs/internal/ssc-prd.md @@ -55,7 +55,7 @@ SyntheticID is created unconditionally. There is no mechanism to check TCF (EU/U ### 2.3 Publishers need a reliable, deterministic signal that can be explicitly shared -Today, regular cookies don't suffice for publisher and partner needs. Additionally, only having these identifiers in the 1st party domain's cookie have created slow, undesirable behaviour in the form of cookie syncs. +Today, regular cookies don't suffice for publisher and partner needs. Additionally, only having these identifiers in the 1st party domain's cookie have created slow, undesirable behaviour in the form of cookie syncs. --- @@ -86,13 +86,13 @@ Today, regular cookies don't suffice for publisher and partner needs. Additional **This iteration** targets publishers running the full Trusted Server stack. SSP, DSP, and identity provider customers interact with EC via the sync and bidstream endpoints but do not require a separate TS deployment. -| Customer type | Deployment mode | Primary value | In scope | -|---|---|---|---| -| Publisher (full TS) | Full TS + EC enabled | Consent-aware first-party ID, bidstream enrichment, identity graph | **Yes** | -| SSP | Partner — integrates via pixel sync and/or S2S pull | Build match table against EC hash; receive enriched bid requests | **Yes** (as partner) | -| DSP | Partner — integrates via S2S batch and/or S2S pull | Push/receive ID mappings; enriched bid requests | **Yes** (as partner) | -| Identity provider | Partner — integrates via S2S batch | Sync resolved IDs into the KV identity graph | **Yes** (as partner) | -| Publisher (EC only) | TS Lite at `ec.publisher.com` | First-party cookie at apex domain without full TS | Deferred (see Section 5) | +| Customer type | Deployment mode | Primary value | In scope | +| ------------------- | --------------------------------------------------- | ------------------------------------------------------------------ | ------------------------ | +| Publisher (full TS) | Full TS + EC enabled | Consent-aware first-party ID, bidstream enrichment, identity graph | **Yes** | +| SSP | Partner — integrates via pixel sync and/or S2S pull | Build match table against EC hash; receive enriched bid requests | **Yes** (as partner) | +| DSP | Partner — integrates via S2S batch and/or S2S pull | Push/receive ID mappings; enriched bid requests | **Yes** (as partner) | +| Identity provider | Partner — integrates via S2S batch | Sync resolved IDs into the KV identity graph | **Yes** (as partner) | +| Publisher (EC only) | TS Lite at `ec.publisher.com` | First-party cookie at apex domain without full TS | Deferred (see Section 5) | --- @@ -106,20 +106,20 @@ TS Lite is a runtime configuration of the existing Trusted Server binary. It is ### 5.2 Route surface in TS Lite -| Route | Full TS | TS Lite | -|---|---|---| -| `GET /static/tsjs=` | Enabled | Disabled | -| `POST /auction` | Enabled | Optional (configurable) | -| `GET /first-party/proxy` | Enabled | Disabled | -| `GET /first-party/click` | Enabled | Disabled | -| `POST /first-party/sign` | Enabled | Disabled | -| `GET /first-party/proxy-rebuild` | Enabled | Disabled | -| HTML injection pipeline | Enabled | Disabled | -| GTM integration | Enabled | Disabled | -| `GET /sync` | Disabled | **Enabled** | -| `GET /identify` | Disabled | **Enabled** | -| `POST /api/v1/sync` | Disabled | **Enabled** | -| `GET /.well-known/trusted-server.json` | Enabled | Enabled | +| Route | Full TS | TS Lite | +| -------------------------------------- | -------- | ----------------------- | +| `GET /static/tsjs=` | Enabled | Disabled | +| `POST /auction` | Enabled | Optional (configurable) | +| `GET /first-party/proxy` | Enabled | Disabled | +| `GET /first-party/click` | Enabled | Disabled | +| `POST /first-party/sign` | Enabled | Disabled | +| `GET /first-party/proxy-rebuild` | Enabled | Disabled | +| HTML injection pipeline | Enabled | Disabled | +| GTM integration | Enabled | Disabled | +| `GET /sync` | Disabled | **Enabled** | +| `GET /identify` | Disabled | **Enabled** | +| `POST /api/v1/sync` | Disabled | **Enabled** | +| `GET /.well-known/trusted-server.json` | Enabled | Enabled | When a disabled route is requested, TS returns `404` with the header `X-ts-error: feature-disabled`. @@ -147,18 +147,20 @@ The EC is generated by HMAC-SHA256 of a fixed input set, using a publisher-speci **Inputs (IP address + salt only):** -| Input | Value | -|---|---| -| IP address | IPv4 as-is; IPv6 summarized to /64 prefix (first 4 hextets) — discards rotating interface ID. On dual-stack, IPv6 is preferred. | +| Input | Value | +| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| IP address | IPv4 as-is; IPv6 summarized to /64 prefix (first 4 hextets) — discards rotating interface ID. On dual-stack, IPv6 is preferred. | | Secret key | Publisher-chosen passphrase, configured in `trusted-server.toml`. Consistent across all of the publisher's own domains. Publishers who share the same passphrase with other publishers form an identity-federated consortium — the same user produces the same EC hash across all consortium members. Publishers using different passphrases produce unrelated hashes with no cross-property linkage. | **Removed from SyntheticID:** + - `User-Agent` - `Accept-Language` - `Accept-Encoding` - Handlebars template (input is now fixed, not configurable) **Output format (unchanged from SyntheticID):** + ``` {64-character hex HMAC-SHA256}.{6-character random alphanumeric suffix} ``` @@ -169,15 +171,15 @@ The 64-character prefix is the stable, deterministic portion used as the KV stor ### 6.2 Cookie attributes -| Attribute | Value | -|---|---| -| Name | `ts-ec` | -| Domain | `.publisher.com` (derived from `publisher.domain` in TOML) | -| Path | `/` | -| Secure | Yes | -| SameSite | `Lax` | -| Max-Age | `31536000` (1 year) | -| HttpOnly | No — JavaScript on `www.publisher.com` may need to read the value for ad stack decoration | +| Attribute | Value | +| --------- | ----------------------------------------------------------------------------------------- | +| Name | `ts-ec` | +| Domain | `.publisher.com` (derived from `publisher.domain` in TOML) | +| Path | `/` | +| Secure | Yes | +| SameSite | `Lax` | +| Max-Age | `31536000` (1 year) | +| HttpOnly | No — JavaScript on `www.publisher.com` may need to read the value for ad stack decoration | ### 6.3 Response header @@ -222,25 +224,28 @@ When a consent signal is required for the user's region, Trusted Server checks s Before creating a new EC, Trusted Server first evaluates the user's region (via Fastly's `x-geo-country` header) to determine whether a consent signal is required. If the region requires a signal, TS reads it using the precedence order in Section 7.1; if no signal is found, creation is blocked (the fail-safe in step 4 applies). If the region does not require a signal, TS creates the EC unconditionally. -| Region | Required signal | Rule | -|---|---|---| -| EU member states | TCF string | Create EC only if `purposeConsents[1]` (store and/or access information on a device) is `true`. If no TCF signal is found, do not create EC (7.1 step 4 applies). | -| United Kingdom | TCF string | Same as EU | -| US states with privacy laws (CA, CO, CT, VA, TX, OR, MT, DE, NH, NJ, TN, IN, IA, KY, NE, MD, MN, RI) | GPP string | Create EC unless user has opted out of sale or sharing of personal data. If no GPP signal is found, do not create EC (7.1 step 4 applies). | -| Rest of world | None required | Create EC on first visit regardless of whether any consent signal is present. Section 7.1 step 4 does not apply. | +| Region | Required signal | Rule | +| ---------------------------------------------------------------------------------------------------- | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| EU member states | TCF string | Create EC only if `purposeConsents[1]` (store and/or access information on a device) is `true`. If no TCF signal is found, do not create EC (7.1 step 4 applies). | +| United Kingdom | TCF string | Same as EU | +| US states with privacy laws (CA, CO, CT, VA, TX, OR, MT, DE, NH, NJ, TN, IN, IA, KY, NE, MD, MN, RI) | GPP string | Create EC unless user has opted out of sale or sharing of personal data. If no GPP signal is found, do not create EC (7.1 step 4 applies). | +| Rest of world | None required | Create EC on first visit regardless of whether any consent signal is present. Section 7.1 step 4 does not apply. | ### 7.3 Consent withdrawal (real-time enforcement) On every request, Trusted Server decodes the consent signal (a microsecond in-memory operation with no I/O). If consent is not present or has been revoked: **If `ts-ec` cookie is present:** + 1. Delete the cookie by issuing `Set-Cookie: ts-ec=; Max-Age=0; Domain=.publisher.com; Path=/; Secure; SameSite=Lax` 2. Delete the KV identity graph entry: `kv_store.delete(ec_hash)` — this operation takes approximately 25ms and runs in the request path **If no `ts-ec` cookie is present:** + - Do nothing **If consent is present:** + - Proceed with normal EC create-or-refresh flow **Known tradeoff:** The KV delete adds approximately 25ms of latency to the first request after consent withdrawal. This is an intentional product decision — real-time consent enforcement is a differentiating capability of Trusted Server, and the latency cost is acceptable. @@ -321,9 +326,9 @@ Concurrent writes from different partners to the same KV entry must not overwrit Two KV stores are required: -| Store | TOML key | Contents | -|---|---|---| -| Identity graph | `ec_store` | EC hash → identity graph JSON | +| Store | TOML key | Contents | +| ---------------- | --------------- | ---------------------------------- | +| Identity graph | `ec_store` | EC hash → identity graph JSON | | Partner registry | `partner_store` | Partner ID → config + API key hash | The existing `counter_store` and `opid_store` settings (currently defined but unused in `settings.rs`) can be deprecated in a follow-on cleanup. @@ -332,15 +337,15 @@ The existing `counter_store` and `opid_store` settings (currently defined but un The EC cookie is deterministic (derived from IP + publisher salt) and lives in the browser. It does not depend on KV Store availability. KV Store holds identity enrichment only — resolved partner UIDs accumulated over time. The degraded behavior policy follows from this: **EC always works; enrichment degrades gracefully.** -| Operation | KV unavailable or error | Rationale | -|---|---|---| -| EC cookie creation | Set the cookie. Skip the KV entry creation silently. Log the failure at `warn` level. | The cookie is the identity anchor — it does not require KV. The KV entry will be created on the next request once KV recovers. | -| EC cookie refresh (existing user) | Refresh the cookie. Skip the KV `last_seen` update silently. Log at `warn`. | Same as above — the cookie continues working. Stale `last_seen` is acceptable. | -| `/sync` KV write | Redirect to `return` with `ts_synced=0&ts_reason=write_failed`. | The browser redirect must not be blocked by KV availability. This case is already specified in Section 9.4. | -| `/identify` KV read | Return `200` with `ec` hash (from cookie) and `degraded: true`. Set `uids: {}` and `eids: []`. | The EC hash is still valid and useful for attribution and analytics. Empty uids signal that enrichment is unavailable, not that the user has no synced partners. `degraded: true` lets callers distinguish transient KV failure from a genuinely unenriched user. | -| S2S batch write (`/api/v1/sync`) | Return `207` with all mappings rejected, `reason: "kv_unavailable"`. | The request was valid; the failure is infrastructure. Partners should retry the batch. | -| S2S pull sync write (async) | Discard the resolved uid. Log at `warn`. Retry will occur on the next qualifying request per the `pull_sync_ttl_sec` window. | Async path — no user-facing impact. | -| Consent withdrawal KV delete | Expire the cookie immediately. Log the KV delete failure at `error` level. Retry the KV delete on the next request for this user. | Cookie deletion is the primary enforcement mechanism. KV delete failure must not block or delay the cookie expiry. | +| Operation | KV unavailable or error | Rationale | +| --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| EC cookie creation | Set the cookie. Skip the KV entry creation silently. Log the failure at `warn` level. | The cookie is the identity anchor — it does not require KV. The KV entry will be created on the next request once KV recovers. | +| EC cookie refresh (existing user) | Refresh the cookie. Skip the KV `last_seen` update silently. Log at `warn`. | Same as above — the cookie continues working. Stale `last_seen` is acceptable. | +| `/sync` KV write | Redirect to `return` with `ts_synced=0&ts_reason=write_failed`. | The browser redirect must not be blocked by KV availability. This case is already specified in Section 9.4. | +| `/identify` KV read | Return `200` with `ec` hash (from cookie) and `degraded: true`. Set `uids: {}` and `eids: []`. | The EC hash is still valid and useful for attribution and analytics. Empty uids signal that enrichment is unavailable, not that the user has no synced partners. `degraded: true` lets callers distinguish transient KV failure from a genuinely unenriched user. | +| S2S batch write (`/api/v1/sync`) | Return `207` with all mappings rejected, `reason: "kv_unavailable"`. | The request was valid; the failure is infrastructure. Partners should retry the batch. | +| S2S pull sync write (async) | Discard the resolved uid. Log at `warn`. Retry will occur on the next qualifying request per the `pull_sync_ttl_sec` window. | Async path — no user-facing impact. | +| Consent withdrawal KV delete | Expire the cookie immediately. Log the KV delete failure at `error` level. Retry the KV delete on the next request for this user. | Cookie deletion is the primary enforcement mechanism. KV delete failure must not block or delay the cookie expiry. | **`degraded: true` in `/identify` responses** @@ -360,7 +365,7 @@ When a KV read fails, the `/identify` response includes `"degraded": true` in th #### Problem -Code attestation (reproducible WASM builds + published binary hashes) proves that the TS binary running on Fastly's infrastructure matches the open-source repository. It does not, however, prove that the *data* inside `ec_store` was written by that attested binary. A malicious or compromised operator could write arbitrary identity mappings directly into the KV store — bypassing all code paths — and buyers would have no way to detect it. +Code attestation (reproducible WASM builds + published binary hashes) proves that the TS binary running on Fastly's infrastructure matches the open-source repository. It does not, however, prove that the _data_ inside `ec_store` was written by that attested binary. A malicious or compromised operator could write arbitrary identity mappings directly into the KV store — bypassing all code paths — and buyers would have no way to detect it. #### Solution: JOSE-signed KV entry bodies @@ -376,7 +381,7 @@ At read time, the TS binary verifies the JWS signature before consuming any fiel **What it does not prove:** -- That the *input data* (e.g., a partner-supplied UID) was accurate at the time of write. Signal accuracy remains the partner's responsibility. +- That the _input data_ (e.g., a partner-supplied UID) was accurate at the time of write. Signal accuracy remains the partner's responsibility. - Anything about entries written before this feature was deployed. A migration pass will resign existing entries or treat them as unsigned (degraded) until they are refreshed by a normal TS write. #### Attestation record endpoint @@ -394,7 +399,13 @@ Response (application/json): "version": "1.0", "jwks": { "keys": [ - { "kty": "OKP", "crv": "Ed25519", "kid": "ts-2026-A", "use": "sig", "x": "..." } + { + "kty": "OKP", + "crv": "Ed25519", + "kid": "ts-2026-A", + "use": "sig", + "x": "..." + } ] }, "attestation": { @@ -408,12 +419,12 @@ Response (application/json): The `jwks` field is unchanged — it continues to serve request signing keys on its existing rotation schedule. The `attestation` object is a separate namespace and does not affect existing consumers of this endpoint. -| Field | Description | -|---|---| -| `attestation.binary_hash` | SHA-256 hex of the deployed WASM binary. Cross-referenced with Fastly's signed deployment manifest in the reproducible builds PRD. | -| `attestation.alg` | JWS algorithm used for all KV entry signatures. Fixed at `ES256` (ECDSA P-256). | -| `attestation.jwk` | Public key in JWK format (RFC 7517). Buyers use this to verify signatures in KV-derived `user.eids`. Distinct from the `jwks` request-signing keys. | -| `attestation.expires_at` | UTC timestamp after which the attestation key should be considered untrustworthy. Buyers must re-fetch before this time. | +| Field | Description | +| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | +| `attestation.binary_hash` | SHA-256 hex of the deployed WASM binary. Cross-referenced with Fastly's signed deployment manifest in the reproducible builds PRD. | +| `attestation.alg` | JWS algorithm used for all KV entry signatures. Fixed at `ES256` (ECDSA P-256). | +| `attestation.jwk` | Public key in JWK format (RFC 7517). Buyers use this to verify signatures in KV-derived `user.eids`. Distinct from the `jwks` request-signing keys. | +| `attestation.expires_at` | UTC timestamp after which the attestation key should be considered untrustworthy. Buyers must re-fetch before this time. | **Key TTL:** 90 days. The attestation key rotates on each new TS deployment. The previous key's `expires_at` is set 7 days after rotation to allow in-flight impressions to drain. @@ -433,7 +444,7 @@ The `jwks` field is unchanged — it continues to serve request signing keys on #### Relationship to reproducible builds PRD -JOSE-signed KV entries close the *data integrity* gap that code attestation leaves open. Reproducible builds and published binary hashes address the *code integrity* layer — proving that the deployed binary matches the audited source. These are complementary controls that together form a complete trust chain for buyers. +JOSE-signed KV entries close the _data integrity_ gap that code attestation leaves open. Reproducible builds and published binary hashes address the _code integrity_ layer — proving that the deployed binary matches the audited source. These are complementary controls that together form a complete trust chain for buyers. The reproducible builds feature has broader scope than the identity graph (it applies to all TS behaviour, not just KV writes) and will be specified in a dedicated PRD. The `attestation.binary_hash` field in `trusted-server.json` anticipates that PRD — buyers can record it today, and the reproducible builds PRD will define the process for independently verifying it against Fastly's signed deployment manifest. @@ -455,12 +466,12 @@ GET /sync ### 9.3 Parameters -| Parameter | Required | Description | -|---|---|---| -| `partner` | Yes | Partner ID, must match a registered partner in `partner_store` KV | -| `uid` | Yes | Partner's user ID for this user | -| `return` | Yes | Callback URL to redirect to after sync (must match partner's `allowed_return_domains`) | -| `consent` | No | TCF or GPP string from the partner's context, used if no consent signal is present on the request | +| Parameter | Required | Description | +| --------- | -------- | ------------------------------------------------------------------------------------------------- | +| `partner` | Yes | Partner ID, must match a registered partner in `partner_store` KV | +| `uid` | Yes | Partner's user ID for this user | +| `return` | Yes | Callback URL to redirect to after sync (must match partner's `allowed_return_domains`) | +| `consent` | No | TCF or GPP string from the partner's context, used if no consent signal is present on the request | ### 9.4 Flow @@ -473,11 +484,11 @@ GET /sync **`ts_synced` values:** -| Value | Meaning | -|---|---| -| `ts_synced=1` | KV write succeeded — partner uid is now in the identity graph | -| `ts_synced=0&ts_reason=no_ec` | No EC cookie present — user has not established an EC on this publisher | -| `ts_synced=0&ts_reason=no_consent` | Consent absent or invalid — write suppressed | +| Value | Meaning | +| ------------------------------------ | --------------------------------------------------------------------------- | +| `ts_synced=1` | KV write succeeded — partner uid is now in the identity graph | +| `ts_synced=0&ts_reason=no_ec` | No EC cookie present — user has not established an EC on this publisher | +| `ts_synced=0&ts_reason=no_consent` | Consent absent or invalid — write suppressed | | `ts_synced=0&ts_reason=write_failed` | KV write failed after retries — partner should retry on a future pixel fire | Partners should treat `ts_synced=0` as a signal that the mapping was not stored. The `ts_reason` parameter is informational; partners should not gate their own behavior on specific reason values. @@ -493,6 +504,7 @@ Partners should treat `ts_synced=0` as a signal that the mapping was not stored. **As an SSP**, I want to fire a sync pixel when I see a user so that I can associate my user ID with the EC hash and receive enriched bid requests when the publisher calls Trusted Server for auction. **Acceptance criteria:** + - [ ] `GET /sync?partner=ssp_x&uid=abc&return=https://sync.ssp.com/ack` returns a redirect to the `return` URL within 50ms (excluding KV write time) - [ ] KV entry for the EC hash contains `ids.ssp_x.uid = "abc"` after a successful sync; response redirects to `return` with `ts_synced=1` - [ ] If no `ts-ec` cookie is present, redirects to `return` with `ts_synced=0&ts_reason=no_ec`; no KV write performed @@ -555,13 +567,13 @@ Maximum batch size per request: 1000 mappings (subject to revision based on KV w **HTTP status rules:** -| Condition | Status | -|---|---| -| All mappings accepted | `200 OK` | -| Some mappings accepted, some rejected | `207 Multi-Status` | -| Auth valid, batch valid, but **all** mappings rejected | `207 Multi-Status` with `accepted: 0` | -| Auth invalid | `401 Unauthorized` (no body processing) | -| Batch exceeds 1000 mappings or malformed JSON | `400 Bad Request` (no body processing) | +| Condition | Status | +| ------------------------------------------------------ | --------------------------------------- | +| All mappings accepted | `200 OK` | +| Some mappings accepted, some rejected | `207 Multi-Status` | +| Auth valid, batch valid, but **all** mappings rejected | `207 Multi-Status` with `accepted: 0` | +| Auth invalid | `401 Unauthorized` (no body processing) | +| Batch exceeds 1000 mappings or malformed JSON | `400 Bad Request` (no body processing) | A `207` with `accepted: 0` signals "your request was received and processed correctly, but none of the submitted EC hashes were found or eligible." This is distinct from an auth or protocol error. Partners should treat this as a data signal — either the EC hashes are stale/unknown, or consent has been withdrawn for all submitted users — and should not retry the same batch without investigating the underlying cause. @@ -580,6 +592,7 @@ Before writing a mapping, Trusted Server checks the KV metadata for the given EC **As a DSP**, I want to push my user ID mappings to Trusted Server in bulk so that the publisher's auction requests are enriched with my resolved ID and I can bid on users I recognize. **Acceptance criteria:** + - [ ] `POST /api/v1/sync` with a valid Bearer token and a batch of up to 1000 mappings returns a response within 5 seconds - [ ] Accepted mappings are written to the corresponding KV identity graph entries within 1 second - [ ] Mappings for unknown `ec_hash` values are rejected with `ec_hash_not_found` @@ -610,13 +623,13 @@ The partner's resolution endpoint receives the EC hash and IP address. The partn **What it solves and what it doesn't:** -| User scenario | Pixel sync | S2S batch | S2S pull | -|---|---|---|---| -| New user, Chrome, 3p cookies available | Works (bootstraps KV) | Not applicable | No server-side mapping yet — no-op | -| Returning user after prior pixel sync | Redundant (already in KV) | Works | Works (partner has IP or bid-stream mapping) | -| Safari user, partner has IP-based mapping | Blocked / unreliable | Works if partner knows EC hash | Works — partner resolves from their IP graph | -| User unknown to partner by any signal | No uid to sync | No uid to push | No uid to return — no-op | -| Authenticated user with hashed email | Works | Works | Works | +| User scenario | Pixel sync | S2S batch | S2S pull | +| ----------------------------------------- | ------------------------- | ------------------------------ | -------------------------------------------- | +| New user, Chrome, 3p cookies available | Works (bootstraps KV) | Not applicable | No server-side mapping yet — no-op | +| Returning user after prior pixel sync | Redundant (already in KV) | Works | Works (partner has IP or bid-stream mapping) | +| Safari user, partner has IP-based mapping | Blocked / unreliable | Works if partner knows EC hash | Works — partner resolves from their IP graph | +| User unknown to partner by any signal | No uid to sync | No uid to push | No uid to return — no-op | +| Authenticated user with hashed email | Works | Works | Works | S2S pull does not solve the cold-start problem for users the partner has never seen. It degrades gracefully to a no-op in those cases. @@ -692,6 +705,7 @@ The following fields are added to the partner record schema (Section 13.3): **As an SSP**, I want Trusted Server to call my resolution endpoint when it sees a user I might know, so that my uid is available for bidstream decoration without requiring the publisher to include a sync pixel in their page. **Acceptance criteria:** + - [ ] When a request arrives with a valid `ts-ec` cookie and a partner with `pull_sync_enabled: true` has no KV entry (or a stale entry), a pull call is dispatched asynchronously after the response is sent - [ ] A successful pull response with a non-null `uid` results in a KV write within 1 second - [ ] A `null` or `404` response results in no KV write and no error logged above `DEBUG` level @@ -733,26 +747,29 @@ Some SSPs run client-side header bidding wrappers (e.g., Amazon TAM, certain Ind A script on the publisher's page calls `/identify` via `fetch()`. Because `ec.publisher.com` is same-site with the publisher's domain, the browser sends the `ts-ec` cookie and consent cookies automatically. No forwarding required. ```js -const identity = await fetch('https://ec.publisher.com/identify') - .then(r => r.json()); +const identity = await fetch('https://ec.publisher.com/identify').then((r) => + r.json() +) // GAM key-value targeting -googletag.pubads().setTargeting('ts_ec', identity.ec); -googletag.pubads().setTargeting('ts_uid2', identity.uids.uid2); +googletag.pubads().setTargeting('ts_ec', identity.ec) +googletag.pubads().setTargeting('ts_uid2', identity.uids.uid2) // Prebid.js userIds injection -pbjs.setConfig({ userSync: { userIds: [{ name: 'uid2', value: { id: identity.uids.uid2 } }] } }); +pbjs.setConfig({ + userSync: { userIds: [{ name: 'uid2', value: { id: identity.uids.uid2 } }] }, +}) ``` **Pattern 2 — Origin-server proxy (for use case 2 when TS winner headers are unavailable)** A server-side caller must forward the following from the original browser request: -| Header to forward | Required | -|---|---| -| `Cookie: ts-ec=` or `X-ts-ec: ` | Yes — without this, TS cannot identify the user | +| Header to forward | Required | +| ------------------------------------------------------- | --------------------------------------------------------------------- | +| `Cookie: ts-ec=` or `X-ts-ec: ` | Yes — without this, TS cannot identify the user | | `Cookie: euconsent-v2=` or `Cookie: gpp=` | Yes — without this, TS returns `consent: denied` and no identity data | -| `X-consent-advertising: ` | Optional — takes precedence over cookie consent if present | +| `X-consent-advertising: ` | Optional — takes precedence over cookie consent if present | #### Cookie and consent handling @@ -813,12 +830,12 @@ No `ts-ec` cookie and no `X-ts-ec` header was found on the request. The user has In addition to the JSON body, TS sets the following response headers for server-to-server callers, logging, and future use. These are not the primary integration contract — callers should read the JSON body. -| Header | Value | -|---|---| -| `X-ts-ec` | `` or absent if no EC | -| `X-ts-eids` | Base64-encoded JSON array of OpenRTB 2.6 `user.eids` objects | +| Header | Value | +| ------------------- | ------------------------------------------------------------- | +| `X-ts-ec` | `` or absent if no EC | +| `X-ts-eids` | Base64-encoded JSON array of OpenRTB 2.6 `user.eids` objects | | `X-ts-` | Resolved UID per partner (e.g., `X-ts-uid2`, `X-ts-liveramp`) | -| `X-ts-ec-consent` | `ok` or `denied` | +| `X-ts-ec-consent` | `ok` or `denied` | ### 12.3 Mode B: Full auction orchestration (`/auction`) @@ -835,11 +852,11 @@ Trusted Server owns the full auction path in Mode B. TS builds the OpenRTB reque When TS delivers auction winners to the publisher's ad server endpoint, the response includes EC identity so the publisher's ad server has full context for its outbid decision without needing to call `/identify` separately: -| Header | Value | -|---|---| -| `X-ts-ec` | `` | -| `X-ts-eids` | Base64-encoded JSON array of OpenRTB 2.6 `user.eids` objects | -| `X-ts-ec-consent` | `ok` or `denied` | +| Header | Value | +| ----------------- | ------------------------------------------------------------ | +| `X-ts-ec` | `` | +| `X-ts-eids` | Base64-encoded JSON array of OpenRTB 2.6 `user.eids` objects | +| `X-ts-ec-consent` | `ok` or `denied` | ### 12.4 OpenRTB 2.6 `user.eids` structure @@ -881,6 +898,7 @@ Each partner registered in `partner_store` declares: **As a publisher using Mode A for analytics/attribution**, I want to call `/identify` from a browser script so that I can tag events and impressions with the user's EC hash and resolved partner UIDs using URL parameters. **Acceptance criteria:** + - [ ] `GET /identify` returns `200` with a valid JSON body within 30ms when EC is present and consent is valid - [ ] `uids` object contains one key per partner with `bidstream_enabled: true` and a resolved UID; partners with no resolved UID are omitted - [ ] If consent is denied, response is `403 Forbidden` with body `{"consent": "denied"}` @@ -890,6 +908,7 @@ Each partner registered in `partner_store` declares: **As a publisher using a client-side wrapper for non-TS SSPs**, I want to call `/identify` from my Prebid.js configuration so that SSPs outside TS's auction receive the same identity enrichment as TS-orchestrated bids, enabling a fair winner comparison. **Acceptance criteria:** + - [ ] `GET /identify` called from the browser returns resolved UIDs for all registered partners with a KV entry for this user - [ ] A partner with no KV entry for this user is omitted from `uids` — no empty or null entries - [ ] Response is available within 30ms so it does not block Prebid.js auction timeout @@ -897,6 +916,7 @@ Each partner registered in `partner_store` declares: **As a publisher using Mode B**, I want Trusted Server to build and send enriched OpenRTB requests to Prebid Server and deliver winners to my ad server with full EC context, so my ad server can make outbid decisions without additional identity lookups. **Acceptance criteria:** + - [ ] Outbound OpenRTB request to PBS contains `user.id` equal to the EC value - [ ] `user.eids` contains one entry per partner with `bidstream_enabled: true` and a resolved UID in the KV graph - [ ] `user.consent` contains the decoded TCF string when available @@ -937,20 +957,20 @@ The following documentation changes are required alongside the EC feature: ## 15. Open Questions -| # | Question | Owner | Status | -|---|---|---|---| -| 1 | Partner provisioning: TS will expose a `/admin/partners/register` endpoint authenticated at the publisher level (bearer token issued per publisher Fastly service), so publishers can onboard SSP/DSP partners without touching KV directly. Engineering to define the exact auth mechanism. | Engineering | **Resolved** — `/admin/partners/register` endpoint, publisher-authenticated | -| 2 | Should TS Lite expose a `GET /health` endpoint so partners can programmatically verify their service is running and their partner config is active in KV? | Product | **N/A** — TS Lite deferred (see Section 5) | +| # | Question | Owner | Status | +| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | --------------------------------------------------------------------------- | +| 1 | Partner provisioning: TS will expose a `/admin/partners/register` endpoint authenticated at the publisher level (bearer token issued per publisher Fastly service), so publishers can onboard SSP/DSP partners without touching KV directly. Engineering to define the exact auth mechanism. | Engineering | **Resolved** — `/admin/partners/register` endpoint, publisher-authenticated | +| 2 | Should TS Lite expose a `GET /health` endpoint so partners can programmatically verify their service is running and their partner config is active in KV? | Product | **N/A** — TS Lite deferred (see Section 5) | --- ## 16. Success Metrics -| Metric | Target | Measurement method | -|---|---|---| -| EC match rate (returning users) | >90% within 30 days | Fastly real-time logs: ratio of requests with existing `ts-ec` cookie vs. new EC generations | -| Consent enforcement accuracy | 0 ECs created for opted-out EU/UK users | Log audit: verify no `ts-ec` `Set-Cookie` in responses where consent signal is absent | -| KV sync latency (pixel sync) | p99 <75ms end-to-end | Fastly log timing on `/sync` endpoint | -| S2S batch API throughput | >500 mappings/sec sustained | Load test prior to partner onboarding | -| S2S pull sync resolution rate | >30% of pull calls return a non-null uid within 60 days of first partner go-live | Fastly log: pull call outcomes per partner | -| Identity graph fill rate | >50% of EC hashes with at least 1 resolved partner ID within 60 days of partner go-live | KV scan sample | +| Metric | Target | Measurement method | +| ------------------------------- | --------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | +| EC match rate (returning users) | >90% within 30 days | Fastly real-time logs: ratio of requests with existing `ts-ec` cookie vs. new EC generations | +| Consent enforcement accuracy | 0 ECs created for opted-out EU/UK users | Log audit: verify no `ts-ec` `Set-Cookie` in responses where consent signal is absent | +| KV sync latency (pixel sync) | p99 <75ms end-to-end | Fastly log timing on `/sync` endpoint | +| S2S batch API throughput | >500 mappings/sec sustained | Load test prior to partner onboarding | +| S2S pull sync resolution rate | >30% of pull calls return a non-null uid within 60 days of first partner go-live | Fastly log: pull call outcomes per partner | +| Identity graph fill rate | >50% of EC hashes with at least 1 resolved partner ID within 60 days of partner go-live | KV scan sample | From cabe85f71348bf2b810205e6fe07e01d33905aa8 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 19 Mar 2026 08:22:13 -0700 Subject: [PATCH 4/4] Tecnical specification for EC --- docs/internal/ssc_technical_spec.md | 1842 +++++++++++++++++++++++++++ 1 file changed, 1842 insertions(+) create mode 100644 docs/internal/ssc_technical_spec.md diff --git a/docs/internal/ssc_technical_spec.md b/docs/internal/ssc_technical_spec.md new file mode 100644 index 00000000..0fc599b0 --- /dev/null +++ b/docs/internal/ssc_technical_spec.md @@ -0,0 +1,1842 @@ +# Technical Specification: Edge Cookie (EC) + +**Status:** Draft +**Author:** Engineering +**PRD reference:** `docs/internal/ssc-prd.md` +**Last updated:** 2026-03-18 + +--- + +## Table of Contents + +1. [Overview](#1-overview) +2. [Architecture Overview](#2-architecture-overview) +3. [Module Structure](#3-module-structure) +4. [EC Identity Generation](#4-ec-identity-generation) +5. [Cookie and Header Handling](#5-cookie-and-header-handling) +6. [Consent Enforcement](#6-consent-enforcement) +7. [KV Store Identity Graph](#7-kv-store-identity-graph) +8. [Pixel Sync Endpoint (`GET /sync`)](#8-pixel-sync-endpoint-get-sync) +9. [S2S Batch Sync API (`POST /api/v1/sync`)](#9-s2s-batch-sync-api-post-apiv1sync) +10. [S2S Pull Sync (TS-Initiated)](#10-s2s-pull-sync-ts-initiated) +11. [Identity Resolution Endpoint (`GET /identify`)](#11-identity-resolution-endpoint-get-identify) +12. [Bidstream Decoration (`/auction` Mode B)](#12-bidstream-decoration-auction-mode-b) +13. [Partner Registry and Admin Endpoint](#13-partner-registry-and-admin-endpoint) +14. [Configuration](#14-configuration) +15. [Constants and Header Names](#15-constants-and-header-names) +16. [Error Handling](#16-error-handling) +17. [Request Routing](#17-request-routing) +18. [Testing Strategy](#18-testing-strategy) +19. [Implementation Order](#19-implementation-order) + +--- + +## 1. Overview + +Edge Cookie (EC) replaces SyntheticID as the primary user identity mechanism in Trusted Server. It uses a simpler, more stable signal (IP address + publisher passphrase), adds consent enforcement, and backs identity with a server-side KV graph that accumulates partner IDs over time. + +EC is the intended full replacement for SyntheticID. The PRD explicitly states backward compatibility is a non-goal. Coexistence in this spec is a **temporary implementation detail only** — not a product commitment. SyntheticID runs alongside EC solely because the cutover and removal work belongs to a follow-on spec; it is not preserved for compatibility reasons. EC is authoritative where present. + +**Prerequisites (completed before this epic begins):** + +The following work is handled in separate epics that must ship before this epic starts: + +- **SyntheticID → EC rename** — all SyntheticID references in both the product and codebase are renamed to Edge Cookie / EC terminology (e.g. `get_or_generate_synthetic_id` → `get_or_generate_ec`, `ts-synthetic` cookie → `ts-ec`, product-facing naming). This spec assumes the rename is already in place; any SyntheticID naming in existing code shown here reflects the current codebase state at time of writing. +- **Consent implementation** — The consent pipeline (`build_consent_context()`, `ConsentContext`, TCF/GPP/US-Privacy decoding) is implemented and available as a stable interface before this epic. PR `#380` merged to `main`. This spec uses `ConsentContext` as a pre-existing contract and adds only the EC-specific `ec_consent_granted()` gating layer on top. + +**Deferred from this spec (not in scope):** + +- TS Lite deployment mode (PRD Section 5) +- JOSE-signed KV entries / buyer attestation, and the associated `/.well-known/trusted-server.json` attestation object + `Cache-Control: max-age=3600` response (PRD Section 8.7). The existing discovery endpoint and its tests (`endpoints.rs:579–594`) assert only `version` and `jwks` fields — this spec does not modify that endpoint. Any addition of the PRD-required `attestation` field is deferred to when JOSE signing ships. +- Data deletion framework JWT endpoint (PRD Section 7.4) — the formal IAB-compliant deletion endpoint is deferred. The PRD explicitly acknowledges that manual KV deletion is the interim process until the formal endpoint ships, and states that regulated onboarding requires the formal endpoint to be in place first. This spec implements the manual-deletion-only interim; the JWT endpoint is a prerequisite for regulated onboarding and must be tracked separately. +- Winner notification EC headers on publisher ad server delivery (§12.5) — the current `/auction` path returns JSON inline to the JS caller; there is no server-to-server delivery step. §12.5 is deferred until that delivery architecture exists. +- SyntheticID cutover — removing the old SyntheticID module, `X-Synthetic-*` headers, and related code is a follow-on spec. This spec only adds EC alongside the renamed SyntheticID code. + +--- + +## 2. Architecture Overview + +``` +Browser Request + │ + ▼ +┌─────────────────────────────────────────────────┐ +│ main.rs (router) │ +│ extract GeoInfo → enforce auth → route_request │ +└──────────┬──────────────────────────────────────┘ + │ +Two-phase model (matches existing codebase pattern): + +Phase 1 — pre-routing (like `GeoInfo::from_request()`): + ┌─────────────────────────────────────────┐ + │ EcContext::read_from_request() │ + │ - read ts-ec cookie / X-ts-ec header │ + │ - build_consent_context() → ConsentContext │ + │ - ec_consent_granted(consent) │ + │ No generation. No cookie writes. │ + └──────┬──────────────────────────────────┘ + │ +Phase 2 — inside organic handlers only: + ┌───────┼──────────────────────────────────────────────────┐ + │ │ │ + ▼ ▼ ▼ +handle_publisher_request() integration_registry.handle_proxy() +calls ec_context.generate_if_needed() calls ec_context.generate_if_needed() + +EC route handlers (GET /sync, GET /identify, POST /auction, +POST /api/v1/sync, POST /admin/*) NEVER call generate_if_needed(). +EcContext is available to them in read-only form. /auction reads +EC identity but never bootstraps it — the publisher page-load path +generates the EC before any auction request arrives. + +finalize_response() — after every handler: + - consent withdrawn + cookie present? → delete_ec_cookie() [always] + - ec_generated == true? → set_ec_on_response() [new cookie only] +``` + +EC state flows through an `EcContext` struct created once per request and passed through handlers. + +--- + +## 3. Module Structure + +New files in `crates/common/src/`: + +``` +crates/common/src/ + ec/ + mod.rs — EcContext, pub re-exports + identity.rs — EC generation (HMAC-SHA256, IP normalization) + cookie.rs — create_ec_cookie(), delete_ec_cookie(), set_ec_on_response() + consent.rs — ec_consent_granted() [thin gating layer over prerequisite ConsentContext] + kv.rs — KvIdentityGraph, read/write/delete identity entries + partner.rs — PartnerRecord, PartnerStore, load_partner() + sync_pixel.rs — handle_sync() handler + sync_batch.rs — handle_batch_sync() handler + pull_sync.rs — dispatch_pull_sync() async + identify.rs — handle_identify() handler + admin.rs — handle_register_partner() handler +``` + +Existing files modified: + +| File | Change | +| -------------------------------- | ----------------------------------------------------- | +| `crates/common/src/settings.rs` | Add `EdgeCookie` settings struct | +| `crates/common/src/constants.rs` | Add EC header/cookie name constants | +| `crates/common/src/error.rs` | Add `EdgeCookie` error variant | +| `crates/common/src/auction/` | Inject EC into `user.id`, `user.eids`, `user.consent` | +| `crates/fastly/src/main.rs` | Register new routes, run EC middleware | + +--- + +## 4. EC Identity Generation + +### 4.1 Module: `ec/identity.rs` + +The EC generation mirrors the SyntheticID approach (`synthetic.rs`) but strips volatile inputs. + +```rust +/// Generates a fresh EC value from IP address and publisher passphrase. +/// +/// Output format: `{64-char hex HMAC-SHA256}.{6-char random alphanumeric}` +/// +/// # Errors +/// +/// Returns `EdgeCookie` error if HMAC computation fails. +pub fn generate_ec(ip: IpAddr, passphrase: &str) -> Result>; + +/// Normalizes an IP address for use as an HMAC input. +/// +/// - IPv4: returned as-is (`"203.0.113.1"`) +/// - IPv6: truncated to /64 prefix — first 4 hextets joined by `:`, lower-cased +/// (`"2001:db8:85a3:0"`) +/// - On dual-stack, the caller must supply the IPv6 address; this function does +/// not choose between them. +pub fn normalize_ip(ip: IpAddr) -> String; + +/// Extracts the stable 64-character hex prefix from a full EC value. +/// +/// The prefix is used as the KV store key. The `.suffix` is discarded. +/// +/// Returns `None` if the value is not in `{64-hex}.{6-alnum}` format. +pub fn ec_hash(ec_value: &str) -> Option<&str>; +``` + +**HMAC inputs (fixed — no template):** + +| Input | Value | +| --------- | ------------------------- | +| Message | `normalize_ip(client_ip)` | +| Key | `settings.ec.passphrase` | +| Algorithm | HMAC-SHA256 | + +**Output format:** `{64-char lowercase hex}.{6-char random alphanumeric}` + +The random suffix is generated with `fastly::rand` (same approach as SyntheticID). Once set in a cookie the full value is preserved; only the hash prefix is used as the KV key. + +**IPv6 /64 prefix:** Split on `:`, take first 4 groups, join with `:`. Example: +`2001:db8:85a3:0000:0000:8a2e:0370:7334` → `2001:db8:85a3:0`. + +**IP source:** Use `req.get_client_ip_addr()` — Fastly's trusted API that returns the verified client IP without relying on any request header. This is the same source used by the existing `synthetic.rs` IP handling. Do not fall back to `X-Forwarded-For` or any other header — those are forgeable by clients. Return an error if the API returns `None`; do not create an EC without an IP. + +On dual-stack: prefer IPv6 if the returned address is IPv6; otherwise use IPv4. + +### 4.2 EC Retrieval Priority + +Pre-routing, EC state is read (not generated) from the inbound request: + +1. `X-ts-ec` request header (forwarded by publisher infrastructure) +2. `ts-ec` cookie +3. Neither present → `ec_value = None`, `ec_was_present = false` + +Generation (step 3 above becoming a new EC) happens only inside organic handlers — see §5.4. This logic lives in `EcContext::read_from_request()` (phase 1) and `EcContext::generate_if_needed()` (phase 2). + +### 4.3 `EcContext` + +```rust +/// Per-request Edge Cookie state. Constructed pre-routing in read-only form; +/// organic handlers call `generate_if_needed()` to mint new ECs. +pub struct EcContext { + /// Full EC value (`hash.suffix`), if present on request or generated this request. + pub ec_value: Option, + /// Whether the `ts-ec` **cookie** was present on the inbound request. + /// This is the only field that gates consent-withdrawal cookie deletion — + /// the PRD's delete branch is conditioned on the cookie, not on X-ts-ec header. + pub cookie_was_present: bool, + /// Whether any EC value was available (cookie OR X-ts-ec header). + pub ec_was_present: bool, + /// Set to true by `generate_if_needed()` when a new EC is minted this request. + /// `finalize_response()` uses this to decide whether to write a Set-Cookie header. + pub ec_generated: bool, + /// Full consent context from the prerequisite consent pipeline. + /// Use `ec_consent_granted(&self.consent)` to derive a grant/deny decision. + /// Raw TCF/GPP strings (for KV writes and `user.consent`) are on `consent.raw_tc_string` + /// and `consent.raw_gpp_string`. + pub consent: ConsentContext, + /// Client IP extracted from `req` during `read_from_request()`. + /// Stored here so pull sync can use it after `req` has been consumed by routing. + /// `None` only if Fastly's `get_client_ip_addr()` returns `None`. + pub client_ip: Option, +} + +impl EcContext { + /// Phase 1: reads cookie/header and builds consent context. Does not generate. Does not write KV. + /// Called pre-routing, like `GeoInfo::from_request()` in the current `main.rs`. + /// Calls `build_consent_context()` with the EC hash (if cookie present) as `synthetic_id` + /// so KV-persisted consent can be loaded for users without fresh consent cookies. + pub fn read_from_request( + req: &Request, + settings: &Settings, + geo: Option<&GeoInfo>, + ) -> Result>; + + /// Phase 2: generates a new EC if none is present and consent is granted. + /// Called only inside organic handlers (`handle_publisher_request`, + /// `integration_registry.handle_proxy`). Never called by EC route handlers + /// or the auction handler — those consume EC identity but never bootstrap it. + /// Sets `ec_generated = true` when a new EC is minted, and writes the initial + /// KV entry via `kv.create_or_revive()` (best-effort — logs warn on failure, + /// does not block). Using `create_or_revive` (not `create`) ensures that a user + /// who re-consents within the 24-hour tombstone window recovers immediately. + pub fn generate_if_needed( + &mut self, + req: &Request, + settings: &Settings, + kv: &KvIdentityGraph, + ) -> Result<(), Report>; + + /// Returns the stable 64-char hex prefix, or `None` if no EC. + pub fn ec_hash(&self) -> Option<&str>; +} +``` + +**`finalize_response()` behavior** (updated signature: `finalize_response(settings, geo, ec_context, kv, response)`): + +1. If `!ec_consent_granted(&consent) && cookie_was_present`: call `delete_ec_cookie()` and `kv.write_withdrawal_tombstone(ec_hash)`. This runs on **every route** — consent withdrawal is always real-time enforced. Keyed on `cookie_was_present`, not `ec_was_present`, because only a cookie-held EC can be deleted by the browser. +2. If `ec_generated == true`: call `set_ec_on_response()` — sets `Set-Cookie` and `X-ts-ec`. KV create already happened inside `generate_if_needed()`; `finalize_response()` does NOT write KV beyond the tombstone. +3. Handler-built response headers (`X-ts-ec`, `X-ts-eids` set directly by `/identify`) are not modified. + +**Note on `kv_degraded`:** Not on `EcContext` — `read_from_request()` does not read KV. Handlers track degraded state locally. `/identify` returns `degraded: true` in the JSON body on KV read failure; the auction handler treats a failed read as `eids: []`. + +```` + +--- + +## 5. Cookie and Header Handling + +### 5.1 Cookie attributes + +| Attribute | Value | +|-----------|-------| +| Name | `ts-ec` | +| Domain | `.{publisher.domain}` — derived by prepending `.` to `settings.publisher.domain`, **not** `settings.publisher.cookie_domain` | +| Path | `/` | +| Secure | Yes | +| SameSite | `Lax` | +| Max-Age | `31536000` (1 year) | +| HttpOnly | No | + +### 5.2 Module: `ec/cookie.rs` + +The `cookie_domain` parameter passed to all functions below is computed as +`format!(".{}", settings.publisher.domain)`. Do **not** use +`settings.publisher.cookie_domain` — that field is used by other cookie helpers +and does not carry the EC ownership guarantee. + +```rust +/// Builds the `Set-Cookie` header value for a newly generated EC. +pub fn create_ec_cookie(ec_value: &str, cookie_domain: &str) -> String; + +/// Builds the `Set-Cookie` header value that expires (deletes) the EC cookie. +pub fn delete_ec_cookie(cookie_domain: &str) -> String; +// Sets Max-Age=0 with same Domain/Path/Secure/SameSite attributes. + +/// Sets the EC cookie and `X-ts-ec` response header on a response. +pub fn set_ec_on_response(response: &mut Response, ec_value: &str, cookie_domain: &str); + +/// Removes the EC cookie and clears `X-ts-ec` response header. +pub fn clear_ec_on_response(response: &mut Response, cookie_domain: &str); +```` + +### 5.3 Response header + +`X-ts-ec: {ec_hash.suffix}` is set on every response where an EC is present. + +This header is added to `INTERNAL_HEADERS` in `constants.rs` so it is stripped before proxying to downstream backends, consistent with existing `X-ts-*` handling. + +### 5.4 Per-request EC lifecycle + +**Phase 1 — pre-routing** (always runs, all routes): + +``` +EcContext::read_from_request() + Read ts-ec cookie / X-ts-ec header → ec_value, ec_was_present + build_consent_context(jar, req, config, geo, ec_hash?) → consent: ConsentContext + ec_generated = false +``` + +**Phase 2 — inside organic handlers only** (`handle_publisher_request`, `handle_proxy`): + +``` +ec_context.generate_if_needed(&req, settings, &kv) + └── ec_consent_granted(&consent) && ec_value == None? + → generate_ec(passphrase, ip) + → ec_value = Some(new_ec) + → ec_generated = true + → kv.create_or_revive(ec_hash, &entry) (best-effort, log warn if fails) + // create_or_revive overwrites a tombstone (ok=false) on re-consent + // no-ops if a live entry (ok=true) already exists +``` + +**`finalize_response(settings, geo, ec_context, &kv, response)` — always runs, all routes:** + +``` + ├── !ec_consent_granted(&consent) && cookie_was_present? + │ → delete_ec_cookie() (always — real-time withdrawal enforcement) + │ → kv.write_withdrawal_tombstone(ec_hash) (synchronous — see §6.2) + │ Tombstone fails? log error, do NOT block — no retry possible on browser path + │ cookie deletion is the authoritative enforcement mechanism + │ + ├── ec_was_present == true && ec_generated == false && ec_consent_granted(&consent)? + │ → kv.update_last_seen(ec_hash, now()) (returning user — debounced at 300s) + │ + └── ec_generated == true? + → set_ec_on_response() (Set-Cookie + X-ts-ec on response) +``` + +EC route handlers (`GET /sync`, `GET /identify`, `POST /api/v1/sync`, `POST /admin/*`) never call `generate_if_needed()`. `finalize_response()` will still delete the cookie on those routes if consent is withdrawn — that is intentional. + +**One rule:** `Set-Cookie` is written if and only if `ec_generated == true` (first-time generation). There is no cookie refresh or Max-Age reset on returning users. The PRD defers a blanket refresh-on-every-request strategy to a future iteration. + +--- + +## 6. Consent Enforcement + +### 6.1 Prerequisite contracts + +Consent decoding shipped in `#380` (already merged). This spec treats the following as stable, pre-existing contracts — it does not implement them: + +- **`build_consent_context(input: &ConsentPipelineInput) -> ConsentContext`** — the main entry point. Extracts, decodes, and normalizes signals from cookies and headers. +- **`ConsentContext`** — carries: `raw_tc_string`, `raw_gpp_string`, `raw_us_privacy`, `gdpr_applies: bool`, `tcf: Option`, `gpp: Option`, `us_privacy: Option`, `expired: bool`, `gpc: bool`, `jurisdiction: Jurisdiction`, `source: ConsentSource` +- **`TcfConsent.has_storage_consent()`** — true when TCF Purpose 1 (store/access on device) is granted +- **`Jurisdiction { Gdpr, UsState(String), NonRegulated, Unknown }`** — detected privacy regime (from geo + config) +- **`UsPrivacy.opt_out_sale: PrivacyFlag`** — CCPA opt-out (`Yes`/`No`/`NotApplicable`) + +### 6.1.1 EC consent gating + +The consent pipeline surfaces signals — it does not make grant/deny decisions. EC defines its own gating function in `ec/consent.rs`: + +```rust +/// Returns true when consent is sufficient to create or maintain an EC for this request. +/// Uses Jurisdiction (geo-derived) as the primary enforcement signal. +pub fn ec_consent_granted(consent: &ConsentContext) -> bool { + match &consent.jurisdiction { + // GDPR: require TCF Purpose 1 (storage) and not expired + Jurisdiction::Gdpr => { + let tcf_ok = consent.tcf.as_ref() + .or_else(|| consent.gpp.as_ref()?.eu_tcf.as_ref()) + .map_or(false, |t| t.has_storage_consent()); + tcf_ok && !consent.expired + } + // US state privacy law: require no opt-out + Jurisdiction::UsState(_) => { + !consent.gpc + && consent.us_privacy.as_ref() + .map_or(true, |p| p.opt_out_sale != PrivacyFlag::Yes) + } + // Non-regulated region: always granted + Jurisdiction::NonRegulated => true, + // Unknown geo (Fastly lookup failed): fail-closed — treat as regulated + Jurisdiction::Unknown => false, + } +} +``` + +`EcContext::read_from_request()` calls `build_consent_context()` then stores the result. All downstream logic (EC generation gating, withdrawal detection) calls `ec_consent_granted(&self.consent)`. No consent decoding logic lives in this epic. + +### 6.2 Consent withdrawal — KV delete + +When `ec_consent_granted(&consent)` returns `false` for a user whose **`ts-ec` cookie** is present (`cookie_was_present == true`). A user identified only by the `X-ts-ec` request header is not subject to cookie deletion — there is no cookie to expire. + +1. Issue `Set-Cookie: ts-ec=; Max-Age=0; ...` (synchronous — must not fail silently) +2. Write tombstone: `kv.write_withdrawal_tombstone(ec_hash)` — sets `consent.ok = false`, clears partner IDs, TTL 24h — approximately 25ms + +The tombstone write runs in the request path (not async) to ensure real-time enforcement. Using a tombstone rather than a hard delete preserves the `consent_withdrawn` signal for batch sync clients for 24 hours — otherwise batch sync cannot distinguish consent withdrawal from an EC that never existed. + +If the tombstone write fails: + +- Log at `error` level with EC hash +- Do not block the response — cookie deletion is the primary enforcement mechanism +- **No retry is possible on the browser path.** Once the cookie is deleted, subsequent browser requests carry no EC value (`ec_hash()` returns `None`), so there is no hash to tombstone. A failed tombstone means batch sync clients may see `ec_hash_not_found` (after TTL expiry) rather than `consent_withdrawn` — this is accepted degradation. The cookie deletion remains the authoritative enforcement mechanism. + +--- + +## 7. KV Store Identity Graph + +### 7.1 Module: `ec/kv.rs` + +Two KV stores are used. Their names are configured in `trusted-server.toml`: + +| Store | TOML key | Purpose | +| ---------------- | ------------------ | ---------------------------------- | +| Identity graph | `ec.ec_store` | EC hash → identity JSON | +| Partner registry | `ec.partner_store` | Partner ID → config + API key hash | + +### 7.2 Identity graph schema + +**KV key:** 64-character hex hash (the stable prefix from `ec_value`, without `.suffix`). + +**KV value (JSON, max ~5KB):** + +```json +{ + "v": 1, + "created": 1741824000, + "last_seen": 1741910400, + "consent": { + "tcf": "CP...", + "gpp": "DBA...", + "ok": true, + "updated": 1741910400 + }, + "geo": { + "country": "US", + "region": "CA" + }, + "ids": { + "ssp_x": { "uid": "abc123", "synced": 1741824000 }, + "liveramp": { "uid": "LR_xyz", "synced": 1741890000 } + } +} +``` + +**KV metadata (max 2048 bytes, readable without streaming body):** + +```json +{ "ok": true, "country": "US", "v": 1 } +``` + +The `ok` field in metadata is a **historical consent record for S2S consumers only** — it is set to `false` by `write_withdrawal_tombstone()` so that batch sync clients (`POST /api/v1/sync`) can return `consent_withdrawn` rather than `ec_hash_not_found` during the 24-hour tombstone TTL. + +**`consent.ok` is NOT used to make the withdrawal decision on the main request path.** Consent withdrawal is determined entirely from `ec_consent_granted(&ec_context.consent)` on the current request. When withdrawal is detected, the cookie is deleted and `write_withdrawal_tombstone()` is called in-path (setting `ok = false`, 24h TTL — see §6.2). Engineers must not add a KV read to the consent withdrawal hot path based on this field. + +**Rust types:** + +```rust +pub struct KvEntry { + pub v: u8, + pub created: u64, + pub last_seen: u64, + pub consent: KvConsent, + pub geo: KvGeo, + pub ids: HashMap, +} + +pub struct KvConsent { + pub tcf: Option, + pub gpp: Option, + pub ok: bool, + pub updated: u64, +} + +pub struct KvGeo { + pub country: String, + pub region: Option, +} + +pub struct KvPartnerId { + pub uid: String, + pub synced: u64, +} + +pub struct KvMetadata { + pub ok: bool, + pub country: String, + pub v: u8, +} +``` + +### 7.3 TTL + +All KV writes use `time_to_live_sec = 31536000` (1 year), matching the cookie `Max-Age`. + +### 7.4 Conflict resolution — atomic read-modify-write + +Concurrent writes from different partners must not overwrite each other. Each partner's ID is namespaced under `ids[partner_id]` — a write for `ssp_x` must not clobber an existing `liveramp` entry. + +Implementation uses Fastly KV Store's **generation markers** (optimistic concurrency): + +```rust +pub struct KvIdentityGraph { + store_name: String, +} + +impl KvIdentityGraph { + pub fn new(store_name: impl Into) -> Self; + + /// Reads the full entry, returning the generation marker for CAS writes. + pub fn get( + &self, + ec_hash: &str, + ) -> Result, Report>; + + /// Reads only the metadata fields (consent flag, country). + pub fn get_metadata( + &self, + ec_hash: &str, + ) -> Result, Report>; + + /// Creates a new entry. Returns `Ok(())` if successful, `Err` if the key + /// already exists (concurrent create) or on KV error. + pub fn create( + &self, + ec_hash: &str, + entry: &KvEntry, + ) -> Result<(), Report>; + + /// Creates a new entry, OR overwrites an existing tombstone (`consent.ok = false`) + /// with a fresh entry when the user re-consents within the tombstone TTL. + /// + /// Behavior: + /// - No existing key → behaves identically to `create()`. + /// - Existing key with `consent.ok = false` (tombstone) → overwrites with + /// the new entry via CAS. Retries up to `MAX_CAS_RETRIES` on conflict. + /// - Existing key with `consent.ok = true` (live entry) → no-op, returns `Ok(())`. + /// + /// Called by `generate_if_needed()` instead of `create()`. This ensures that + /// re-consent recovery is immediate — a user who withdraws and then re-consents + /// within the 24-hour tombstone window gets a fresh identity entry without delay. + pub fn create_or_revive( + &self, + ec_hash: &str, + entry: &KvEntry, + ) -> Result<(), Report>; + + /// Atomically merges `ids[partner_id]` into the existing entry using a + /// generation marker. Retries up to `MAX_CAS_RETRIES` (3) times on + /// generation conflict before returning `Err`. + pub fn upsert_partner_id( + &self, + ec_hash: &str, + partner_id: &str, + uid: &str, + synced: u64, + ) -> Result<(), Report>; + + /// Updates `last_seen` timestamp, but only if the stored value is more than + /// 300 seconds older than `timestamp`. This debounce prevents KV write + /// thrashing under bursty traffic — Fastly KV enforces a 1 write/sec limit + /// per key. Callers should log `warn` on failure and continue. + pub fn update_last_seen( + &self, + ec_hash: &str, + timestamp: u64, + ) -> Result<(), Report>; + + /// Writes a withdrawal tombstone for consent enforcement. + /// + /// Instead of hard-deleting the KV entry, this overwrites it with + /// `consent.ok = false`, clears all partner IDs, and sets a 24-hour TTL. + /// The tombstone allows batch sync clients (`POST /api/v1/sync`) to return + /// `consent_withdrawn` rather than `ec_hash_not_found` for the tombstone TTL. + /// + /// After the 24-hour TTL expires, the entry is gone. Any subsequent `get()` + /// returns `None` (`ec_hash_not_found`) — the distinction is time-bounded. + /// + /// Caller must handle `Err` by logging at `error` level; the cookie deletion + /// in `finalize_response()` is the primary enforcement mechanism. + pub fn write_withdrawal_tombstone( + &self, + ec_hash: &str, + ) -> Result<(), Report>; + + /// Hard-deletes the entry. Used only for data deletion requests (IAB deletion + /// framework — deferred). For consent withdrawal, use `write_withdrawal_tombstone()`. + pub fn delete(&self, ec_hash: &str) -> Result<(), Report>; +} +``` + +`MAX_CAS_RETRIES = 3`. If all retries fail on a generation conflict, return `Err` — callers handle per-endpoint policy (Section 8.4 for pixel sync, Section 10 for batch). + +### 7.5 KV degraded behavior + +| Operation | KV unavailable | Action | +| ---------------------------------- | -------------- | --------------------------------------------------------------------- | +| EC cookie creation | KV error | Set cookie. Skip KV create. Log `warn`. | +| `/sync` KV write | KV error | Redirect with `ts_synced=0&ts_reason=write_failed`. | +| `/identify` KV read | KV error | Return `200` with `ec` set, `degraded: true`, empty `uids`/`eids`. | +| `POST /api/v1/sync` | KV error | Return `207` with all mappings rejected, `reason: "kv_unavailable"`. | +| Pull sync KV write | KV error | Discard uid. Log `warn`. Retry on next qualifying request. | +| Consent withdrawal tombstone write | KV error | Delete cookie (primary enforcement). Log `error`. Next request: no cookie → no EC regenerated. | + +--- + +## 8. Pixel Sync Endpoint (`GET /sync`) + +### 8.1 Module: `ec/sync_pixel.rs` + +```rust +pub async fn handle_sync( + settings: &Settings, + kv: &KvIdentityGraph, + partner_store: &PartnerStore, + req: &Request, + ec_context: &EcContext, +) -> Result>; +``` + +### 8.2 Query parameters + +| Parameter | Required | Description | +| --------- | -------- | ----------------------------------------------------------------- | +| `partner` | Yes | Partner ID — must exist in `partner_store` | +| `uid` | Yes | Partner's user ID for this user | +| `return` | Yes | Redirect-back URL (must match partner's `allowed_return_domains`) | +| `consent` | No | Fallback TCF/GPP string if no consent cookie on request | + +### 8.3 Flow + +``` +1. Parse query params. Missing required params → 400. + +2. Read ts-ec cookie. + Absent → redirect to {return}?ts_synced=0&ts_reason=no_ec + +3. Look up partner record in partner_store. + Not found → 400. + +4. Validate return URL host against partner.allowed_return_domains. + - Exact hostname match only — no suffix or wildcard. + - Mismatch → 400. + +5. Evaluate consent. Use `ec_context.consent` (built pre-routing via + `build_consent_context()`). The optional `consent` query param is a **fallback + only** — used solely when `ec_context.consent.is_empty()` (no cookies or + headers carried consent signals). If any signal exists, the query param is + ignored entirely. When the fallback applies, re-call `build_consent_context()` + with a synthetic request or cookie jar that includes the consent param value. + `!ec_consent_granted(...)` → redirect to {return}?ts_synced=0&ts_reason=no_consent + +6. Check anti-stuffing rate limit (sync_rate_limit per EC hash per partner per hour). + Exceeded → `429 Too Many Requests` (no redirect — the `return` URL is never called). + +7. kv.upsert_partner_id(ec_hash, partner_id, uid, now()) + KV write failure → redirect to {return}?ts_synced=0&ts_reason=write_failed + +8. Success → redirect to {return}?ts_synced=1 +``` + +`ts_synced` values: + +| Value | Meaning | +| ------------------------------------ | ----------------------------- | +| `ts_synced=1` | KV write succeeded | +| `ts_synced=0&ts_reason=no_ec` | No EC cookie present | +| `ts_synced=0&ts_reason=no_consent` | Consent absent or denied | +| `ts_synced=0&ts_reason=write_failed` | KV write failed after retries | + +Rate limit exceeded returns `429 Too Many Requests` directly — the partner's `return` URL is not called in this case. + +### 8.4 Return URL construction + +Append `ts_synced` (and optional `ts_reason`) to the `return` URL: + +- If the URL already has a query string, append `&ts_synced=...` +- If not, append `?ts_synced=...` + +Do not modify any other query parameters on the `return` URL. + +### 8.5 Security + +- `return` URL validated by exact hostname match against `partner.allowed_return_domains`. No subdomain wildcard matching. +- No HMAC signature required on inbound sync request. +- Rate limit: `partner.sync_rate_limit` writes per EC hash per partner per hour. Default: 100. Configurable per partner in `partner_store`. + +--- + +## 9. S2S Batch Sync API (`POST /api/v1/sync`) + +### 9.1 Module: `ec/sync_batch.rs` + +```rust +pub async fn handle_batch_sync( + settings: &Settings, + kv: &KvIdentityGraph, + partner_store: &PartnerStore, + req: Request, +) -> Result>; +``` + +### 9.2 Authentication + +`Authorization: Bearer ` header required. The `api_key` is looked up in `partner_store` by a constant-time comparison of its SHA-256 hash against the stored `api_key_hash`. Key rotation does not require binary redeployment — partners update `partner_store` directly via `/admin/partners/register`. + +Returns `401 Unauthorized` with no body processing if auth fails. + +### 9.2.1 API-key rate limiting + +After successful auth, check the API-key level rate limit: `partner.batch_rate_limit` requests per partner per minute (default 60). Uses the same Fastly rate-limiting API as pixel sync (§14.3), with key `batch:{partner_id}`. + +Exceeded → `429 Too Many Requests` with body `{ "error": "rate_limit_exceeded" }`. No mappings are processed. + +### 9.3 Request format + +``` +POST /api/v1/sync +Content-Type: application/json +Authorization: Bearer + +{ + "mappings": [ + { + "ec_hash": "<64-character hex hash>", + "partner_uid": "abc123", + "timestamp": 1741824000 + } + ] +} +``` + +Maximum batch size: 1000 mappings. Requests exceeding this receive `400 Bad Request`. + +### 9.4 Processing + +The authenticated partner's ID (from the `PartnerRecord` resolved via API key in §9.2) determines the `ids[partner_id]` namespace for all writes in this batch. A partner can only write to their own namespace. + +For each mapping: + +1. Validate `ec_hash` format (must be exactly 64 lowercase hex characters). Invalid format → reject with `reason: "invalid_ec_hash"`. +2. Read KV metadata for `ec_hash`. If not found → reject with `reason: "ec_hash_not_found"`. If `consent.ok = false` → reject with `reason: "consent_withdrawn"`. +3. `kv.upsert_partner_id(ec_hash, partner_id, partner_uid, timestamp)`. The upsert internally skips the write if the existing `ids[partner_id].synced ≥ timestamp` (idempotent — counted as accepted, no error). On KV failure → reject all remaining mappings with `reason: "kv_unavailable"`, return `207`. + +### 9.5 Response format + +```json +{ + "accepted": 998, + "rejected": 2, + "errors": [ + { "index": 45, "reason": "ec_hash_not_found" }, + { "index": 72, "reason": "consent_withdrawn" } + ] +} +``` + +HTTP status rules: + +| Condition | Status | +| -------------------------------------- | ------------------------------------------------------ | +| All mappings accepted | `200 OK` | +| Some accepted, some rejected | `207 Multi-Status` | +| All rejected (auth valid, batch valid) | `207 Multi-Status` with `accepted: 0` | +| Auth invalid | `401 Unauthorized` | +| Malformed JSON or > 1000 mappings | `400 Bad Request` | +| KV entirely unavailable | `207 Multi-Status`, all rejected with `kv_unavailable` | + +```rust +pub struct BatchSyncResponse { + pub accepted: usize, + pub rejected: usize, + pub errors: Vec, +} + +pub struct BatchSyncError { + pub index: usize, + pub reason: BatchSyncRejection, +} + +#[derive(Debug, derive_more::Display)] +pub enum BatchSyncRejection { + #[display("invalid_ec_hash")] + InvalidEcHash, + #[display("ec_hash_not_found")] + EcHashNotFound, + #[display("consent_withdrawn")] + ConsentWithdrawn, + #[display("kv_unavailable")] + KvUnavailable, +} +``` + +--- + +## 10. S2S Pull Sync (TS-Initiated) + +### 10.1 Module: `ec/pull_sync.rs` + +Pull sync inverts the batch model: TS calls the partner's resolution endpoint server-to-server and writes the returned UID into the KV graph. No browser redirect is involved. + +```rust +pub struct PullSyncDispatcher { + concurrency_limit: usize, +} + +impl PullSyncDispatcher { + pub fn new(concurrency_limit: usize) -> Self; + + /// Dispatches pull sync calls for all qualifying partners. + /// Called after `send_to_client()` — fires outbound requests using `send_async()`. + /// Takes `client_ip` directly (extracted before `req` is consumed by routing). + pub fn dispatch_background( + &self, + ec_context: &EcContext, + client_ip: IpAddr, + partners: &[PartnerRecord], + kv: &KvIdentityGraph, + ); +} + +/// Calls a single partner's resolution endpoint and writes the result to KV. +async fn pull_one_partner( + ec_hash: &str, + ip: IpAddr, + partner: &PartnerRecord, + kv: &KvIdentityGraph, +); +``` + +### 10.2 Trigger conditions + +A pull sync is dispatched for a partner when all of the following are true on a request: + +1. A valid `ts-ec` cookie is present +2. `ec_consent_granted(&ec_context.consent) == true` +3. `partner.pull_sync_enabled == true` +4. Either: no entry exists for this partner in the KV graph, or the existing `synced` timestamp is older than `partner.pull_sync_ttl_sec` (default 86400 seconds) +5. Rate limit not exceeded: `partner.pull_sync_rate_limit` calls per EC hash per partner per hour (default 10) + +### 10.3 Execution model + +Pull calls are dispatched using Fastly's background task / `send_async` model after the response is flushed. They do not add latency to the user-facing request. + +Maximum concurrent pull calls per request: `settings.ec.pull_sync_concurrency` (default 3). + +**Architectural divergence from PRD:** The PRD describes excess partner calls being queued and dispatched on subsequent requests for the same user. A persistent queue is not implementable in the stateless Fastly WASM edge environment — there is no cross-request mutable state. This spec adapts the intent using a stateless rotating offset: sort qualifying partners by ID, then use `(unix_timestamp_secs / 3600) % partner_count` as the starting index (wrapping). This ensures different partners are prioritized across different requests without persisted state. Partners not called on a given request remain eligible on the next qualifying request per their `pull_sync_ttl_sec` condition. The practical outcome (all partners eventually called) matches the PRD intent; the mechanism differs due to the platform constraint. + +### 10.4 Outbound request + +``` +GET {partner.pull_sync_url}?ec_hash={64-char-hex}&ip={ip_address} +Authorization: Bearer {partner.ts_pull_token} +``` + +Before dispatching, `pull_sync.rs` validates that `pull_sync_url`'s hostname is present in `partner.pull_sync_allowed_domains`. If not, the call is skipped and an `error` is logged — this is a configuration error that should not occur at runtime if admin validation is working correctly (§13.2 step 3). + +Only the EC hash and IP are sent. No consent strings, geo data, or other partner IDs are included. + +**Expected partner responses:** + +```json +{ "uid": "abc123" } // resolved +{ "uid": null } // not recognized +``` + +Or `404 Not Found`. Both null and 404 are no-ops — no KV write, no error logged above `debug`. + +Any other non-200 response is treated as a transient failure. No retry. The next qualifying request triggers a new attempt. + +### 10.5 KV write on success + +On a non-null `uid`: call `kv.upsert_partner_id(ec_hash, partner_id, uid, now())`. On KV failure: log `warn` and discard the result. Retry occurs on the next qualifying request. + +The write updates `ids[partner_id].synced` to the current timestamp, resetting the `pull_sync_ttl_sec` window. + +--- + +## 11. Identity Resolution Endpoint (`GET /identify`) + +### 11.1 Module: `ec/identify.rs` + +```rust +pub async fn handle_identify( + settings: &Settings, + kv: &KvIdentityGraph, + partner_store: &PartnerStore, + req: &Request, + ec_context: &EcContext, +) -> Result>; +``` + +### 11.2 Call patterns + +**Browser-direct:** The browser sends the request to `ec.publisher.com/identify`. Cookies and consent cookies are sent automatically (same-site). No special header forwarding required. + +**Server-side proxy (for use case 2):** The publisher's origin server must forward: + +| Header | Required | +| ------------------------------------------------------- | -------------------------------------- | +| `Cookie: ts-ec=` or `X-ts-ec: ` | Yes | +| `Cookie: euconsent-v2=` or `Cookie: gpp=` | Yes for EU/UK/US users | +| `X-consent-advertising: ` | Optional — takes precedence if present | + +### 11.3 EC and consent handling + +`/identify` follows `EcContext` retrieval priority (Section 4.2). It does **not** generate a new EC. It does **not** set or modify cookies. + +Consent is evaluated using the same logic as Section 6. + +### 11.4 Response + +**`200 OK` — EC present, consent granted:** + +```json +{ + "ec": "a1b2c3...AbC123", + "consent": "ok", + "degraded": false, + "uids": { + "uid2": "A4A...", + "liveramp": "LR_xyz" + }, + "eids": [ + { "source": "uidapi.com", "uids": [{ "id": "A4A...", "atype": 3 }] }, + { "source": "liveramp.com", "uids": [{ "id": "LR_xyz", "atype": 3 }] } + ] +} +``` + +`uids` contains one key per partner with `bidstream_enabled: true` and a resolved UID in the KV graph. Partners with no resolved UID for this user are omitted. + +**`200 OK` — KV unavailable (degraded):** + +```json +{ + "ec": "a1b2c3...AbC123", + "consent": "ok", + "degraded": true, + "uids": {}, + "eids": [] +} +``` + +**`403 Forbidden` — consent denied:** + +```json +{ "consent": "denied" } +``` + +**`204 No Content` — no EC present.** No body. + +### 11.5 Response headers (supplementary) + +Set on `200` responses only: + +| Header | Value | +| ------------------- | ------------------------------------------------------------ | +| `X-ts-ec` | `{ec_hash.suffix}` | +| `X-ts-eids` | Standard base64 (RFC 4648, with `=` padding) of the JSON array of OpenRTB 2.6 `user.eids` objects. Capped at **4 KB** after encoding. If the encoded value exceeds 4 KB, the array is truncated (fewest partners first — highest `synced` timestamp retained) until it fits, and a `x-ts-eids-truncated: true` header is added. | +| `X-ts-` | Resolved UID per partner (e.g., `X-ts-uid2`). One header per partner with a resolved UID. **Capped at 20 partners** — partners sorted by most-recently synced; excess partners are omitted silently. | +| `X-ts-ec-consent` | `ok` or `denied` | + +These are supplementary — callers should read the JSON body as the primary contract. The 4 KB cap on `X-ts-eids` and the 20-partner cap on `X-ts-` headers reflect typical proxy and browser total-header-budget constraints. Both caps apply independently. + +### 11.6 Performance target + +`/identify` must respond within 30ms (excluding network latency) when EC is present and KV read succeeds. This requires the KV read to be on the fast path with no retries. + +CORS headers must be set to allow browser-direct calls from the publisher's page. The `Access-Control-Allow-Origin` header is dynamically reflected from the `Origin` request header if the origin is an exact match or a subdomain of `settings.publisher.domain`: + +``` +// e.g. publisher.domain = "example.com" +// Allowed: https://example.com, https://www.example.com, https://news.example.com +// Rejected: https://evil.com, https://notexample.com + +Access-Control-Allow-Origin: +Access-Control-Allow-Credentials: true +Access-Control-Allow-Methods: GET, OPTIONS +Access-Control-Allow-Headers: Cookie, X-ts-ec, X-consent-advertising +Access-Control-Expose-Headers: X-ts-ec, X-ts-eids, X-ts-ec-consent, X-ts-eids-truncated, +Vary: Origin +``` + +**`Access-Control-Expose-Headers` note:** The dynamic `X-ts-` headers must be enumerated per-response, not as a static constant. The handler builds the expose list by iterating the partner IDs that have resolved UIDs in the response. `x-ts-eids-truncated` is always included in the expose list (browser JS should be able to detect truncation even when it occurs). + +**Origin validation logic:** CORS headers are only relevant when the `Origin` request header is present (browser requests always send it; server-side proxy calls typically do not). + +- **No `Origin` header present:** Process normally. No CORS headers added. No `403`. This is the server-side proxy path from §11.2 — origin-server calls forwarding `Cookie` and consent headers. +- **`Origin` header present, hostname matches `publisher.domain` or ends with `.{publisher.domain}` and scheme is `https`:** Reflect origin in `Access-Control-Allow-Origin`. Add `Vary: Origin`. +- **`Origin` header present but does not match:** Return `403`. No body. + +Browser `fetch()` with `credentials: "include"` sends an `OPTIONS` preflight. The router handles `OPTIONS /identify` identically — returns `200 OK` with the CORS headers above and no body. + +--- + +## 12. Bidstream Decoration (`/auction` Mode B) + +### 12.1 Changes to existing auction path + +The auction handler (`crates/common/src/auction/`) is modified to inject EC identity into outbound OpenRTB requests. This is **not** a builder tweak — it requires explicit schema additions across multiple files. + +**EC + SyntheticID coexistence (transitional — cutover is out of scope for this spec):** + +EC is the authoritative identity signal where present; SyntheticID continues to run alongside it during the transition. Removal of SyntheticID generation, its cookies, and its response headers is a follow-on spec. + +| Concern | This-spec behavior | +|---------|-------------------| +| `UserInfo.id` | Add `ec_id: Option` to `UserInfo`. When EC is present, `ec_id = Some(ec_value)`. `id` continues to hold synthetic ID unchanged. | +| Outbound OpenRTB `user.id` | Set to `ec_value` when EC present, `synthetic_id` otherwise. | +| `X-Synthetic-*` response headers | **Kept unchanged** — transitional compatibility. Removal is a follow-on. | +| `X-ts-ec` response header | Added alongside `X-Synthetic-*` when EC is present. | +| Publisher and integration proxy paths | Both `get_or_generate_synthetic_id()` and `ec_context.generate_if_needed()` run. | +| `convert_tsjs_to_auction_request()` | Add `ec_context: Option<&EcContext>` parameter alongside existing synthetic logic. | + +**Schema changes required before handler changes:** + +| File | Change | +| -------------- | ------------------------------------------------------------------------------------------------------------------------------- | +| `types.rs` | Add `ec_id: Option` to `UserInfo`. Add `Eid` and `EidUid` OpenRTB 2.6 types. No removals. | +| `openrtb.rs` | Add `eids: Vec` and `consent: Option` to `User` struct. Keep `ext.synthetic_fresh`. | +| `prebid.rs` | Populate `user.id` from EC when present (fall back to synthetic). Add `user.eids`, `user.consent`. Keep existing synthetic fields. | +| `formats.rs` | Accept `ec_context: Option<&EcContext>`. Keep `get_or_generate_synthetic_id()` calls. | +| `endpoints.rs` | Pass `ec_context` to `convert_tsjs_to_auction_request()`. Add `X-ts-ec` header. Keep `X-Synthetic-*`. | + +These changes affect the OpenRTB wire format — confirm with engineering that no existing SSP integrations break before merging. + +### 12.2 `user` object injection + +When an `EcContext` is available on the request, the auction handler performs an explicit KV read before building the OpenRTB request: + +```rust +// In handle_auction(): +let kv_entry = kv.get(ec_context.ec_hash()?).ok().flatten(); + +user.id = ec_context.ec_value.clone(); // full hash.suffix +user.consent = consent_string; // TCF string from ec_context.consent, else None +user.eids = match kv_entry { + Some((entry, _gen)) => build_eids_from_kv(&entry, partner_store), + None => vec![], // KV read failed or no entry — degrade gracefully, omit eids +}; +``` + +`build_eids_from_kv` iterates `kv_entry.ids` and includes only partners with `bidstream_enabled: true` and a non-empty `uid`. Partners without a resolved UID are omitted. + +### 12.3 OpenRTB `user.eids` structure + +```json +{ + "user": { + "id": "a1b2c3...AbC123", + "consent": "CP...", + "eids": [ + { + "source": "liveramp.com", + "uids": [{ "id": "LR_xyz", "atype": 3 }] + }, + { + "source": "uidapi.com", + "uids": [{ "id": "A4A...", "atype": 3 }] + } + ] + } +} +``` + +`atype: 3` for all EC-derived IDs (partner-defined), per OpenRTB 2.6 spec. + +### 12.4 SSP-specific adapter `ext.eids` + +When calling a specific PBS adapter, include only that SSP's resolved ID in the adapter-level `ext.eids`. The full `user.eids` array contains all configured identity providers. + +### 12.5 `/auction` response headers (in-scope) + +The current `/auction` path returns a JSON response inline to the JS caller (`endpoints.rs:71`). There is no server-to-server delivery step to a publisher ad server. EC headers are added to this existing response: + +| Header | Value | +| ------------------- | ---------------------------------------------------- | +| `X-ts-ec` | `{ec_hash.suffix}` — when EC is present | +| `X-ts-eids` | Standard base64 (RFC 4648) of OpenRTB 2.6 `user.eids` JSON array. Capped at 4 KB — same truncation rules as §11.5. | +| `X-ts-eids-truncated` | `true` — present only when `X-ts-eids` was truncated | +| `X-ts-ec-consent` | `ok` or `denied` | +| `X-Synthetic-ID` | **Transitional** — kept while SyntheticID cutover is pending | +| `X-Synthetic-Fresh` | **Transitional** — kept while SyntheticID cutover is pending | +| `X-Synthetic-Trusted-Server` | **Transitional** — kept while SyntheticID cutover is pending | + +**Deferred:** A future server-to-server winner-notification delivery step to a publisher ad server is not in scope for this iteration. See §1 deferred items. + +--- + +## 13. Partner Registry and Admin Endpoint + +### 13.1 Module: `ec/partner.rs` + +```rust +pub struct PartnerRecord { + /// Partner identifier. Must match `^[a-z0-9_-]{1,32}$` (lowercase, no spaces). + /// Used to build `X-ts-` response headers — header-safety is required. + /// Reserved names that would collide with existing managed headers are rejected + /// at registration: `ec`, `eids`, `ec-consent`, `eids-truncated`, `synthetic`, `ts`, `version`, `env`. + pub id: String, + pub name: String, + pub allowed_return_domains: Vec, + pub api_key_hash: String, // SHA-256 hex of the partner's API key + pub bidstream_enabled: bool, + pub source_domain: String, // OpenRTB source (e.g., "liveramp.com") + pub openrtb_atype: u8, // typically 3 + pub sync_rate_limit: u32, // per EC hash per partner per hour + pub batch_rate_limit: u32, // API-key level: requests per partner per minute (default 60) + pub pull_sync_enabled: bool, + pub pull_sync_url: Option, // required when pull_sync_enabled; validated at registration + pub pull_sync_allowed_domains: Vec, // allowlist of domains TS may call for this partner + pub pull_sync_ttl_sec: u64, // default 86400 + pub pull_sync_rate_limit: u32, // default 10 + pub ts_pull_token: Option, // required when pull_sync_enabled; outbound bearer token +} + +pub struct PartnerStore { + store_name: String, +} + +impl PartnerStore { + pub fn new(store_name: impl Into) -> Self; + + /// Looks up a partner by ID. Returns `None` if not found. + pub fn get(&self, partner_id: &str) -> Result, Report>; + + /// Verifies an API key against the stored hash for a given partner. + /// Uses constant-time comparison. + pub fn verify_api_key(&self, partner_id: &str, api_key: &str) -> bool; + + /// Writes or updates a partner record. + pub fn upsert(&self, record: &PartnerRecord) -> Result<(), Report>; + + /// Looks up the partner owning a given API key hash (for batch sync auth). + /// Iterates all partner records — called once per batch request, not per mapping. + pub fn find_by_api_key_hash(&self, hash: &str) -> Result, Report>; + + /// Returns all partner records with `pull_sync_enabled == true`. + /// Used by the pull sync dispatcher after each organic request. + pub fn pull_enabled_partners(&self) -> Result, Report>; +} +``` + +Partner records are stored as JSON values in `partner_store` KV, keyed by `partner_id`. + +### 13.2 Admin endpoint (`POST /admin/partners/register`) + +**Module:** `ec/admin.rs` + +> **Codebase invariant:** `Settings::ADMIN_ENDPOINTS` in `settings.rs` hard-codes the list of admin routes and its tests verify coverage. Adding `/admin/partners/register` requires updating that constant and the associated auth-coverage tests. Failure to do so will break existing tests. See `settings.rs:391,398` and the test at `settings.rs:1363,1395`. + +```rust +pub async fn handle_register_partner( + settings: &Settings, + partner_store: &PartnerStore, + req: Request, +) -> Result>; +``` + +Authentication: `Authorization: Bearer ` header, validated inside the handler against `settings.ec.admin_token_hash` (SHA-256 constant-time comparison). This is a publisher-level admin credential — separate from partner API keys, and enforced in-handler (not via `[[handlers]]` Basic Auth). Returns `401 Unauthorized` with no body if the token is missing or invalid. + +**Request:** + +``` +POST /admin/partners/register +Authorization: Bearer +Content-Type: application/json + +{ + "id": "ssp_x", + "name": "SSP Example", + "allowed_return_domains": ["sync.example-ssp.com"], + "api_key": "raw_key_to_hash_and_store", + "bidstream_enabled": true, + "source_domain": "example-ssp.com", + "openrtb_atype": 3, + "sync_rate_limit": 100, + "batch_rate_limit": 60, + "pull_sync_enabled": false, + "pull_sync_url": null, + "pull_sync_allowed_domains": [], + "pull_sync_ttl_sec": 86400, + "pull_sync_rate_limit": 10, + "ts_pull_token": null +} +``` + +**Processing:** + +1. Validate `Authorization: Bearer `: SHA-256 hash the token and compare against `settings.ec.admin_token_hash` using constant-time comparison. `401` if missing or invalid. +2. Validate required fields (`id`, `name`, `allowed_return_domains`, `api_key`, `source_domain`). `400` on failure. + Validate `id` format: must match `^[a-z0-9_-]{1,32}$`. Must not be a reserved name + (`ec`, `eids`, `ec-consent`, `eids-truncated`, `synthetic`, `ts`, `version`, `env`). `400` with descriptive message on failure. +3. If `pull_sync_enabled == true`, validate that both `pull_sync_url` and `ts_pull_token` are present and non-empty. `400` with `"pull_sync_url and ts_pull_token are required when pull_sync_enabled is true"` if either is missing. + If `pull_sync_url` is set, validate that its hostname is present in `pull_sync_allowed_domains`. `400` on failure with `"pull_sync_url domain must be in pull_sync_allowed_domains"`. This prevents TS from being directed to call arbitrary URLs — the allowlist must be declared in the same registration payload. +4. Hash `api_key` with SHA-256 before writing — never store plaintext. +5. `partner_store.upsert(record)`. `503` on KV failure. +6. Return `201 Created` with the stored record (without `api_key_hash` raw value). + +**Response:** + +```json +{ + "id": "ssp_x", + "name": "SSP Example", + "registered_at": 1741824000 +} +``` + +--- + +## 14. Configuration + +### 14.1 New `EdgeCookie` settings struct + +Added to `crates/common/src/settings.rs`: + +```rust +#[derive(Debug, Default, Clone, Deserialize, Serialize, Validate)] +pub struct EdgeCookie { + /// Publisher passphrase used as HMAC key for EC generation. + /// Must be identical across all of the publisher's owned domains. + /// Publishers sharing this value with partners form an identity-federated consortium. + #[validate(custom(function = EdgeCookie::validate_passphrase))] + pub passphrase: String, + + /// Fastly KV store name for the EC identity graph. + pub ec_store: String, + + /// Fastly KV store name for the partner registry. + pub partner_store: String, + + /// SHA-256 hex of the publisher admin token for `POST /admin/partners/register`. + /// The plaintext token is provided in the `Authorization: Bearer` header; + /// it is never stored in plaintext. + pub admin_token_hash: String, + + /// Maximum concurrent pull sync calls dispatched per request. + #[serde(default = "EdgeCookie::default_pull_sync_concurrency")] + pub pull_sync_concurrency: usize, +} + +impl EdgeCookie { + fn validate_passphrase(passphrase: &str) -> Result<(), ValidationError>; + // Rejects "passphrase" or empty string as placeholder. + + fn default_pull_sync_concurrency() -> usize { 3 } +} +``` + +Added to `Settings`: + +```rust +pub struct Settings { + // ... existing fields ... + #[serde(default)] + pub ec: EdgeCookie, +} +``` + +### 14.2 TOML configuration example + +```toml +[ec] +passphrase = "publisher-chosen-secret" +ec_store = "ec_identity_store" +partner_store = "ec_partner_store" +admin_token_hash = "sha256-hex-of-publisher-admin-token" +pull_sync_concurrency = 3 +``` + +### 14.3 Rate Limit Storage + +Pixel sync and pull sync rate limits (per EC hash per partner per hour) cannot use in-memory state in a WASM/Fastly Compute environment — there is no shared memory across requests. + +**Implementation:** Use Fastly's Edge Rate Limiting API (`fastly::erl::RateCounter`), which provides distributed per-key counting without KV latency and is designed for high-frequency counting without per-key write limits. + +| Counter | Key format | Window | +| ---------- | ----------------------------- | -------- | +| Pixel sync | `{partner_id}:{ec_hash}` | 1 hour | +| Pull sync | `pull:{partner_id}:{ec_hash}` | 1 hour | +| Batch sync | `batch:{partner_id}` | 1 minute | + +If the rate-limiting API is unavailable in the WASM target, fall back to a KV-based counter (`ec_store` key `rl:{partner_id}:{ec_hash}`, hourly TTL). Engineering to confirm API availability during Step 7 (pixel sync implementation). + +### 14.4 Deprecation note + +`settings.synthetic.counter_store` and `settings.synthetic.opid_store` are currently configured but unused. They are not removed in this iteration — a follow-on cleanup ticket will address them. + +--- + +## 15. Constants and Header Names + +New constants in `crates/common/src/constants.rs`: + +```rust +// EC cookie name +pub const COOKIE_EC: &str = "ts-ec"; + +// EC response header +pub const HEADER_X_TS_EC: &str = "x-ts-ec"; + +// Supplementary identity headers +pub const HEADER_X_TS_EIDS: &str = "x-ts-eids"; +pub const HEADER_X_TS_EC_CONSENT: &str = "x-ts-ec-consent"; +pub const HEADER_X_TS_EIDS_TRUNCATED: &str = "x-ts-eids-truncated"; + +// Consent cookies +pub const COOKIE_TCF: &str = "euconsent-v2"; +pub const COOKIE_GPP: &str = "gpp"; + +// No EC-specific geo/IP header constants — use req.get_client_ip_addr() and GeoInfo::from_request(req). +``` + +The following EC headers must be added to `INTERNAL_HEADERS` in `constants.rs` to ensure they are stripped before proxying to downstream backends: + +- `HEADER_X_TS_EC` (`x-ts-ec`) +- `HEADER_X_TS_EIDS` (`x-ts-eids`) +- `HEADER_X_TS_EC_CONSENT` (`x-ts-ec-consent`) +- `HEADER_X_TS_EIDS_TRUNCATED` (`x-ts-eids-truncated`) +- Dynamic `X-ts-` headers — these cannot be registered statically. The current `INTERNAL_HEADERS` filter uses explicit names, not a wildcard. Engineering must either extend the filter to strip the full `x-ts-` prefix pattern or enumerate all active partner IDs at startup. This must be confirmed before shipping. + +--- + +## 16. Error Handling + +New error variants in `crates/common/src/error.rs`: + +```rust +pub enum TrustedServerError { + // ... existing variants ... + + /// Edge Cookie operation failed. + #[display("Edge Cookie error: {message}")] + EdgeCookie { message: String }, + // Maps to StatusCode::INTERNAL_SERVER_ERROR (500) + // Used for: EC generation failure, req.get_client_ip_addr() returning None + + /// Partner not found in partner_store. + #[display("Partner not found: {partner_id}")] + PartnerNotFound { partner_id: String }, + // Maps to StatusCode::BAD_REQUEST (400) + + /// Partner API key authentication failed. + #[display("Invalid API key for partner: {partner_id}")] + PartnerAuthFailed { partner_id: String }, + // Maps to StatusCode::UNAUTHORIZED (401) +} +``` + +--- + +## 17. Request Routing + +New routes added to `route_request()` in `crates/fastly/src/main.rs`: + +```rust +// EC sync pixel — no auth required (partner validation is internal) +(GET, "/sync") → handle_sync(settings, &ec_context, kv, partner_store, req) + +// EC identity resolution — no auth required (consent-gated) +(GET, "/identify") → handle_identify(settings, &ec_context, kv, partner_store, &req) + +// CORS preflight for /identify — must be registered explicitly, current router dispatches by exact method/path +(OPTIONS, "/identify") → cors_preflight_identify(settings, &req) + +// S2S batch sync — partner API key auth (internal to handler) +(POST, "/api/v1/sync") → handle_batch_sync(settings, kv, partner_store, req) + +// Partner registration — publisher admin auth enforced in-handler (Bearer token) +(POST, "/admin/partners/register") → handle_register_partner(settings, partner_store, req) +``` + +Route ordering: EC routes are inserted before the fallback `handle_publisher_request()`. The `/admin/partners/register` route is NOT covered by the `[[handlers]]` Basic Auth config — it validates `Authorization: Bearer ` against `settings.ec.admin_token_hash` inside `handle_register_partner()`. The `[[handlers]]` block in `trusted-server.toml` must NOT include `/admin/partners/register` in its pattern (or must be narrowed so it does not cover this path). + +### 17.1 EC integration in `main.rs` + +Follows the same pattern as `GeoInfo::from_request()` which already runs pre-routing (line 70): + +```rust +// Pre-routing — read only, no generation (matches GeoInfo pattern). +// EcContext::read_from_request() extracts and stores client_ip internally +// (same req.get_client_ip_addr() call used by GeoInfo::from_request() above). +let mut ec_context = EcContext::read_from_request(&req, settings, geo_info.as_ref())?; +let kv = KvIdentityGraph::new(&settings.ec.ec_store); + +// Route dispatch — req is moved (consumed) inside the matching arm +let result = match (method, path.as_str()) { + // EC-specific routes — receive ec_context read-only + (GET, "/sync") => handle_sync(settings, &kv, partner_store, &req, &ec_context), + (GET, "/identify") => handle_identify(settings, &kv, partner_store, &req, &ec_context), + (OPTIONS, "/identify") => cors_preflight_identify(settings, &req), + (POST, "/api/v1/sync") => handle_batch_sync(settings, &kv, partner_store, req), + (POST, "/admin/partners/register") => handle_register_partner(settings, partner_store, req), + + // /auction — EC-read-only; never generates EC + (POST, "/auction") => handle_auction(settings, orchestrator, &kv, req, &ec_context).await, + + // Organic routes — generate EC if needed, then dispatch + (m, path) if integration_registry.has_route(&m, path) => { + ec_context.generate_if_needed(&req, settings, &kv)?; + integration_registry.handle_proxy(&m, path, settings, req, &ec_context).await + }, + _ => { + ec_context.generate_if_needed(&req, settings, &kv)?; + handle_publisher_request(settings, integration_registry, req, &ec_context) + }, +}; + +// finalize_response runs on every route — enforces cookie write/deletion +finalize_response(settings, geo_info.as_ref(), &ec_context, &kv, &mut response); + +// Send the response to the client first, then continue for background work. +// In Fastly Compute, calling send_to_client() flushes the response immediately; +// the WASM invocation continues running after the client connection is released. +response.send_to_client(); + +// Background pull sync — fires outbound HTTP calls using send_async() (non-blocking). +// req is already consumed above; client_ip is read from ec_context (stored at construction). +// pull_enabled_partners() returns only records with pull_sync_enabled == true. +if let (Some(ip), Ok(pull_partners)) = (ec_context.client_ip, partner_store.pull_enabled_partners()) { + pull_sync_dispatcher.dispatch_background(&ec_context, ip, &pull_partners, &kv); +} +``` + +`PullSyncDispatcher::dispatch_background` uses `Request::send_async()` for each partner call — it fires the outbound HTTP requests and collects the `PendingRequest` handles, then awaits them with a concurrency cap of `settings.ec.pull_sync_concurrency`. This does not add latency to the user-facing response because `send_to_client()` has already been called. + +--- + +## 18. Testing Strategy + +Follow the project's **Arrange-Act-Assert** pattern. Test both happy paths and error conditions. Use `expect()` with `"should ..."` messages. + +### 18.1 Unit tests + +Each module in `ec/` has a `#[cfg(test)]` module covering: + +| Module | Key test cases | +| --------------- | ---------------------------------------------------------------------------------------------- | +| `identity.rs` | IPv4/IPv6 normalization, /64 truncation, HMAC determinism, output format | +| `consent.rs` | `ec_consent_granted()`: each `Jurisdiction` variant, fail-closed `Unknown` case | +| `cookie.rs` | Cookie string format, Max-Age=0 for deletion, domain derivation | +| `kv.rs` | Serialization/deserialization roundtrip, CAS merge logic, metadata extraction | +| `partner.rs` | API key hash verification (constant-time), record serialization | +| `sync_pixel.rs` | All `ts_synced` redirect codes, 429 rate limit, return URL construction | +| `sync_batch.rs` | Status code selection (200/207/401/400/429), per-mapping rejection reasons, API-key rate limit | +| `pull_sync.rs` | Trigger conditions, null/404 no-op, dispatch limit | +| `identify.rs` | All response codes (200/403/204), degraded flag, `uids` filtering | + +### 18.2 Integration tests + +KV behavior is tested with Viceroy (local Fastly Compute simulator) using real KV store operations. Key scenarios: + +- Consent withdrawal: cookie deletion + KV delete in same request +- Concurrent writes: CAS retry logic under simulated generation conflicts +- KV degraded: EC cookie still set when KV create fails +- Full sync-and-identify flow: pixel sync writes, then `/identify` returns the uid + +**Eventually-consistent caveat:** Fastly KV does not guarantee read-after-write consistency. Acceptance criteria that require a sync write to be immediately visible to a subsequent `/identify` read are written too strongly for the production platform. Integration tests under Viceroy may exhibit different consistency behavior than production. Tests for the sync→identify flow should either use retry with backoff (up to 1s) or be documented as a Viceroy-only behavior that is eventually consistent in production. + +### 18.3 JS tests (if applicable) + +If any JS changes are made for EC (e.g., publisher-side `/identify` fetch helper in `crates/js/`), use Vitest with `vi.hoisted()` for mocks. + +--- + +## 19. Implementation Order + +Suggested order to minimize risk and allow incremental testing. Each step should pass `cargo test --workspace` before the next begins. + +| Step | Scope | Deliverable | +| ---- | --------------------------------------------------------- | ----------------------------------------------------------------------------------- | +| 1 | `ec/identity.rs` + constants + settings | `generate_ec()`, `normalize_ip()`, `EcContext` | +| 2 | `ec/consent.rs` | `ec_consent_granted()` gating layer (consent pipeline is a prerequisite) | +| 3 | `ec/cookie.rs` | Cookie creation, deletion, response header | +| 4 | `ec/kv.rs` | `KvIdentityGraph` CRUD with CAS | +| 5 | `ec/partner.rs` + `ec/admin.rs` | `PartnerStore`, `/admin/partners/register` | +| 6 | EC middleware in `main.rs`, `publisher.rs`, `registry.rs` | `EcContext::read_from_request()` pre-routing, `generate_if_needed()`, `finalize_response()` | +| 7 | `ec/sync_pixel.rs` | `GET /sync` handler + route | +| 8 | `ec/identify.rs` | `GET /identify` handler + route | +| 9 | `ec/sync_batch.rs` | `POST /api/v1/sync` handler + route | +| 10 | `ec/pull_sync.rs` | Async pull dispatch after response | +| 11 | Auction integration | EC injection into `user.id`, `user.eids`, `user.consent` | +| 12 | End-to-end integration tests | Viceroy-based flow tests | + +--- + +## 20. Epic and Stories + +### Epic: Implement Server-Side Cookie (SSC) identity system + +Enable the trusted server to generate, persist, and serve a publisher-owned, +privacy-safe Edge Cookie (EC) that can be used for ID sync, identity lookup, +and auction decoration — without relying on third-party cookies. + +**Done when:** All 12 stories below are complete, `cargo test --workspace` and +`cargo clippy` pass with no warnings, and the end-to-end Viceroy flow tests +cover the full sync → identify → auction path. + +**Spec ref:** This document. PRD: `docs/internal/ssc-prd.md`. + +--- + +### Story 1 — EC generation and request context + +Implement the core EC data types, generation logic, and per-request context +struct that all subsequent stories depend on. + +**Scope:** `ec/identity.rs`, `ec/mod.rs`, `trusted-server.toml` `[ec]` section, +`Settings` struct update. + +**Acceptance criteria:** + +- `generate_ec(passphrase, ip)` produces a deterministic 71-char string: + 64-char lowercase hex hash + `.` + 6-char random alphanumeric suffix. + HMAC inputs are `normalize_ip(ip)` as message and `passphrase` as key. +- `normalize_ip()` truncates IPv6 to /64 (first 4 groups), passes IPv4 unchanged. +- IP is sourced from `req.get_client_ip_addr()` — no header fallback. +- `EcContext::read_from_request(req, settings, geo)` reads the `ts-ec` cookie + and `X-ts-ec` header, sets `cookie_was_present`, `ec_was_present`, `ec_value`. + Does not generate. Does not write KV. +- `EcContext::generate_if_needed(req, settings, kv)` generates a new EC when + `ec_value == None && consent == Granted`, sets `ec_generated = true`, and writes + the initial KV entry via `kv.create()` (best-effort). +- `[ec]` settings block parses from TOML: `enabled`, `passphrase`, `ec_store`, + `partner_store`, `pull_sync_concurrency`. +- All unit tests in `identity.rs` pass (HMAC determinism, format, IP normalization). + +**Spec ref:** §2, §3, §4, §5.4, §14.1 + +--- + +### Story 2 — EC consent gating layer *(prerequisite: consent pipeline already merged)* + +Add `ec_consent_granted()` — the thin EC-specific gating function that derives a +grant/deny decision from the pre-existing `ConsentContext`. + +**Scope:** `ec/consent.rs` (new file; consent pipeline itself is a prerequisite) + +**Acceptance criteria:** + +- `ec_consent_granted(consent: &ConsentContext) -> bool` is implemented per §6.1.1. + - `Jurisdiction::Gdpr` → requires `has_storage_consent()` and `!expired` + - `Jurisdiction::UsState(_)` → requires `!gpc` and no CCPA opt-out + - `Jurisdiction::NonRegulated` → `true` + - `Jurisdiction::Unknown` → `false` (fail-closed) +- Unit tests cover each `Jurisdiction` variant × signal combination. + +**Spec ref:** §6.1.1 + +--- + +### Story 3 — EC cookie helpers + +Implement the functions that create and delete the `ts-ec` cookie on responses, +and wire them into `finalize_response()`. + +**Scope:** `ec/cookie.rs`, `finalize_response()` in `main.rs` + +**Acceptance criteria:** + +- `create_ec_cookie()` produces a cookie with `Domain=.{publisher.domain}`, + `Max-Age=31536000`, `SameSite=Lax; Secure`. `HttpOnly` is NOT set + (JS on the publisher page must be able to read the cookie). +- `delete_ec_cookie()` produces a cookie with `Max-Age=0`, same attributes. +- `set_ec_on_response()` sets `Set-Cookie` and `X-ts-ec` response headers. +- `finalize_response()` signature updated to accept `ec_context: &EcContext` and `kv: &KvIdentityGraph`. +- `finalize_response()` deletes the cookie and calls `kv.write_withdrawal_tombstone()` when + `!ec_consent_granted(&consent) && cookie_was_present`. +- `finalize_response()` sets the cookie only when `ec_generated == true`. + No other cookie writes occur. No `suppress_mutation` flag. +- Unit tests cover cookie string format, Max-Age=0 deletion, domain derivation. + +**Spec ref:** §5.1, §5.3, §5.4, §17 (finalize_response) + +--- + +### Story 4 — KV identity graph + +Implement the KV read/write/delete layer for EC identity entries, including +CAS-based concurrent write protection and consent withdrawal delete. + +**Scope:** `ec/kv.rs` + +**Acceptance criteria:** + +- `KvIdentityGraph::get(ec_hash)` returns the deserialized entry and generation + marker as `Option<(KvEntry, u64)>`, or `None` if not found. +- `KvIdentityGraph::get_metadata(ec_hash)` returns `Option` for + cheap consent/country checks without streaming the full body. +- `KvIdentityGraph::create(ec_hash, &entry)` writes a new entry with + `consent.ok = true` using CAS; retries up to 3 times on generation conflict. +- `KvIdentityGraph::create_or_revive(ec_hash, &entry)` creates a new entry OR + overwrites an existing tombstone (`consent.ok = false`) with a fresh entry; + no-ops if a live entry already exists. Called by `generate_if_needed()`. +- `KvIdentityGraph::update_last_seen(ec_hash)` updates `last_seen` without + overwriting partner IDs (CAS merge), and only writes if the stored value is + more than 300s old (debounce to avoid 1 write/sec KV limit). +- `KvIdentityGraph::write_withdrawal_tombstone(ec_hash)` sets `consent.ok = false`, + clears partner IDs, and applies a 24-hour TTL (see §6.3). +- `kv.upsert_partner_id(ec_hash, partner_id, uid, timestamp)` writes to + `ids[partner_id]` and skips if existing `synced >= timestamp` (idempotent). +- KV schema matches §7 exactly (JSON roundtrip test). +- Unit tests cover CAS merge logic, tombstone write, serialization/deserialization + roundtrip, metadata extraction. + +**Spec ref:** §4, §5.4, §6.3 + +--- + +### Story 5 — Partner registry and admin endpoint + +Implement `PartnerRecord`, `PartnerStore`, and the admin registration endpoint +that operators use to onboard ID sync partners. + +**Scope:** `ec/partner.rs`, `ec/admin.rs`, router update + +**Acceptance criteria:** + +- `PartnerRecord` contains all fields from §13.1 including + `pull_sync_allowed_domains` and `batch_rate_limit`. +- `PartnerStore::get()`, `upsert()`, `find_by_api_key_hash()` operate on + `partner_store` KV. +- API key stored as SHA-256 hex; plaintext never written to KV. +- `verify_api_key()` uses constant-time comparison. +- `POST /admin/partners/register` validates `Authorization: Bearer ` inside + the handler against `settings.ec.admin_token_hash` (constant-time SHA-256 comparison). + Returns `401` if missing or invalid — before any request body is read. +- Admin endpoint validates: `pull_sync_url` hostname must be in + `pull_sync_allowed_domains` when set — returns `400` otherwise. +- Returns `201 Created` with the stored record on success; `400` on validation + failure; `503` on KV failure. +- `/admin/partners/register` is added to `Settings::ADMIN_ENDPOINTS` in + `settings.rs` and the auth-coverage tests pass (`settings.rs:1363,1395`). +- Unit tests cover API key hash verification and record serialization. + +**Spec ref:** §13 + +--- + +### Story 6 — EC middleware integration + +Wire `EcContext` into the request pipeline following the two-phase model +(§5.4 and §17.1). `EcContext::read_from_request()` runs pre-routing like +`GeoInfo`; `generate_if_needed()` runs inside organic handlers only. + +**Scope:** `main.rs`, `publisher.rs`, `registry.rs` (route wiring only — no new modules) + +**Acceptance criteria:** + +- `EcContext::read_from_request()` is called before the route match on every + request, passed the existing `geo_info` (no duplicate geo header parsing). +- EC-specific and EC-read-only route handlers (`/sync`, `/identify`, `/auction`, + `/api/v1/sync`, `/admin/*`) receive `ec_context` in read-only form — they never + call `generate_if_needed()`. `/auction` consumes EC identity but never bootstraps it. +- `handle_publisher_request()` and `integration_registry.handle_proxy()` call + `ec_context.generate_if_needed(&req, settings, &kv)` before their handler logic. +- `finalize_response()` receives `ec_context` and `kv` and: + - Deletes the EC cookie and writes a withdrawal tombstone if consent is withdrawn (runs on all routes). + - Sets a new `Set-Cookie` only when `ec_context.ec_generated == true`. +- No existing route behavior changes — EC context is additive. +- `cargo test --workspace` passes with no regressions. + +**Spec ref:** §5, §17 + +--- + +### Story 7 — Pixel sync (`GET /sync`) + +Implement the pixel-based ID sync endpoint that partners use to write their +user ID against an EC hash. + +**Scope:** `ec/sync_pixel.rs`, router update + +**Acceptance criteria:** + +- Missing required query params (`partner`, `uid`, `return`) → `400`. +- No `ts-ec` cookie → redirect to `{return}?ts_synced=0&ts_reason=no_ec`. +- Unknown `partner` ID → `400`. +- `return` URL hostname not in `partner.allowed_return_domains` → `400`. +- Consent uses `ec_context.consent`. The optional `consent` query param is a fallback + only: it is used exclusively when `ec_context.consent.is_empty()` + (no X-consent-advertising header and no framework cookie on the request). + When a fresher signal exists, the param is ignored. Does not mutate `ec_context`. + Denied or absent → redirect to `{return}?ts_synced=0&ts_reason=no_consent`. +- Rate limit exceeded → `429 Too Many Requests` (no redirect). +- KV write failure → redirect to `{return}?ts_synced=0&ts_reason=write_failed`. +- Success → redirect to `{return}?ts_synced=1`. +- Return URL construction correctly appends `&` or `?` based on existing query string. +- Rate counter key: `{partner_id}:{ec_hash}`, 1-hour window, via `fastly::erl::RateCounter`. +- Unit tests cover all redirect/response codes and return URL construction. + +**Spec ref:** §8 + +--- + +### Story 8 — Identity lookup (`GET /identify`) + +Implement the browser-facing endpoint that publishers call to retrieve the EC +hash and synced partner UIDs for the current user. + +**Scope:** `ec/identify.rs`, router update + +**Acceptance criteria:** + +- No `ts-ec` cookie AND no `X-ts-ec` header (`ec_was_present == false`) and `!ec_consent_granted(consent)` → `403 Forbidden`. +- No `ts-ec` cookie AND no `X-ts-ec` header (`ec_was_present == false`) and consent not denied → `204 No Content`. +- Valid EC, consent granted, KV read succeeds → `200` with full JSON body + including `ec`, `consent`, `uids`, `eids`. +- `uids` filtered to partners where `bidstream_enabled = true` and consent + granted. +- KV read failure → `200` with `degraded: true` and empty `uids`/`eids`. +- No `Origin` header (server-side proxy): process normally, no CORS headers, no `403`. +- `Origin` header present and matches `publisher.domain` or subdomain: reflect in + `Access-Control-Allow-Origin` + `Vary: Origin`. +- `Origin` header present but does not match: `403`, no body. +- `OPTIONS /identify` preflight → `200` with CORS headers, no body. +- `generate_if_needed()` is never called — no new EC generated, no `Set-Cookie`. +- Response time target: 30ms p95 (documented, not gate). +- Unit tests cover all response codes, degraded flag, `uids` filtering, + CORS origin validation. + +**Spec ref:** §11 + +--- + +### Story 9 — S2S batch sync (`POST /api/v1/sync`) + +Implement the server-to-server batch sync endpoint for partners to bulk-write +their UIDs against a list of EC hashes. + +**Scope:** `ec/sync_batch.rs`, router update + +**Acceptance criteria:** + +- Missing or invalid `Authorization: Bearer` → `401`. +- API-key rate limit exceeded (`batch_rate_limit` per partner per minute) → `429` + with `{ "error": "rate_limit_exceeded" }`. +- More than 1000 mappings → `400`. +- Per-mapping rejections: `invalid_ec_hash`, `ec_hash_not_found`, + `consent_withdrawn`, `kv_unavailable`. +- KV write failure aborts remaining mappings with `kv_unavailable`; partial + results returned as `207`. +- All mappings accepted → `200`. Any rejection → `207`. +- `kv.upsert_partner_id()` is idempotent: duplicate timestamp counted as + accepted, no error. +- Rate counter key: `batch:{partner_id}`, 1-minute window. +- Unit tests cover status code selection, all rejection reasons, and API-key + rate limit. + +**Spec ref:** §9 + +--- + +### Story 10 — Pull sync dispatch + +Implement the async background task that calls partner resolution endpoints +after a response is flushed, when trigger conditions are met. + +**Scope:** `ec/pull_sync.rs` + +**Acceptance criteria:** + +- Dispatch only when: EC present, consent granted, `pull_sync_enabled = true`, + and either no existing partner entry or existing `synced` is older than + `pull_sync_ttl_sec`. +- Rate limit: `pull_sync_rate_limit` per EC hash per partner per hour; counter + key `pull:{partner_id}:{ec_hash}`. +- Maximum concurrent pulls per request: `settings.ec.pull_sync_concurrency` + (default 3). +- Before calling, validate `pull_sync_url` hostname is in + `pull_sync_allowed_domains`; skip and log `error` if not. +- Outbound request: `GET {pull_sync_url}?ec_hash={hash}&ip={ip}` with + `Authorization: Bearer {ts_pull_token}`. +- `{ "uid": null }` and `404` are no-ops — no KV write, no error logged above + `debug`. +- Any other non-200 → transient failure, no retry, no error above `warn`. +- Dispatch is non-blocking — does not add latency to the user-facing response. +- Unit tests cover trigger conditions, null/404 no-op, domain allowlist check, + dispatch limit enforcement. + +**Spec ref:** §10 + +--- + +### Story 11 — Auction bidstream decoration + +Inject EC identity data into outbound OpenRTB bid requests for publishers with +`bidstream_enabled = true` partners. + +**Scope:** Auction handler (Mode B path in existing auction code) + +**Acceptance criteria:** + +- `user.id` set to `ec_context.ec_value` (the full `hash.suffix` string) when EC present and consent granted; falls back to `synthetic_id` when EC is absent (matching §12.1 coexistence table — EC is authoritative where present, synthetic otherwise). +- `user.eids` populated with one entry per `bidstream_enabled` partner that + has a synced UID, using `partner.source_domain` and `partner.openrtb_atype`. +- `user.consent` set to `ec_context.consent.raw_tc_string` when present. +- No EID entry written for partners with no synced UID. +- KV read failure → `user.eids` omitted (empty); `user.id` still set from EC or synthetic fallback; auction proceeds without EID data (no 5xx). +- No EC present → `user.id` set from synthetic fallback; `user.eids` is empty. +- Unit tests cover EID structure, consent string threading, KV-degraded path. + +**Spec ref:** §12 + +--- + +### Story 12 — End-to-end integration tests + +Write Viceroy-based integration tests covering the full identity lifecycle +across multiple handlers in a single simulated environment. + +**Scope:** `tests/` (integration test crate or new test module) + +**Acceptance criteria:** + +- **Full flow:** First-party page load → EC generated → pixel sync writes + partner UID → `/identify` returns that UID → auction includes EID. +- **Consent withdrawal:** Request with denied consent clears EC cookie and writes + a KV tombstone (`consent.ok = false`, 24h TTL) in the same request; subsequent + `/identify` with consent still denied returns `403` (no cookie + denied → §11.3); + batch sync returns `consent_withdrawn` within the tombstone TTL. +- **KV degraded:** EC cookie is still set when KV create fails; `/identify` + returns `degraded: true`. +- **Concurrent writes:** Two simultaneous EC creates for the same hash resolve + without data loss (CAS retry). +- **Rate limits:** Pixel sync returns `429` after `sync_rate_limit` is + exceeded; batch sync returns `429` after `batch_rate_limit` is exceeded. +- **Pull sync no-op:** Partner returning `{ "uid": null }` produces no KV + write and no error log. +- All tests pass under `cargo test --workspace` with Viceroy. + +**Spec ref:** §18.2