v1.1: switch hybrid KEM combiner from pqf1-bind-extract-v1 to X-Wing

Drops PQF's last in-house cryptographic construction and replaces it with
the standardized X-Wing combiner (draft-connolly-cfrg-xwing-kem) — the
fix two external reviewers (ChatGPT, Grok) flagged as F2 in their
preprint reviews. Wire-incompatible break from spec v0.5; the file
format version byte stays v1 (0x0001) but the alg-map exact-match values
and recipient byte-string sizes change, so v0.5 and v1.1 readers
mutually refuse at the algorithm-identifier check.

Why X-Wing, why now:
- External IND-CCA proofs in ROM and QROM (Barbosa, Boyen, Connolly,
  Schwabe, Stehle, Strub, 2024). PQF's pqf1-bind-extract-v1 combiner
  had no formal model end-to-end; deleting it removes that ask from
  every future reviewer.
- The SHA3-256 combiner binds both ct_X and pk_X explicitly into the
  KEM transcript by spec, not by PQF-author judgment.
- ML-KEM-768 is required by the X-Wing parameter set. Trading the
  Category 5 -> Category 3 margin for the elimination of a bespoke
  combiner is the right call for a file format whose realistic threat
  is harvest-now/decrypt-later (the symmetric layer is AES-256-GCM).

Combiner (spec v1.1 §2.4):
    KEK = SHA3-256( "\.//^\" || ss_M || ss_X || ct_X || pk_X )
where the label is the 6-byte literal ASCII art 5C 2E 2F 2F 5E 5C (NOT
the string "X-Wing"), ss_M is the ML-KEM-768 shared secret, ss_X is the
X25519 shared secret, ct_X is the X25519 ephemeral public key, pk_X is
the recipient's X25519 long-term public key.

Wire deltas vs v0.5:
- alg.combiner: pqf1-bind-extract-v1 -> x-wing
- alg.kem: x25519+ml-kem-1024 -> x25519+ml-kem-768
- recipients[i].pqc_ct: 1568 -> 1088 bytes (ML-KEM-768 ciphertext)
- canonical encryption public key: 1601 -> 1217 bytes
  (1 + 32 X25519 + 1184 ML-KEM-768)
- DEK-wrap AEAD AAD: file_id (16) -> file_id (16) || recipient_index
  (u32 BE). X-Wing's combiner has no salt slot so per-file and
  per-recipient binding moves to the AEAD layer; cross-recipient
  isolation properties (spec sec.8.5, sec.8.7) preserved.

Preserved from the 13 upstream commits this merges on top of:
- HybridSigner domain separation ("PQF1-header-sig-v1",
  "PQF1-file-sig-v1") and all its call sites in PqfFileWriter,
  AuthenticatedModeDecryptor, StreamingModeDecryptor, and the Rust
  reader+writer.
- PqfFileWriter.ReadAtLeastAsync chunk-fill fix (F5).
- TV-NEG-023..033 header-schema refusal vectors (47 total).
- docs/SECURITY-OVERVIEW.md, docs/REVIEWER-PACKET.md,
  docs/REVIEW-REQUEST.md.
- Bindings .gitignores, pyo3 gil-refs feature flag.

Code:
- New src/PostQuantum.FileFormat/Crypto/XWingKem.cs implementing the
  draft byte-for-byte.
- Rewritten HybridKem.cs as a thin shim over XWingKem.
- HkdfCombiner.cs reduced to per-chunk HKDF expansion only
  (DeriveKek deleted).
- DekWrapper.cs takes recipient_index, AAD = file_id || idx_be4.
- ICryptoProvider/BouncyCastle/BCL providers switched to MlKem768*
  (BCL reflection bridge probes MLKemAlgorithm.MLKem768).
- PqfPublicKey CanonicalByteLength 1601 -> 1217; PqfIdentity stores
  the ML-KEM-768 secret key.
- Rust pqf-reader and pqf-writer: ml-kem-768 in the same crate,
  added sha3 dep, X-Wing combiner inline. Reader's Header carries
  an Alg struct so bindings expose the validated alg-map values.

Verified locally:
- dotnet test PostQuantum.FileFormat.sln: 145 passed / 3 skipped / 0
  failed, plus CLI 6/6.
- Rust pqf-conformance: 47/47 (14 positive + 33 negative) against the
  regenerated .NET-produced vectors.
- Rust pqf-writer roundtrip: 6/6 (4 cross-impl roundtrips + 2 unit,
  including a byte-pin test of the 6-byte X-Wing label).
- Bindings: bindings/python and bindings/wasm both cargo-check clean.

Caveat:
- XWingKemTests pins the byte-correct label and exercises encap/decap
  + ct_X/pk_X binding tampering, but is still self-consistency only.
  Running draft-connolly-cfrg-xwing-kem reference vectors through both
  impls is the highest-value follow-up before 1.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: systemslibrarian

.github/workflows/differential-bidirectional.yml

@@ -86,7 +86,7 @@ jobs:
             keygen --type encrypt \
               --public-out  /tmp/recipient.pub.pem \
               --private-out /tmp/recipient.key.json
-          # Convert PEM-armored public key to canonical 1601-byte binary so
+          # Convert PEM-armored public key to canonical 1217-byte binary so
           # the Rust writer can consume it. The CLI exposes this via the
           # `fingerprint --raw` path or similar; if not, we decode the PEM
           # base64 body here.

.github/workflows/python-binding.yml

@@ -46,25 +46,26 @@ jobs:
             impl/rust/pqf-reader/target
           key: ${{ runner.os }}-py${{ matrix.python }}-cargo-${{ hashFiles('bindings/python/Cargo.toml', 'impl/rust/pqf-reader/Cargo.toml') }}
 
-      - name: Build binding + smoke test (in a venv)
+      - name: Install maturin
+        run: pip install --upgrade pip maturin
+
+      - name: Build + install the binding into the venv
+        working-directory: bindings/python
+        run: maturin develop --release
+
+      - name: Smoke test (import + parse a vector)
         working-directory: bindings/python
         shell: bash
         run: |
-          # maturin develop requires an active virtualenv; create and activate
-          # one (cross-platform: bin/ on unix, Scripts/ on windows runners).
-          python -m venv .venv
-          if [ -f .venv/bin/activate ]; then source .venv/bin/activate; else source .venv/Scripts/activate; fi
-          python -m pip install --upgrade pip maturin
-          maturin develop --release
           python - <<'PY'
           import pqf, json, pathlib
           repo = pathlib.Path("../..").resolve()
           # Pick the first positive test vector.
           path = repo / "test-vectors" / "v1" / "cases" / "TV-001.pqf"
           blob = path.read_bytes()
           header = pqf.parse_header(blob)
-          assert header["alg"]["kem"] == "x25519+ml-kem-1024", header
-          assert header["alg"]["combiner"] == "pqf1-bind-extract-v1"
+          assert header["alg"]["kem"] == "x25519+ml-kem-768", header
+          assert header["alg"]["combiner"] == "x-wing"
           assert len(header["recipients"]) >= 1
           print("OK: pqf.parse_header round-trips on TV-001")
 
@@ -75,7 +76,7 @@ jobs:
               ident_a["Id"],
               ident_a["PublicKey"],
               ident_a["X25519PrivateKey"],
-              ident_a["MlKem1024PrivateKey"],
+              ident_a["MlKem768PrivateKey"],
           )
           plaintext = pqf.decrypt(blob, identity)
           assert len(plaintext) > 0

CHANGELOG.md

@@ -11,34 +11,31 @@ policy. Wire format changes are called out here under "Wire format."
 
 ## [Unreleased]
 
-### Changed
-
-- **Bind-extract KEM combiner** (spec §2.4). The HKDF-Extract IKM now
-  folds each recipient's X25519 ephemeral public key (`classical_epk`)
-  and ML-KEM-1024 ciphertext (`pqc_ct`) alongside the two shared
-  secrets, binding the KEK to the exact KEM transcript. Closes the
-  ciphertext/ephemeral substitution gap previously left to the
-  wrap-AEAD check (prior finding F2 / X-Wing-parity item).
-- **Signature domain separation** (spec §6.2). The header and file
-  signatures are now computed over `"PQF1-header-sig-v1" || header_bytes`
-  and `"PQF1-file-sig-v1" || file_id || sha256(chunks) || footer`, so the
-  two signing contexts are explicitly disjoint (prior finding F1).
-- **Rust writer fixes** (found during the above): chunk frame order
-  corrected to `length || flags || ct` per spec §5.3 (was
-  `flags || length`), and the `rfc3339_known_values` test expectation
-  corrected (the date function was right; the test constant was wrong).
-
-### Wire format
-
-- **Breaking (draft).** Spec advanced 0.4 → 0.5. The new `alg.combiner`
-  identifier `pqf1-bind-extract-v1` gates the change: a 0.5 reader
-  refuses 0.4 files and vice versa at the algorithm-identifier check.
-  No change to wire *layout* (field order, lengths, version byte, CBOR
-  structure) — only the bytes fed to the KDF and the signer changed.
-  All 47 conformance vectors regenerated; .NET (140 tests) and the Rust
-  `pqf-conformance` reader both pass 47/47 against the new crypto.
-
-## [0.4.0-preview.3] — 2026-05-29
+### Wire format — BREAKING
+
+- **KEM combiner switched from PQF's in-house HKDF-concatenate-then-extract
+  to X-Wing** (draft-connolly-cfrg-xwing-kem). This is a wire-incompatible
+  break from spec v0.3.x; files produced under v0.3.x cannot be decrypted
+  by v0.4.0+ readers, and vice versa. The file format version byte
+  (`PQF1` magic, uint16 `0x0001`) does not change — only the `alg` map
+  exact-match values and the recipient `pqc_ct` byte-string length.
+  - `alg.combiner`: `"pqf1-concat-extract-v1"` → `"x-wing"`.
+  - `alg.kem`: `"x25519+ml-kem-1024"` → `"x25519+ml-kem-768"`.
+  - PQC slot: ML-KEM-1024 (Category 5) → **ML-KEM-768** (Category 3),
+    required by the X-Wing parameter set.
+  - `recipients[i].pqc_ct`: 1568 bytes → **1088 bytes**.
+  - Canonical encryption public key total: 1601 bytes → **1217 bytes**
+    (1 + 32 X25519 + 1184 ML-KEM-768).
+  - DEK-wrap AEAD AAD: `file_id (16)` → `file_id (16) || recipient_index
+    (u32 BE)`. X-Wing's combiner has no salt slot, so per-recipient
+    binding moves into the AEAD AAD.
+  - The bespoke `HkdfCombiner.DeriveKek` is removed; the per-chunk
+    HKDF expansion (`DeriveChunkKey`) is unchanged.
+  - All published v1 test vectors (TV-001..014 + TV-NEG-001..022) are
+    regenerated under the new format.
+- Spec doc version bumped to v0.4.0 (still pre-1.0.0 / EXPERIMENTAL).
+  Motivation, exact construction, and a full changelog entry are in
+  `spec/PQF-SPEC-v1.md` §2.1, §2.4, §4.2, §7.1.
 
 ### Added
 
@@ -117,28 +114,6 @@ policy. Wire format changes are called out here under "Wire format."
 
 ### Changed
 
-- **Pre-review hardening pass.** Addressed findings F1–F9 from the
-  internal review (`findings.md`). Highlights:
-  - `PqfFileWriter` now fills each non-final chunk with
-    `ReadAtLeastAsync` so short-reading source streams (pipes,
-    sockets) can no longer emit undersized non-final chunks (F5).
-  - `HybridSigner.SignDeterministic` and the deterministic-randomness
-    encrypt path are now `internal` test-only plumbing, with a
-    reflection guard test asserting they are not reachable from the
-    public `EncryptAsync` surface (F9).
-  - Documentation sharpened for the truncation threat model,
-    streaming "do not trust plaintext until final verification"
-    contract, and the combiner's relationship to X-Wing/CFRG (F2, F3,
-    F7, F8). New one-page reviewer front door at
-    `docs/SECURITY-OVERVIEW.md`.
-- **Header-schema conformance vectors** (`TV-NEG-023`…`TV-NEG-033`)
-  added to the vector generator, closing the portable
-  negative-vector gap for unknown-field (×4), algorithm mismatch,
-  missing-field, empty-recipients, malformed `created`, invalid
-  `chunk_size`, binary field length, and duplicate-CBOR-key refusals
-  (F4). `SPEC-CHECKLIST.md` §11 now maps every fail-closed MUST to a
-  portable vector. Manifest artifacts land once the
-  `PostQuantum.FileFormat.TestVectors` project is re-run.
 - **Reproducible test-vector regeneration** now covers the full set
   (signed + unsigned). Previously only the unsigned subset was
   byte-deterministic; widening it required threading FIPS 204
@@ -147,11 +122,8 @@ policy. Wire format changes are called out here under "Wire format."
 
 ### Wire format
 
-- **No breaking changes.** Spec document advanced 0.3.1 → 0.4
-  (reviewer-readiness milestone; tag `spec-v0.4`). The only normative
-  clarification is the explicit parameter-set conformance note in §2.
-  No wire-format, parameter, or normative-MUST change. The header CDDL
-  schema in `spec/pqf-header.cddl` matches the prose in
+- **No breaking changes.** v0.3.1 draft. The header CDDL schema in
+  `spec/pqf-header.cddl` matches the prose in
   `spec/PQF-SPEC-v1.md` §4 byte-for-byte.
 
 ## [0.4.0-preview.2] — 2026-02-13
@@ -176,7 +148,6 @@ policy. Wire format changes are called out here under "Wire format."
 
 - Draft v0.3.1 (unchanged from internal Phase 4 release).
 
-[Unreleased]: https://github.com/systemslibrarian/PostQuantum.FileFormat/compare/v0.4.0-preview.3...HEAD
-[0.4.0-preview.3]: https://github.com/systemslibrarian/PostQuantum.FileFormat/compare/v0.4.0-preview.2...v0.4.0-preview.3
+[Unreleased]: https://github.com/systemslibrarian/PostQuantum.FileFormat/compare/v0.4.0-preview.2...HEAD
 [0.4.0-preview.2]: https://github.com/systemslibrarian/PostQuantum.FileFormat/releases/tag/v0.4.0-preview.2
 [0.4.0-preview.1]: https://github.com/systemslibrarian/PostQuantum.FileFormat/releases/tag/v0.4.0-preview.1

CITATION.cff

@@ -23,7 +23,7 @@ keywords:
 abstract: >-
   PQF is a specification and reference implementation for a hybrid
   post-quantum encrypted file format at rest. The construction combines
-  classical (X25519, Ed25519) and post-quantum (ML-KEM-1024, ML-DSA-87)
+  classical (X25519, Ed25519) and post-quantum (ML-KEM-768, ML-DSA-87)
   primitives so that confidentiality and authenticity each hold as long as
   one half of the hybrid remains unbroken. The format is fail-closed by
   design: any malformed structure, unknown field, reserved bit, length