Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,29 @@ All notable changes to zencodec are documented here.

## [Unreleased]

### Added

- **Cross-codec color-emission policy** (`zencodec::color`) —
`resolve_color_emit(&SourceColor, &EncodeCapabilities, ColorPolicy) -> ColorPlan`,
a pure `no_std` decision of which color carriers (ICC vs CICP) to write for a
target, with no CMS and no codec dependencies.
- `ColorPolicy { Compatibility, Balanced (default), Compact, Verbatim, Custom(ColorFields) }`;
`ColorPlan { cicp: Option<Cicp>, icc: IccDisposition }`;
`IccDisposition { KeepSource, SynthesizeFrom(Cicp), Drop }`. Handles the
grayscale/CMYK terminal states and never emits a redundant `SynthesizeFrom(sRGB)`.
- `ColorFields::new` makes `ColorPolicy::Custom` constructible downstream.
- `EncodeCapabilities` gains `cicp_is_valid_carrier` (standardized carrier —
JXL/AVIF/HEIC `nclx`, PNG `cICP`) and `cicp_safe_sole_carrier` (safe CICP-only,
JXL) (+ `with_*`); `IccRetention` gains `DropIfCicpRepresentable`,
`DropIfCicpSafeSoleCarrier`. The plan lowers to `zenpixels_convert`'s
`finalize_for_output_with` (`icc_profile_for_primaries` materializes a
`SynthesizeFrom` from a `const fn` table — no CMS, never a silent drop).
- `helpers::set_exif_orientation` rewrites a blob's EXIF orientation tag inline
(offset-preserving) so a baked-upright pixel buffer and its embedded tag can't
disagree (the double-rotation hazard). Applied by the pipeline, not by the
color resolver.
- Design + rejected alternatives: `docs/color-emit-model.md`.

## [0.1.21] - 2026-05-29

### Added
Expand Down
28 changes: 27 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,4 +66,30 @@ Tiny, stable crate defining the common interface that all zen* codecs implement:

## Known Issues

(none)
Three bugs verified during the cross-codec color/metadata scenario-matrix
research (2026-06-01). The first is in this crate; the other two are recorded
here as cross-repo findings (do NOT edit those repos from here — flag to the
owner). Full design context: [`docs/color-emit-model.md`](docs/color-emit-model.md).

1. **Double-rotation hazard (this crate, `src/metadata.rs`).** When a decoder
bakes orientation upright it sets `Metadata::orientation = Identity`, but the
EXIF blob still carries the `Orientation` tag (e.g. `6`). `Metadata::filtered`
keeps that tag, so the field says `Identity` while the blob says `Rotate90` —
they disagree, and a consumer that re-applies the EXIF tag rotates twice. The
test at `src/metadata.rs:816` currently locks in keeping the stale tag. The
byte-level fix now exists — `helpers::set_exif_orientation(blob, 1)` rewrites
the inline tag offset-preservingly. **Still TODO:** the pipeline (the layer
that bakes orientation) must actually call it on the emitted blob, and the
`metadata.rs:816` test should be updated to expect a rewritten tag, not a
stale one. This is a pipeline-applied fix, not a `Metadata::filtered` change.

2. **AVIF descriptor-CICP override (zenavif, `src/codec.rs:824-831`).**
`apply_descriptor_color` overrides a metadata-set CICP unconditionally,
ignoring a CICP explicitly provided via `Metadata`. It should check for a
caller-supplied CICP before overriding from the pixel descriptor.

3. **Missing signal-range conversion kernels (zenpixels-convert).** No
`Narrow <-> Full` range conversion kernels exist, so a range mismatch refuses
zero-copy but can relabel without rescaling — a black-crush risk. Needs
`ConvertStep::{Expand,Contract}NarrowToFull`. Until then, range must be
preserved verbatim, never relabeled.
132 changes: 132 additions & 0 deletions docs/color-emit-model.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
# Color emission model (grounded design)

Status: **canonical.** This records the *minimal* shared color surface and — just
as importantly — the designs that were tried, dogfooded, adversarially reviewed,
and **rejected**, so they don't get rebuilt. Companion analysis:
[`cross-codec-color-metadata.md`](cross-codec-color-metadata.md).

## Thesis

The only thing that genuinely needs to be **shared** across codecs is a *pure
color-carrier policy*: given a source's color (`SourceColor`) and a target's
capabilities (`EncodeCapabilities`), decide which carriers to write (ICC vs
CICP). Everything else — pixel+metadata materialization, specialized
coefficient-domain transcodes, the decode→re-encode orchestration — already has
a home and must **not** be pulled into a grand "emit model" or a cross-codec
trait.

This was reached the hard way: an over-built `EmitFacts`/`EmitIntent`/`EmitPlan`
"scenario" model + a `TranscodeEncoder` trait were dogfooded into 5 codecs and
adversarially reviewed; the review + a full read of zenpixels/zenpipe killed
them (see *Rejected designs*). The grounded surface is ~360 lines with **zero
codec dependencies**.

## The shared surface — `zencodec::color`

```rust
pub fn resolve_color_emit(
src: &SourceColor, // what the source file signalled (cicp / icc / channel_count)
target: &EncodeCapabilities, // which carriers the target format has + their quality
policy: ColorPolicy,
) -> ColorPlan; // { cicp: Option<Cicp>, icc: IccDisposition }

pub enum ColorPolicy { Compatibility, Balanced /*default*/, Compact, Verbatim, Custom(ColorFields) }
pub enum IccDisposition { KeepSource, SynthesizeFrom(Cicp), Drop }
pub struct ColorFields { icc: IccRetention, cicp: CicpEmission } // ::new(icc, cicp)
pub enum CicpEmission { WhereValidCarrier /*default*/, WhereverSupported, Never }
```

Pure, `no_std`, **no CMS, no codec deps**. It emits a *plan*; the bytes are
materialized one layer up. `SourceColor` is the type the pipeline actually
produces (decode → `ImageInfo.source_color`; the bridge to encode is a flat
`Metadata`). The resolver also handles the grayscale/CMYK terminal states
(suppress CICP, keep ICC) and never emits a redundant `SynthesizeFrom(sRGB)`.

### Capabilities (three flags drive it)

- `cicp()` — has a CICP carrier slot at all.
- `cicp_is_valid_carrier()` — the carrier is standardized/honored, so CICP is
emitted by default (JXL enum, AVIF/HEIC `nclx`, **PNG `cICP`**). Distinct from
authority — PNG isn't the decode authority but is a valid carrier.
- `cicp_safe_sole_carrier()` — safe to ship CICP-only and drop the ICC (JXL only;
AVIF/HEIC/PNG keep the ICC alongside).

## Lowering the plan (where the bytes happen)

A codec or the pipeline lowers `ColorPlan` to bytes through **zenpixels-convert's
`finalize_for_output_with`** — which already converts pixels *and* emits matching
`OutputMetadata` atomically (pixels and embedded color cannot diverge):

- `ColorPlan.cicp` → the format's native CICP carrier.
- `IccDisposition::KeepSource` → `OutputProfile::SameAsOrigin` (re-embed source ICC).
- `IccDisposition::SynthesizeFrom(cicp)` → `zenpixels_convert::icc_profile_for_primaries`
(a `const fn` table of bundled profiles — **no CMS, no allocation**; returns
`None` for BT.709/sRGB so the assumed default is never embedded).
- `IccDisposition::Drop` → no ICC.

So "synthesize an ICC" can never silently lose color and never needs a CMS in the
codec — it's a table lookup.

## Orientation (separate, tiny)

The double-rotation hazard (a decoder bakes orientation upright but the embedded
EXIF blob still says `Rotate90`) is closed by
`helpers::set_exif_orientation(blob, value)` — an offset-preserving inline rewrite
of the 0x0112 tag. It's applied by the **pipeline**, which knows when it baked
orientation. It is *not* part of color policy and not a "unified plan".

## Transcodes (pairwise, self-contained — not shared)

Specialized lossless/coefficient transcodes are **not** a generic capability:

- **JPEG → JPEG** (orient / recompress): entirely inside zenjpeg
(`zenjpeg::lossless`, `zenjpeg::recompress`).
- **JPEG → JXL** (lossless embed): inside jxl-encoder via jbrd (its own
`JpegData` parser — the JXL spec's recompression feature, **needs no zenjpeg**).

These preserve metadata verbatim, so they don't even call the color resolver.
The set of real pairs is tiny and well-known. The **dispatch** belongs in
**zenpipe**, which already depends on every codec — a small finite table of known
pairs calling those functions directly, plus `resolve_color_emit` on the
decode→re-encode path. No codec ever learns about another.

zenpipe already has the sketch: `try_lossless_jpeg` (in `lossless.rs`, currently
only called from tests) is the precedent to wire and generalize. **That's a later
piece**, tracked separately.

## Rejected designs (do not rebuild)

- **`EmitFacts { Fresh | Decoded | Passthrough }` + `PixelFidelity`** — nothing in
the pipeline produces a `ColorOrigin`/fidelity: decode attaches no color to the
buffer; provenance lives in `ImageInfo.source_color` and the carrier is a flat
`Metadata`. A codec `Encoder` only ever sees `with_metadata(Metadata)`, so it
could only ever build `Fresh` — the scenario machinery was dead code. The
`PixelDescriptor` already *is* the current gamut, so deriving `Reauthored` was
redundant. `resolve_color_emit(&SourceColor, …)` takes the type that flows.
- **`TranscodeEncoder` trait in zencodec** — a generic "output codec transcodes
from source-format X" trait forces every output codec to *ingest* every input
format (JXL←JPEG needs JPEG parsing; PNG←? needs zenpng; …) → **every codec
depends on every other codec**. The real pairs are ~2 and each self-contained.
zenpipe (deps-all) dispatches; no trait.
- **`EmitIntent` unifying color + metadata + orientation into one knob** —
aesthetic, not grounded. `MetadataPolicy` (#17) and `ColorPolicy` are fine
apart; orientation is a one-helper correctness fix, not a policy axis.
- **A resolver that produces final `Metadata` bytes** — a third metadata producer
alongside `Metadata::filtered` and `OutputMetadata`. Atomicity is already
`finalize_for_output_with`'s job.

## What landed (the surviving red-team fixes)

The 5-codec dogfood + adversarial review found real defects; the ones that
survived the grounding, all small and all on the `resolve_color_emit` shape:

1. `ColorFields::new` / `CicpEmission` are constructible → `ColorPolicy::Custom`
is actually reachable downstream.
2. `cicp_is_valid_carrier` tier → PNG/WebP emit cICP under Balanced instead of
laundering wide-gamut color through a synthesized ICC.
3. No redundant `SynthesizeFrom(sRGB)` (the canned table returns `None` for sRGB).
4. `set_exif_orientation` for the double-rotation hazard.

The `SynthesizeFrom`-silently-drops-color critical dissolves under lowering —
`icc_profile_for_primaries` always materializes a non-sRGB profile, never a CMS,
never a silent drop.
38 changes: 38 additions & 0 deletions src/capabilities.rs
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,9 @@ pub struct EncodeCapabilities {
exif: bool,
xmp: bool,
cicp: bool,
// CICP carrier quality (distinct from `cicp` = "has a CICP carrier slot")
cicp_is_valid_carrier: bool,
cicp_safe_sole_carrier: bool,
// Operation support
stop: bool,
animation: bool,
Expand Down Expand Up @@ -143,6 +146,8 @@ impl EncodeCapabilities {
exif: false,
xmp: false,
cicp: false,
cicp_is_valid_carrier: false,
cicp_safe_sole_carrier: false,
stop: false,
animation: false,
push_rows: false,
Expand Down Expand Up @@ -181,6 +186,27 @@ impl EncodeCapabilities {
pub const fn cicp(&self) -> bool {
self.cicp
}
/// Whether this format has a standardized, real-world-honored CICP carrier —
/// so CICP can be emitted as a color signal by default.
///
/// True for JXL codestream enum, AVIF/HEIC `nclx`, and PNG `cICP`. Distinct
/// from [`cicp`](Self::cicp) (= "has a CICP carrier slot at all") and from
/// [`cicp_safe_sole_carrier`](Self::cicp_safe_sole_carrier) (= "safe to ship
/// CICP *only* and drop the ICC"). Gates CICP emission under
/// [`CicpEmission::WhereValidCarrier`](crate::color::CicpEmission::WhereValidCarrier).
pub const fn cicp_is_valid_carrier(&self) -> bool {
self.cicp_is_valid_carrier
}
/// Whether it is safe in practice to ship CICP as the *sole* color carrier
/// and drop a redundant ICC profile for this format.
///
/// Stricter than [`cicp_is_valid_carrier`](Self::cicp_is_valid_carrier): a
/// format can have a valid CICP carrier yet still need an ICC kept for
/// real-world tool compatibility. As of 2026 this is true only for JXL
/// (matches libjxl's `want_icc=false` default); AVIF/HEIC/PNG keep the ICC.
pub const fn cicp_safe_sole_carrier(&self) -> bool {
self.cicp_safe_sole_carrier
}
/// Whether `with_stop` on encode jobs is respected (not a no-op).
pub const fn stop(&self) -> bool {
self.stop
Expand Down Expand Up @@ -316,6 +342,18 @@ impl EncodeCapabilities {
self.cicp = v;
self
}
/// Set whether this format has a standardized CICP carrier.
/// See [`cicp_is_valid_carrier`](Self::cicp_is_valid_carrier).
pub const fn with_cicp_is_valid_carrier(mut self, v: bool) -> Self {
self.cicp_is_valid_carrier = v;
self
}
/// Set whether CICP is safe as the sole color carrier (drop redundant ICC).
/// See [`cicp_safe_sole_carrier`](Self::cicp_safe_sole_carrier).
pub const fn with_cicp_safe_sole_carrier(mut self, v: bool) -> Self {
self.cicp_safe_sole_carrier = v;
self
}
/// Set whether cooperative cancellation via [`Stop`](enough::Stop) is supported.
pub const fn with_stop(mut self, v: bool) -> Self {
self.stop = v;
Expand Down
Loading