perf(lzma2/xz): linear-time match finder + faster optimal parse

[MagicalTux]

Problem

The xz/lzma2 optimal-parse encoder used a fixed 64 Ki-bucket hash head table. For incompressible or low-match input, distinct 3-byte prefixes collided into the same buckets as the input grew, so per-bucket chains lengthened with length and every probe walked O(n/table) work — encode was effectively O(n²) until the max_chain cap engaged. xz encode of 4 MiB of random data took ~6.7 s and kept worsening; native xz handles it in constant time.

Changes (compressed output is byte-for-byte unchanged at every level)

1. Linear match finder — size the hash head table to the match-finder window (a Vec + head_mask), like liblzma sizes its hash to the dictionary, so average chain length stays O(1). hash3 now returns a full 32-bit mix masked per-probe.
2. Length-price cache — cache length-symbol prices per pos_state, refreshed every 128 committed decisions, instead of an 8-bit bittree walk per length per position.
3. Distance-price hoist — the new-match distance price depends on length only through the dist-state bucket (saturates at DIST_STATES), so it's recomputed only when that bucket changes (one call for the common length≥5 band).
4. Word-at-a-time match comparison — match_len_at compares 8 bytes per step via LE u64 + trailing_zeros.

Results

Deterministic instruction counts (2 MiB, before → after): text 3.1×, all-zeros 4.0×, mixed source code 1.6× fewer instructions. Random encode is now linear and ~1.1× faster than native xz -6; realistic source code is ~0.85–0.9× native (was far slower).

Verification

• Full suite: 61/61 test binaries pass.
• 60-case randomized roundtrip fuzz (sizes incl. 64 KiB chunk boundaries, levels 0–9) decoded by both native xz and our own decoder.
• Cross-checked output decodes under native xz for text/zeros/random/source.

🤖 Generated with Claude Code