Throughput optimizations across the codec suite#90

Merged

MagicalTux merged 32 commits into

masterfrom

optimize-codecs

Jun 12, 2026

Commits on Jun 12, 2026

checksum: CRC-32 slice-by-8 (642 -> 2525 MB/s, 3.9x)

MagicalTux
and
claude
committed
rle90: bulk-copy literal runs in decoder (~1268 -> ~4600 MB/s, 3.5x)

MagicalTux
and
claude
committed
deflate: vectorize decoder match-copy incl. overlapping runs

MagicalTux
and
claude
committed
deflate: replace per-literal modulo with a wrap branch in emit_byte

MagicalTux
and
claude
committed
deflate64: vectorize decoder match-copy incl. overlapping runs

MagicalTux
and
claude
committed
lzma2: bulk match-copy in decode_chunk (xz/lzma2 decode)

MagicalTux
and
claude
committed
lzma2: bulk overlapping match-copy in decode_chunk (xz/lzma2 decode)

MagicalTux
and
claude
committed
lzma: bulk overlapping match-copy in decoder drain loops (.lzma decode)

MagicalTux
and
claude
committed
brotli: keep bit accumulator across Huffman LUT hits

MagicalTux
and
claude
committed
brotli: skip literal context lookup when there is a single tree

MagicalTux
and
claude
committed
brotli: widen Huffman fast-path LUT from 9 to 11 bits

MagicalTux
and
claude
committed
zstd: faster Huffman literal decode via peek/consume

MagicalTux
and
claude
committed
zstd: skip zero-bit reads and inline FSE state transitions

MagicalTux
and
claude
committed
zstd: inline RevBitReader::read fast path, split wide reads out of line

MagicalTux
and
claude
committed
zstd: hoist LL/ML base+extra tables to module-level const

MagicalTux
and
claude
committed
zstd: fetch each FSE entry once per sequence (symbol + advance share load)

MagicalTux
and
claude
committed
perf(decoders): bulk overlapping match copy in lz4/lz5/lzo/snappy

MagicalTux
and
claude
committed
perf(lzw): single-pass string emit, drop scratch stack

MagicalTux
and
claude
committed
perf(lzo): skip-step accelerator in encoder match search

MagicalTux
and
claude
committed
perf(snappy): skip-step accelerator in encoder match search

MagicalTux
and
claude
committed
bzip2: cut SA-IS allocations and inline induced-sort hot paths

MagicalTux
and
claude
committed
bzip2: recurse SA-IS reduced problem in place (drop per-level copy)

MagicalTux
and
claude
committed
xpress_huffman: amortize decoder history trim (O(n²) → O(n))

MagicalTux
and
claude
committed
lznt1: bulk copy_within for non-overlapping match copies

MagicalTux
and
claude
committed
hpack: byte-wide FSA Huffman decoder

MagicalTux
and
claude
committed
arc_crunch: single-write LZW string assembly + literal fast path

MagicalTux
and
claude
committed
arc_squash: single-write LZW string assembly + literal fast path

MagicalTux
and
claude
committed
delta: vectorizable filter loop via direct predecessor indexing

MagicalTux
and
claude
committed
lha: bulk match-copy in static-Huffman decode hot loop

MagicalTux
and
claude
committed
rar1/2/3/5: bulk LZ77 match-copy in decode window loops

MagicalTux
and
claude
committed
zip_implode/reduce/shrink: bulk match-copy in decode loops

MagicalTux
and
claude
committed
docs: changelog entry for codec throughput optimizations

MagicalTux
and
claude
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Throughput optimizations across the codec suite#90

Throughput optimizations across the codec suite#90
MagicalTux merged 32 commits into
masterfrom
optimize-codecs

Commits on Jun 12, 2026

checksum: CRC-32 slice-by-8 (642 -> 2525 MB/s, 3.9x)

rle90: bulk-copy literal runs in decoder (~1268 -> ~4600 MB/s, 3.5x)

deflate: vectorize decoder match-copy incl. overlapping runs

deflate: replace per-literal modulo with a wrap branch in emit_byte

deflate64: vectorize decoder match-copy incl. overlapping runs

lzma2: bulk match-copy in decode_chunk (xz/lzma2 decode)

lzma2: bulk overlapping match-copy in decode_chunk (xz/lzma2 decode)

lzma: bulk overlapping match-copy in decoder drain loops (.lzma decode)

brotli: keep bit accumulator across Huffman LUT hits

brotli: skip literal context lookup when there is a single tree

brotli: widen Huffman fast-path LUT from 9 to 11 bits

zstd: faster Huffman literal decode via peek/consume

zstd: skip zero-bit reads and inline FSE state transitions

zstd: inline RevBitReader::read fast path, split wide reads out of line

zstd: hoist LL/ML base+extra tables to module-level const

zstd: fetch each FSE entry once per sequence (symbol + advance share load)

perf(decoders): bulk overlapping match copy in lz4/lz5/lzo/snappy

perf(lzw): single-pass string emit, drop scratch stack

perf(lzo): skip-step accelerator in encoder match search

perf(snappy): skip-step accelerator in encoder match search

bzip2: cut SA-IS allocations and inline induced-sort hot paths

bzip2: recurse SA-IS reduced problem in place (drop per-level copy)

xpress_huffman: amortize decoder history trim (O(n²) → O(n))

lznt1: bulk copy_within for non-overlapping match copies

hpack: byte-wide FSA Huffman decoder

arc_crunch: single-write LZW string assembly + literal fast path

arc_squash: single-write LZW string assembly + literal fast path

delta: vectorizable filter loop via direct predecessor indexing

lha: bulk match-copy in static-Huffman decode hot loop

rar1/2/3/5: bulk LZ77 match-copy in decode window loops

zip_implode/reduce/shrink: bulk match-copy in decode loops

docs: changelog entry for codec throughput optimizations