feat(chunked-prefill): add chunked-prefill for multi gpu and mac#429

Open

wasamtc wants to merge 60 commits intoGradientHQ:mainfrom

wasamtc:tangcong/mac_chunk_v3

Commits on Jan 14, 2026

Disable Docker image build workflow
wasamtc
authored

Commits on Jan 16, 2026

Merge branch 'GradientHQ:main' into main
wasamtc
authored

Commits on Jan 17, 2026

Merge branch 'GradientHQ:main' into main
wasamtc
authored

Commits on Jan 19, 2026

Commits on Jan 21, 2026

Merge branch 'GradientHQ:main' into main
wasamtc
authored

Commits on Jan 23, 2026

Merge branch 'GradientHQ:main' into main
wasamtc
authored

Commits on Jan 29, 2026

base gpu version:single node can run successly
wasamtc
committed

Commits on Jan 30, 2026

fix[chunked-prefill]:reordered list use origin list
wasamtc
committed
fix[chunked-prefill]:transfer all chunks instead of last chunk
wasamtc
committed
fix[chunked-prefill]:restore last chunk
wasamtc
committed

Commits on Jan 31, 2026

Commits on Feb 1, 2026

fix[chunked-prefill]: add log
wasamtc
committed
fix[chunked-prefill]: change last peer just send to_forward_reqs
wasamtc
committed
fix[chunked-prefill]: change message type bug
wasamtc
committed
fix[chunked-prefill]: last send forward_reqs when forward_reqs is not none
wasamtc
committed
fix[chunked-prefill]: last send forward_reqs when forward_reqs is not none
wasamtc
committed
fix[chunked-prefill]: handle_reqs should do other if not last_peer
wasamtc
committed
fix[chunked-prefill]: add logs
wasamtc
committed
fix[chunked-prefill]: add logs
wasamtc
committed
fix[chunked-prefill]: add logs
wasamtc
committed
fix[chunked-prefill]: add logs
wasamtc
committed
fix[chunked-prefill]: add logs
wasamtc
committed
fix[chunked-prefill]: add logs
wasamtc
committed
fix[chunked-prefill]: add logs
wasamtc
committed
fix[chunked-prefill]: if req in running_batch, change status
wasamtc
committed
fix[chunked-prefill]: add logs
wasamtc
committed
fix[chunked-prefill]: admit req update if req exists
wasamtc
committed
fix[chunked-prefill]: add param for handle-input-reqs
wasamtc
committed
fix[chunked-prefill]: every req can admit once per admit
wasamtc
committed

Commits on Feb 2, 2026

Commits on Feb 3, 2026

fix[chunked-prefill]: get matched tokens before chunked
wasamtc
committed
fix[chunked-prefill]: use effective total_len if it exits
wasamtc
committed
fix[chunked-prefill]: use new reqs instead of old chunked_reqs
wasamtc
committed
feat[chunked-prefill]: the max chunked-prefill
wasamtc
committed
fix[prefix-cache]: use correct hidden_states
wasamtc
committed
fix[chunked-prefill]: delete kig
wasamtc
committed

Commits on Feb 4, 2026

Merge branch 'GradientHQ:main' into main
wasamtc
authored
Merge branch 'main' into tangcong/mac_chunk_v3
wasamtc
committed
fix[chunked-prefill]: fix format
wasamtc
committed

Commits on Feb 9, 2026

delete comments and add verify for chunked-prfill param
wasamtc
committed