Unified KV Cache Compression Methods for Auto-Regressive Models
-
Updated
Jan 4, 2025 - Python
#
kv-cache-compression
Here are 26 public repositories matching this topic...
LLM KV cache compression made easy
python transformers inference pytorch kv-cache large-language-models llm long-context kv-cache-compression
-
Updated
Apr 14, 2026 - Python
Awesome-LLM-KV-Cache: A curated list of 📙Awesome LLM KV Cache Papers with Codes.
-
Updated
Mar 3, 2025
[NeurIPS'25 Oral] Query-agnostic KV cache eviction: 3–4× reduction in memory and 2× decrease in latency (Qwen3/2.5, Gemma3, LLaMA3)
-
Updated
Feb 11, 2026 - Python
Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)
-
Updated
Apr 13, 2025 - Python
[ICLR 2025] Palu: Compressing KV-Cache with Low-Rank Projection
-
Updated
Feb 20, 2025 - Python
Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)
-
Updated
Apr 18, 2024 - Python
This is the official repo of "QuickLLaMA: Query-aware Inference Acceleration for Large Language Models"
-
Updated
Jul 16, 2024 - Python
First open-source implementation of Google TurboQuant (ICLR 2026) -- near-optimal KV cache compression for LLM inference. 5x compression with near-zero quality loss.
machine-learning compression deep-learning pytorch transformer attention quantization iclr vector-quantization memory-optimization kv-cache google-research llm vllm llm-inference kv-cache-compression
-
Updated
Apr 17, 2026 - Python
xKV: Cross-Layer SVD for KV-Cache Compression
-
Updated
Nov 30, 2025 - Python
(ACL2025 oral) SCOPE: Optimizing KV Cache Compression in Long-context Generation
-
Updated
May 28, 2025 - Jupyter Notebook
Accurate and fast KV cache compression with a gating mechanism
-
Updated
Apr 5, 2026 - Python
First open-source KVTC implementation (NVIDIA, ICLR 2026) -- 8-32x KV cache compression via PCA + adaptive quantization + entropy coding
compression pytorch nvidia transformer pca attention dynamic-programming quantization deflate entropy-coding memory-optimization kv-cache llm llm-inference kv-cache-compression iclr-2026
-
Updated
Apr 17, 2026 - Python
LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation
-
Updated
Sep 17, 2025 - Python
Native Windows build of vLLM 0.19.0 — no WSL, no Docker. Pre-built wheels + 33-file Windows patch + Multi-TurboQuant KV cache compression (6 methods, 2x cache capacity). PyTorch 2.10 + CUDA 12.6 + Triton + Flash-Attention 2.
windows gpu cuda pytorch nvidia triton msvc quantization kv-cache awq llm llm-serving vllm llm-inference flash-attention qwen kv-cache-compression turboquant vllm-windows multi-turboquant
-
Updated
Apr 12, 2026 - Python
AI agent skill implementing Google's TurboQuant compression algorithm (ICLR 2026) — 6x KV cache memory reduction, 8x speedup, zero accuracy loss. Compatible with Claude Code, Codex CLI, and all Agent Skills-compatible tools.
python deep-learning transformer quantization codex vector-quantization model-compression inference-optimization kv-cache ai-agent llm llm-compression kv-cache-compression claude-code coding-agent iclr-2026 agent-skill turboquant polarquant qjl
-
Updated
Mar 28, 2026 - Python
Near-optimal vector quantization for LLM KV cache compression. Python implementation of TurboQuant (ICLR 2026) — PolarQuant + QJL for 3-bit quantization with minimal accuracy loss and up to 8x memory reduction.
python deep-learning numpy transformer attention quantization vector-quantization model-compression inference-optimization memory-optimization kv-cache post-training-quantization llm llm-inference llm-compression kv-cache-compression iclr-2026 turboquant polarquant qjl
-
Updated
Mar 28, 2026 - Python
Repository for the paper: https://arxiv.org/abs/2510.00231
-
Updated
Oct 6, 2025 - Python
Drop-in KV cache compression for MLX on Apple Silicon. Brings PolarQuant (Google, ICLR 2026) to mlx-lm with first-class Gemma 4 support: MatFormer, dual head_dim, hybrid sliding/global attention, cross-layer KV sharing. 3-bit → 4.8× smaller cache, 0.995 logit cosine @ 4-bit.
memory-efficient gemma mlx on-device-ai kv-cache apple-silicon llm kv-cache-compression mlx-lm turboquant polarquant gemma-4
-
Updated
Apr 16, 2026 - Python
Test Google's new TurboQuant KV-cache compression (ICLR 2026) on your local machine — measure real speed, memory, and accuracy differences across compression modes.
-
Updated
Apr 7, 2026 - Python
Improve this page
Add a description, image, and links to the kv-cache-compression topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the kv-cache-compression topic, visit your repo's landing page and select "manage topics."