[FEATURE] TurboQuant support

[SirDank]

References:

• Feature Request: TurboQuant support ggml-org/llama.cpp#20977
• https://github.com/TheTom/turboquant_plus
• ggml : add CPU TurboQuant KV cache types (TBQ3_0 / TBQ4_0) ggml-org/llama.cpp#21089

[animehacker]

I've been working on extending unixsysdev's tq3_0 implementation with V cache support and flash attention. Repo here: https://github.com/animehacker/llama-turboquant

What this adds on top of unixsysdev's work:

• Normalization fix (1/32 → 1/√32 for the asymmetric K-side WHT)
• V cache compression (non-transposed storage + graph-side dequant to work around GGML's element-scatter path)
• Flash attention with tq3_0 (dequant tq3_0 → F32 → F16 in the attention graph, then use existing FA kernel)
• CPU backend F32 dequant path for pipeline parallelism

Tested on Llama-3.3-70B-Instruct-Q4_K_M, 2x RTX 3090:

• 72K context with tq3_0 K+V (4.57x compression)
• WikiText-2 PPL: 4.40 vs 4.09 baseline (+7.6%)

To be clear: this implements PolarQuant (Stage 1) only — WHT rotation + 3-bit Lloyd-Max. QJL residual correction is not included.

Paper with implementation details: https://oliverchurch.com/turboquant-for-ggml-achieving-4.57x-kv-cache-compression-in-llama.cpp.html

[leehack]

I'm definitely interested in supporting and keeping an eye on the llama.cpp project. Once TQ support becomes available in llama.cpp, it will be straightforward to enable it in llamadart.