Does the current implementation support prefix/encoder cache?

[Syd-Q]

I tested the current implementation and it seems that prefix cache is not yet supported. When I input the same image and prompt multiple times, the inference speed remains the same each time — neither the image nor the prompt appears to be reused.

Do you have plans to add prefix/encoder cache functionality (similar to what vLLM offers) in the future?

[nvluxiaoz]

We plan to extend such functionality but it may not reach vLLM/sgLang parity soon because of the complexity of such KV Cache management system. We support system prompt KV Cache Reuse at this time.