Bug with some LLMs #93
Description
The new 2.0.3 version is really great! Before, I was getting around 3 t/s for the Qwen3.5 2b model. Now I'm getting 5.2 t/s with the same model. Thanks to your optimizations, there's a significant increase in speed and performance !
But, I was also able to load the Gemma 2 2b parameters one. Now I can't load it, the app crashes if I try.
@Siddhesh2377