Skip to content

fix(ruvllm): detect tied word embeddings for Llama 3.2 in candle backend#645

Open
ohdearquant wants to merge 1 commit into
ruvnet:mainfrom
ohdearquant:fix/candle-llama-tie-embeddings
Open

fix(ruvllm): detect tied word embeddings for Llama 3.2 in candle backend#645
ohdearquant wants to merge 1 commit into
ruvnet:mainfrom
ohdearquant:fix/candle-llama-tie-embeddings

Conversation

@ohdearquant

Copy link
Copy Markdown
Contributor

Problem

Llama 3.2 1B/3B tie the output projection to the input embeddings — the checkpoint ships no separate lm_head.weight tensor. The candle backend hardcoded tie_word_embeddings: false, so loading a Llama 3.2 checkpoint fails on the missing lm_head.weight.

Fix

Detect the tie from tensor presence instead of hardcoding:

let tie_word_embeddings = !vb.contains_tensor("lm_head.weight");

Models that ship an explicit lm_head.weight are unaffected (detected as untied, same as before); Llama 3.2 1B/3B now load correctly.

Scope

6 lines, candle_backend.rs only, behind the candle feature. No new dependencies.

Verification

cargo check -p ruvllm --features candle passes clean (candle-transformers 0.9.2).

🤖 Generated with Claude Code

Llama 3.2 1B/3B tie the output projection to the input embeddings, so the
checkpoint has no separate lm_head.weight tensor. The candle backend hardcoded
tie_word_embeddings: false, which fails to load these models. Detect the tie
from tensor presence instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant