perf(inference): broader Candle inference deletion (deferred from #1273) — audit plasticity LoRA reachability first

[joelteply]

Follow-on to #1273 (Candle Qwen3.5 deletion, PR #1279) — broader Candle inference cleanup deferred from #1273's original scope.

PR #1279 deleted the Qwen3.5-specific Candle path (Qwen35GgufBackend + vendored quantized_qwen35), but the broader Candle inference chain remains:

• inference/candle_adapter.rs (~1500 LOC) — CandleAdapter is imported but never instantiated by AIProviderModule::register_adapters. Dead in production.
• inference/model.rs (~900 LOC) — ContinuumModel has no callers outside the inference module.
• inference/quantized.rs (~300 LOC) — load_quantized_model, load_default_quantized only called by candle_adapter.rs.
• inference/backends/{generate, load_gguf_backend} — only called by the dead chain above + bin/* debug binaries.
• inference/vendored/{quantized_llama, qwen2, compact_llama}.rs — Candle backends not used by any registered adapter.
• inference/backends/{llama_gguf, llama_safetensors, qwen2_safetensors}.rs — used only by the dead chain or by plasticity tests.
• bin/inference_test.rs, bin/test_qwen_gguf.rs, bin/diagnose_prefill.rs, bin/mixed_quant.rs — not exposed as Cargo binaries; orphaned source files.

Why deferred from #1273

Initial #1273 plan was a single atomic deletion. cargo check after deleting model.rs revealed backends/llama_safetensors.rs:20 uses model::rebuild_with_stacked_lora, and modules/plasticity/validation.rs:766, :802 uses compact_llama_safetensors::CompactLlamaSafetensorsBackend — these are test code (#[test] blocks), but they're plasticity tests for LoRA training infrastructure. Whether plasticity LoRA training is itself a live production path or a vestigial training surface is the question that determines the deletion blast radius.

Acceptance

1. Audit plasticity LoRA training reachability. Is crate::modules::plasticity exercised by any live IPC handler / CLI command / chat hot path? Or is it test-only / experimental infrastructure that's never invoked in production?
2. If plasticity is dead in production: delete the entire Candle chain — candle_adapter.rs, ContinuumModel (and model.rs if everything in it depends on it), quantized.rs, qwen2/llama vendored backends, llama_gguf/llama_safetensors/qwen2_safetensors backends, and the orphaned bin/* files. ~4000+ LOC.
3. If plasticity is live in production: scope deletion to only the chain that's truly unreachable — likely candle_adapter.rs + quantized.rs (since plasticity uses safetensors backends directly, not via CandleAdapter). Document which Candle infrastructure stays and why.

Why this matters

Per Joel's "if not UI/UX it is rust" rule and the no-CPU-fallback alpha contract, dead Candle infrastructure that contradicts these rules on paper but never executes is the same anti-pattern this audit (#1262) was filed for. The deferred scope is the bulk of the dead-code mass.

Lane: alpha flywheel #1272 lane 6 (continues #1262 audit work).

[joelteply]

Plasticity reachability audit complete — broader Candle deletion is SAFE

TL;DR: the entire Candle inference chain (CandleAdapter, ContinuumModel, quantized.rs, the GGUF/safetensors backends, the vendored compact_llama/quantized_llama/qwen2 modules, the orphaned bin/* files) is dead in production. Plasticity's IPC handlers do NOT call into Candle inference — they work on safetensors/GGUF files directly via plasticity's own helpers (compactor::*, pipeline::*). Only plasticity TESTS pull in the Candle backends.

Method

Greps from canary tip:

1. Plasticity IPC reachability (production):

   • ipc/mod.rs:967 registers PlasticityModule::new() ✓ production
   • 5 commands exposed: plasticity/{analyze,compact,compress,topology,pipeline}
   • Plasticity mod.rs has ZERO imports from crate::inference::*
   • handle_compact works on safetensors files via compactor::compact_model[_sharded]
   • handle_compress works via pipeline::compress
   • None of the IPC handlers call LlamaSafetensorsBackend, CompactLlamaSafetensorsBackend, ContinuumModel, or CandleAdapter.

2. Plasticity test-only Candle usage:

   • modules/plasticity/validation.rs:19 use crate::inference::vendored::compact_llama::{detect_topology, CompactLlama} is a top-level import, but the imports are only used inside the #[cfg(test)] mod tests block at line 210+.
   • modules/plasticity/validation.rs:766, :802 — CompactLlamaSafetensorsBackend references are inside #[test] fn bodies.
   • These tests can be deleted alongside the backends they test (they exercise dead-in-production code).

3. CandleAdapter, ContinuumModel reachability (re-confirmed from perf(inference): verify + delete Candle Qwen3.5/Qwen2/llama path if vestigial post-llama.cpp #1273 verification):

   • CandleAdapter imported in ai_provider.rs:23 but never instantiated by register_adapters.
   • ContinuumModel has zero external callers.
   • backends::load_gguf_backend / backends::generate only called by the dead chain above + orphaned bin/* (not exposed as Cargo [[bin]]).

Proposed phased deletion plan for #1280

Keeping each phase atomic at the cargo-build boundary so reviewers see one logical step per PR:

Phase 1 — Delete adapter + model lifecycle. ~2500 LOC.

• inference/candle_adapter.rs (~1500 LOC) — CandleAdapter + LlamaCppCandleWrapper
• inference/model.rs (~900 LOC) — ContinuumModel, select_best_device, load_model_by_id, rebuild_with_stacked_lora
• inference/quantized.rs (~300 LOC) — load_quantized_model, load_default_quantized
• Drop the pub use re-exports in inference/mod.rs and ai/mod.rs:44
• Drop the CandleAdapter import in modules/ai_provider.rs:23
• Drop doc-comment references in http/mod.rs:10, :201 and modules/inference.rs:62
• Caveat: select_best_device is invoked by the no-CPU-fallback regression test (tests/no_cpu_fallback_contract.rs — shipped test(inference,#1275): regression test for no-CPU-fallback alpha contract #1282). The test asserts the panic message is in the source. After deletion, the loud-fail must move somewhere still reachable — likely the LlamaCppAdapter boot path — and the test must be updated.

Phase 2 — Delete GGUF/safetensors backends + vendored Candle modules. ~2000+ LOC.

• inference/backends/{generate, load_gguf_backend} and the LlamaGgufBackend, LlamaSafetensorsBackend, Qwen2SafetensorsBackend, CompactLlamaSafetensorsBackend files (Phase 1 made these dead)
• inference/vendored/{compact_llama, quantized_llama, qwen2}.rs (only consumed by the deleted backends)
• The [#cfg(test)] mod tests blocks in modules/plasticity/validation.rs that reference the deleted backends + the top-level use for compact_llama::{detect_topology, CompactLlama} (which becomes unused)
• Helper functions in validation.rs that are themselves only test scaffolding (build_synthetic_compact_weights etc.) — keep if used by surviving plasticity tests, delete otherwise
• Verify plasticity/{analyze,compact,compress,topology,pipeline} IPC commands still pass their non-Candle tests

Phase 3 — Delete orphaned bin/* files. ~500 LOC.

• bin/inference_test.rs, bin/test_qwen_gguf.rs, bin/diagnose_prefill.rs, bin/mixed_quant.rs — not exposed as Cargo [[bin]] (only dequantize-gguf, vrm-convert-textures, vrm-inspect are). After Phase 2 removes their backends::load_gguf_backend callees, these files won't even compile.

Why this matters

This is the bulk of dead Candle infrastructure mass that #1262's audit identified. Deleting it:

• Removes the ambiguity about which inference path is live (only LlamaCppAdapter remains)
• Eliminates ~5000 LOC of code that contradicts the no-CPU-fallback contract on paper but never executes
• Reduces compile time for the lib crate (the vendored Candle backends are non-trivial)
• Aligns the codebase with Joel's "if not UI/UX it is rust, but only the LIVE paths matter" lens

Risk

The select_best_device panic in inference/model.rs:82 is referenced by the no-CPU-fallback regression test (#1282). Phase 1 must move that loud-fail to a still-reachable site (likely LlamaCppBackend::load or LlamaCppAdapter::ensure_loaded) and update the test to assert the new location. Without that, Phase 1 deletion silently weakens the contract this whole audit was filed to enforce — which would be the same anti-pattern in reverse.

Status

Audit complete; not claiming Phase 1 PR yet — large enough scope (~2500 LOC across 3 files + 6 cross-references) that I want a peer review on the phasing before kicking off. Releasing #1280 for whoever has the cycles to take Phase 1 first.