LLM Loading Failure — AttributeError in LlamaModel.__del__

[2P2O5]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [√] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• [√] I carefully followed the README.md.
• [√] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• [√] I reviewed the Discussions, and have a new bug or useful enhancement to share.

Summary

• Problem: When model loading fails (triggering a ValueError), the destructor LlamaModel.__del__ throws an AttributeError: 'LlamaModel' object has no attribute 'sampler', which obscures the original error message and introduces an extra exception.

Expected Behavior

• When model loading fails, the code should throw and display only ValueError: Failed to load model from file: <path>, without throwing an AttributeError during object destruction or cleanup.

Current Behavior

• Running python main.py produces the following stack trace:

  Traceback (most recent call last):
    File "/home/dhl/L/main.py", line 22, in <module>
      llm = Llama(
            ^^^^^^
    File "/home/dhl/L/.venv/lib/python3.11/site-packages/llama_cpp/llama.py", line 374, in __init__
      internals.LlamaModel(
    File "/home/dhl/L/.venv/lib/python3.11/site-packages/llama_cpp/_internals.py", line 58, in __init__
      raise ValueError(f"Failed to load model from file: {path_model}")
  ValueError: Failed to load model from file: /home/dhl/L/model/Qwen3VL-2B-Instruct-Q8_0.gguf
  
  Exception ignored in: <function LlamaModel.__del__ at 0x7fa17345bf60>
  Traceback (most recent call last):
    File "/home/dhl/L/.venv/lib/python3.11/site-packages/llama_cpp/_internals.py", line 86, in __del__
      self.close()
    File "/home/dhl/L/.venv/lib/python3.11/site-packages/llama_cpp/_internals.py", line 78, in close
      if self.sampler is not None:
         ^^^^^^^^^^^^
  AttributeError: 'LlamaModel' object has no attribute 'sampler'
  

Expected-only ValueError (Correct Behavior After Fix)

Traceback (most recent call last):
  File "/home/dhl/L/main.py", line 22, in <module>
    llm = Llama(
          ^^^^^^
  File "/home/dhl/L/.venv/lib/python3.11/site-packages/llama_cpp/llama.py", line 374, in __init__
    internals.LlamaModel(
  File "/home/dhl/L/.venv/lib/python3.11/site-packages/llama_cpp/_internals.py", line 58, in __init__
    raise ValueError(f"Failed to load model from file: {path_model}")
ValueError: Failed to load model from file: /home/dhl/L/model/Qwen3VL-2B-Instruct-Q8_0.gguf

What I Changed & Tested

• I added an existence check to the cleanup logic in LlamaModel.close() / LlamaModel.__del__() to avoid accessing the missing sampler attribute when the object is not fully initialized. Modified file:

  • llama_cpp/_internals.py

• Core check I modified (locally tested and validated):

  Original code snippet:

  def close(self):
      if self.sampler is not None:
          ...

  Modified code (resolves AttributeError):

  def close(self):
      if hasattr(self, "sampler") and self.sampler is not None: 
         ...

Steps to Reproduce

1. Execute in the root directory of the repository:

   python main.py

2. If the model file path is invalid or model loading fails (e.g., using the provided model file model/Qwen3VL-2B-Instruct-Q8_0.gguf), you will see the ValueError above, followed by the AttributeError thrown during destruction.

Environment and Context

• Operating System: Linux (collected outputs from local machine below)

  Commands executed:

  lscpu
  uname -a
  python3 --version
  pip list | egrep "numpy|llama-cpp-python|llama_cpp"

  Collected output:

  Architecture:                    x86_64
  CPU op-mode(s):                  32-bit, 64-bit
  Address sizes:                   48 bits physical, 48 bits virtual
  Byte Order:                      Little Endian
  CPU(s):                          4
  Online CPU(s) list:              0-3
  Vendor ID:                       AuthenticAMD
  Model name:                      AMD FX-9830P RADEON R7, 12 COMPUTE CORES 4C+8G
  CPU family:                      21
  Model:                           101
  Thread(s) per core:              2
  Cores per socket:                2
  Socket(s):                       1
  Stepping:                        1
  Frequency boost:                 enabled
  CPU max MHz:                     3000.0000
  CPU min MHz:                     1400.0000
  BogoMIPS:                        5999.86
  Flags:                           ...
  Virtualization features:         AMD-V
  L1d:                             128 KiB (4 instances)
  L1i:                             192 KiB (2 instances)
  L2:                              2 MiB (2 instances)
  NUMA nodes:                      1
  NUMA node0 CPU(s):               0-3
  Vulnerabilities:                 ...
  ---
  Linux dhl 6.1.0-35-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.137-1 (2025-05-07) x86_64 GNU/Linux
  ---
  Python 3.11.2
  ---
  (No matching packages found by the pip filter; `numpy`, `llama-cpp-python`, or `llama_cpp` not listed)
  

  Note: I can run pip list without filtering to provide the full package list if needed.

• Library file where the issue occurs: .venv/lib/python3.11/site-packages/llama_cpp/_internals.py

• Location of test execution: main.py (repository root)

• Model path used: model/Qwen3VL-2B-Instruct-Q8_0.gguf

Failure Logs

• The primary error (ValueError) and the AttributeError during destruction are pasted in the "Current Behavior" section above.

Diagnosis / Root Cause

• When LlamaModel.__init__ throws a ValueError after model loading failure, the object is still garbage-collected and __del__ is called.
• Since the object is not fully initialized (attributes like self.sampler / self.custom_samplers may not be assigned), directly accessing self.sampler in __del__ throws an AttributeError.
• The solution is to add an existence check (hasattr(self, 'sampler')) for optional attributes in close() / __del__, or ensure these attributes are initialized even in error paths of the constructor.

Suggested Fixes

1. Ensure all attributes accessed by __del__ or close are initialized to default values (e.g., self.sampler = None; self.custom_samplers = []) in LlamaModel.__init__ and LlamaContext.__init__; OR
2. Add hasattr checks in close() / __del__() (the approach I used locally and validated).

Additional Notes

• I can submit a small PR (or patch snippet) to ensure these attributes are initialized even in error paths. Please let me know if you'd like me to formalize my changes into a PR.