Bug: Local LLMs that don't support tool calling cause backend crash (e.g., deepseek-r1)

[androemeda]

Problem Description

The SETUP.md documentation recommends deepseek-r1:8b and deepseek-r1:1.5b as local chat models for Ollama. However, these models do not currently support native tool/function calling via the Ollama API.

Because the backend relies on tool calling to interact with the VCell API and search the knowledge base, using these models causes a hard crash (400 Bad Request) the moment a user attempts to send a chat message.

---

Steps to Reproduce

1. Set up the local backend with Ollama as per SETUP.md.

2. Pull and configure one of the originally recommended models in .env:

PROVIDER=local
AZURE_DEPLOYMENT_NAME=deepseek-r1:1.5b

3. Start the backend and frontend.

4. Send any message in the chat interface (e.g., "List all Calcium models").

---

Expected Behavior

The backend should either:

• Process the query successfully using tools, or
• If the model does not support tools, gracefully fall back to a mechanism that allows the user to continue chatting, even if with limited capabilities, instead of crashing.

---

Actual Behavior

The backend fails with an unhandled exception and returns a 500 Internal Server Error to the frontend.

Backend logs show an error similar to:

Error code: 400 - {
  "error": {
    "message": "registry.ollama.ai/library/deepseek-r1:1.5b does not support tools",
    "type": "api_error"
  }
}

---

Proposed Fix

Refactor llms_service.py to add a graceful fallback mechanism.

Implementation Outline

• Wrap the native tool calling API in a try...except block.
• If the model returns a 400 error indicating that tools are not supported:
  • Log a warning
  • Fall back to a prompt-based tool selection approach.

Fallback Workflow

1. Ask the LLM to output a structured JSON indicating which tool to call.
2. Execute the selected tool.
3. Ask the LLM to generate a final response using the tool output.

Code Organization

Move the newly created TOOL_SELECTION_PROMPT into a dedicated file:

app/utils/tool_selection_prompt.py

to follow existing project patterns.

Documentation Update

Update SETUP.md recommendations:

• Change the primary recommendation to models that natively support tool calling (e.g., llama3.1:8b).
• List deepseek-r1 models as alternative options, clearly noting:
  • Their limitation (no native tool calling)
  • That the backend will automatically use the fallback mechanism.

[androemeda]

@vcellmike I’ve opened a PR (#46 ) to fix this issue. Could you please take a look when you have time? Happy to make any changes if needed !

[manideep1428]

@androemeda can you share where we apply the Gsoc for virtual cell

I want to do commit I know only 5 days can share link that introduce my self and appy for it