Agent chat fails with Gemini: artifacts tool schema has invalid array type without items

[mj-deving]

Bug

The Chrome extension agent chat (and /v1/agent API) fails with a 400 Bad Request when using Gemini models with automation enabled.

Error

GenerateContentRequest.tools[0].function_declarations[5].parameters.properties[content].any_of[2].items: missing field

Root Cause

In dist/esm/daemon/agent.js, the artifacts tool definition has:

content: {
    description: "Content to store (string or JSON-serializable object)",
    type: ["string", "object", "array", "number", "boolean", "null"],
},

Gemini's API internally converts type: [...] arrays into an anyOf schema. The "array" variant (index 2) requires an items field to be valid, but none is provided. This causes the 400 error.

Other providers (OpenAI, Anthropic) are more lenient and accept the union type without items, so this only affects Gemini.

Reproduction

curl -s -X POST "http://127.0.0.1:8787/v1/agent" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "automationEnabled": true,
    "messages": [{"role": "user", "content": "Hello"}],
    "model": "google/gemini-2.0-flash",
    "tools": ["summarize", "navigate", "repl", "ask_user_which_element", "skill", "artifacts"]
  }'

Workaround

Change the content property type in the artifacts tool definition to a simple string:

content: {
    description: "Content to store (string or JSON-serializable object). For arrays or complex objects, pass as a JSON string.",
    type: "string",
},

Environment

• summarize v0.11.1
• Windows 11
• Model: google/gemini-2.0-flash
• Chrome extension with automation enabled

🤖 Generated with Claude Code

[steipete]

$1

[steipete]

Confirmed on current main: this is still valid.

The artifacts tool schema in src/daemon/agent.ts still declares:

• content.type = ["string", "object", "array", "number", "boolean", "null"]

with no items schema for the array branch. That matches the Gemini error you reported.

Current tests only verify that the artifacts tool exists, not that its schema is Gemini-safe:

• tests/daemon.agent.test.ts

Fix path:

1. safest: make content a plain string and require JSON-string payloads for structured content
2. or: switch to an explicit Gemini-safe union schema and give the array branch items: {}
3. add a regression test so this schema stays Gemini-compatible

So: keep open.

[steipete]

Fixed on main in abb711c (fix: make daemon artifacts schema gemini-safe).

What changed:

• daemon agent artifacts.content now uses a Gemini-safe string schema instead of an invalid array shape
• regression coverage added for the daemon agent path
• verified with a live Gemini smoke using keys from .profile

Tests run:

• pnpm -s vitest run tests/daemon.agent.test.ts
• live Gemini daemon/artifacts smoke

Closing as fixed.