feat(appkit): tools(plugins) DX, runAgent plugins arg, shared toolkit-resolver

[MarioCadenas]

DX centerpiece. Introduces a function-form tools field on
AgentDefinition that receives a plugins map, eliminating the need
to mention each plugin twice in user code. Extracts the shared
toolkit resolver that the agents plugin, auto-inherit, and standalone
runAgent all now go through. Refactors the agent runtime into
core/agent/ so peer plugins can depend on it without crossing the
sibling boundary.

tools(plugins) — the function form

AgentDefinition.tools accepts either a plain record (for agents
that only use inline tools) or a function (plugins) => Record<string, AgentTool> that receives a plugins map and returns a
tool record. The function runs once at agent setup; the result is
cached as the agent's resolved tool record.

const support = createAgent({
  instructions: "You help customers.",
  model: "databricks-claude-sonnet-4-5",
  tools(plugins) {
    return {
      ...plugins.analytics.toolkit(),
      ...plugins.files.toolkit({ only: ["uploads.read"] }),
      get_weather: tool({
        name: "get_weather",
        description: "Weather",
        schema: z.object({ city: z.string() }),
        execute: async ({ city }) => `Sunny in ${city}`,
      }),
    };
  },
});

await createApp({
  plugins: [server(), analytics(), files(), agents({ agents: { support } })],
});

Each plugin is mentioned exactly once, in createApp({ plugins: [...] }). No held variables, no spread markers, no fromPlugin
indirection.

resolveToolkitFromProvider(pluginName, provider, opts?)

packages/appkit/src/core/agent/toolkit-resolver.ts. Single source
of truth for "turn a ToolProvider into a keyed record of
ToolkitEntry markers". Prefers provider.toolkit(opts) when
available (core plugins implement it), falls back to walking
getAgentTools() and synthesizing namespaced keys
(${pluginName}.${localName}) for third-party providers, honoring
only / except / rename / prefix the same way.

Used by three call sites, previously all copy-pasted:

1. AgentsPlugin.buildToolIndex — function-form resolution pass
2. AgentsPlugin.applyAutoInherit — markdown auto-inherit path
3. runAgent — standalone-mode plugin tool dispatch

The underlying filtering primitive applyToolkitOptions (in
core/agent/toolkit-options.ts) is also used by
buildToolkitEntries in core/agent/build-toolkit.ts.

runAgent gains plugins?: PluginData[]

packages/appkit/src/core/agent/run-agent.ts. When an agent def's
tools is the function form, runAgent builds a plugins map lazily
from RunAgentInput.plugins. Plugin instances are constructed on
the first plugins.<name>.toolkit(...) call and cached, so the same
instance handles both spread-time resolution and dispatch-time tool
execution. Hosted/MCP tools are rejected up front with a clear error
rather than failing mid-conversation when the adapter tries to
dispatch them — they require a live MCP client that only exists
inside the agents plugin's lifecycle.

Agent runtime relocated to core/agent/

The framework-level agent primitives (createAgent, runAgent,
tool helpers, types, system-prompt composition, markdown agent
loader) moved from plugins/agents/ to core/agent/. The
HTTP-facing agents() plugin in plugins/agents/ consumes these
but no longer owns them — peer plugins (analytics, files, genie,
lakebase) can depend on the runtime without reaching across the
sibling boundary.

Type plumbing

• AgentTools simplified to Record<string, AgentTool> (no symbol
  keys, no FromPluginMarker).
• AgentToolsFn = (plugins: Plugins) => AgentTools.
• AgentDefinition.tools?: AgentTools | AgentToolsFn.
• Plugins = Record<string, PluginToolkitProvider> — plain
  string-keyed map; users refer to plugins by the name they pass to
  createApp({ plugins: [...] }). Per-plugin keyed autocomplete on
  the plugins parameter is a separate, larger design problem
  tracked for follow-up.

DoS-limit hardening

• Concurrent-stream rate limit (maxConcurrentStreamsPerUser)
  enforced on both /chat and /invocations (previously only on
  /chat).
• countUserStreams now O(1) via a per-user counter map kept in
  sync with activeStreams through trackStream / untrackStream
  helpers.
• Tool-call budget and approval gate now travel through a shared
  RunState so sub-agent dispatches enforce the same limits as
  top-level calls.
• Per-tool-call timeout (limits.toolCallTimeoutMs, default 5min)
  combined with parent abort signal via AbortSignal.any.
• threadStore.{get,create,addMessage} failures in _handleChat
  and _handleInvocations return HTTP 500 instead of hanging the
  client connection.

Reload safety

AgentsPlugin.reload() builds a fresh registry first and only swaps
on success, so a malformed markdown file or missing tool reference
no longer leaves the live registry in a half-rebuilt state. Markdown
agent loader fs.* calls are now async to avoid blocking the event
loop during reload.

Exports

Function-form types (AgentToolsFn, Plugins,
PluginToolkitProvider) exported from @databricks/appkit/beta.
fromPlugin, FromPluginMarker, isFromPluginMarker,
FROM_PLUGIN_MARKER, FromPluginSpread, and the
NamedPluginFactory.pluginName field (its only consumer) are
removed — files deleted, callers migrated.

Test plan

• 2273 tests passing across the full vitest workspace.
• New tests cover: function-form spread of
  plugins.<name>.toolkit(), function-form opt-out from
  auto-inherit (object and empty-object forms), function form with
  provider lacking .toolkit(), function form invoked exactly once
  at setup, sub-agent recursive execution in standalone runAgent,
  hosted-tool rejection at index-build time, threadStore failure
  paths returning HTTP 500 (5 paths in route-handler-errors.test.ts),
  /invocations honouring maxConcurrentStreamsPerUser, and the
  userStreamCounts invariant across track/untrack lifecycles.
• Typecheck clean across all 7 workspace projects.
• Build + docs:build clean.

PR Stack

1. Shared agent types + LLM adapters — feat(appkit): shared agent types and LLM adapter implementations #301
2. Tool primitives + ToolProvider surfaces — feat(appkit): tool primitives and ToolProvider surfaces on core plugins #302
3. Plugin infrastructure (attachContext + PluginContext) — feat(appkit): plugin infrastructure — attachContext + PluginContext mediator #303
4. agents() plugin + createAgent(def) + markdown-driven agents — feat(appkit): agents() plugin, createAgent(def), and markdown-driven agents #304
5. tools(plugins) DX + runAgent plugins arg + toolkit-resolver (this PR)
6. Reference app + dev-playground + docs — feat(appkit): reference agent-app, dev-playground chat UI, docs, and template #306

Demo

agent-demo.mp4

[pkosiec]

Agentic review:

Fix Plan: Code Review Findings for agent/v2/5-fromplugin-runagent

Context

Code review of branch agent/v2/5-fromplugin-runagent vs agent/v2/4-agents-plugin identified 16 findings across 12 parallel reviewers (correctness, testing, maintainability, project-standards, agent-native, learnings-researcher, security, api-contract, reliability, adversarial, kieran-typescript, performance). This plan addresses all actionable findings in priority order.

P1 — High Priority Fixes

Fix 1–2: Wrap threadStore calls in route handlers with try-catch

Files:

• packages/appkit/src/plugins/agents/agents.ts — _handleChat (~line 700) and _handleInvocations (~line 740)

Problem: If threadStore.get(), .create(), or .addMessage() throws (e.g., DB unavailable), the async Express handler propagates uncaught and the client connection hangs with no HTTP response.

Fix:

• In _handleChat: wrap lines 702–718 (threadStore.get → threadStore.create → threadStore.addMessage) in a try-catch that returns res.status(500).json({ error: "Thread operation failed" })
• In _handleInvocations: wrap lines 740–764 (threadStore.create → for-loop of addMessage) in a try-catch that returns res.status(500).json({ error: "Thread operation failed" })

Fix 3: Early hosted-tool rejection in runAgent()

File: packages/appkit/src/core/agent/run-agent.ts — buildStandaloneToolIndex() (~line 144)

Problem: Hosted/MCP tools are included in the standalone tool index but error only at dispatch time. The adapter sees these tools, may try to use them, and gets a confusing mid-conversation error.

Fix: After the tool classification loop (after line 184), scan the index for kind: "hosted" entries. If any exist AND they weren't produced by toolkitEntryToStandalone (which already warns in the description), collect their names and throw:

runAgent: agent definition includes hosted/MCP tools [${names}] which are not supported in standalone mode. Use createApp({ plugins: [agents(...)] }) for MCP support.

Fix 4: Replace countUserStreams() O(n) scan with per-user counter

File: packages/appkit/src/plugins/agents/agents.ts

Problem: countUserStreams() (line 143) iterates all active streams on every chat request. At scale this is O(n) where n = total concurrent streams across all users.

Fix:

• Add a private field: private userStreamCounts = new Map<string, number>()
• In _streamAgent where activeStreams.set(requestId, ...) is called (line 779): increment the counter for userId
• In the finally block where activeStreams.delete(requestId) (line 895) and in _handleCancel where activeStreams.delete(streamId) (line 1081): decrement the counter for userId
• Replace countUserStreams(userId) body with: return this.userStreamCounts.get(userId) ?? 0

P2 — Moderate Fixes

Fix 5: Extract shared toolkit filtering helper

Files:

• packages/appkit/src/core/agent/toolkit-resolver.ts (lines 41–61)
• packages/appkit/src/core/agent/build-toolkit.ts (lines 25–62)

Problem: Both files independently implement identical prefix/only/except/rename filtering for ToolkitOptions. Bug fixes must be applied in two places.

Fix: Extract a shared helper function (e.g., filterToolkitKeys) that takes (localName: string, opts: ToolkitOptions, pluginName: string) => string | null (returns the final key or null if filtered out). Use it in both buildToolkitEntries() and the fallback path of resolveToolkitFromProvider().

Fix 6: Use isToolProvider type guard in resolveStandaloneProvider

File: packages/appkit/src/core/agent/run-agent.ts — resolveStandaloneProvider() (line 280–298)

Problem: instance as unknown as ToolProvider bypasses TypeScript entirely. A proper type guard exists at core/plugin-context.ts:308.

Fix: Import isToolProvider from ../plugin-context (or extract it to a shared location if circular imports are a concern). Replace the manual guard (lines 285–296) with:

if (!isToolProvider(instance)) {
  throw new Error(`runAgent: plugin '${pluginName}' is not a ToolProvider ...`);
}
// instance is now narrowed to ToolProviderPlugin
cache.set(pluginName, instance);
return instance;

Note: isToolProvider checks for asUser which standalone providers won't use. May need a lighter guard (isToolProviderLike) that only checks getAgentTools + executeAgentTool. Evaluate at implementation time.

Fix 7–8: Add missing tests

File: packages/appkit/src/core/agent/tests/run-agent.test.ts

Test 7: Plugin without ToolProvider methods

it("throws when plugin lacks getAgentTools/executeAgentTool", async () => {
  // Create a PluginData with a class that doesn't implement ToolProvider
  // Call runAgent with fromPlugin referencing it
  // Assert throws with "not a ToolProvider" message
});

Test 8: Sub-agent recursive execution

it("executes sub-agent tools via recursive runAgent", async () => {
  // Define parent agent with agents: { helper: childDef }
  // Mock adapter to call "agent-helper" tool
  // Verify child agent runs and returns text
});

P2 — Advisory (No Code Changes)

• (9) fromPlugin marker uses Symbol.for — forgeable in theory. Agent defs come from developer code; acceptable risk.
• (10) Sub-agent runner uses global agent registry — names come from static config, not runtime input.
• (11) New approval_pending event type in unions — downstream consumers with exhaustive switches need updating. Document in changelog.
• (12) AgentsPlugin 1200+ lines — consider future extraction. Not blocking.

P3 — Low Priority (Deferred)

• (13) New optional effect field — backward compatible, no action needed.
• (14) Cache getPluginNames() at setup time — minor optimization.
• (15) defineTool() trivial wrapper — DX decision, defer.

Verification

1. Build: pnpm build
2. Lint + typecheck: pnpm check:fix && pnpm -r typecheck
3. Tests: pnpm test — ensure all existing tests pass and new tests (7, 8) pass
4. Manual smoke test: Run pnpm dev and verify agents plugin loads, chat endpoint works, and sub-agent delegation works