Summary
Add a nav agent command that runs an interactive REPL loop on the client, communicates with LLMs through a new Chat gRPC service on the Navigator gateway server, and executes tools by shelling out to nav CLI commands to operate the cluster.
Design
- Agent loop runs client-side. The gRPC
Chat service handles single LLM calls only (server-streaming RPC). The client orchestrates multi-turn tool-calling loops.
- Tools shell out to the CLI. The agent executes tools by running
nav <subcommand> <args> as subprocesses (e.g., nav sandbox list, nav cluster status). No direct gRPC calls for tools.
- Binary resolution: Use
std::env::current_exe() by default, with NAV_AGENT_CLI env var override for dev.
- Cluster context pinning: Every shell-out includes
--cluster <name> so subcommands target the correct cluster.
- Provider-agnostic tool calling protocol. Proto defines its own
Tool, ToolCall, ChatMessage messages. The server translates to/from OpenAI or Anthropic format based on the resolved inference route's protocol.
- Route selection via
routing_hint = "agent". The operator creates an inference route with routing_hint: agent. The Chat service filters routes using the existing list_sandbox_routes function.
- Streaming responses.
ChatStream is a server-streaming RPC that streams ContentDelta events as tokens arrive via SSE, then sends a final ChatMessage with any tool_calls.
Components
Proto (proto/chat.proto)
Chat service with ChatStream server-streaming RPC
ChatStreamRequest, ChatMessage, Tool, ToolCall, ChatStreamEvent, ContentDelta, ChatError messages
Server (crates/navigator-server/src/chat/)
ChatService — gRPC implementation with route resolution, protocol detection, spawned streaming task
openai.rs — OpenAI format translation and SSE streaming with tool call accumulation
anthropic.rs — Anthropic format translation (system prompt extraction, tool_use/tool_result blocks) and SSE streaming
- Multiplexer extended from 2 to 3 gRPC services
CLI (crates/navigator-cli/src/agent.rs)
- 9 tools:
cluster_status, sandbox_list, sandbox_get, sandbox_create, sandbox_delete, sandbox_logs, provider_list, provider_get, inference_route_list
- Binary resolution with
NAV_AGENT_CLI env var override
- Interactive REPL using
rustyline with tool-calling loop
--routing-hint and --system-prompt CLI arguments
Acceptance Criteria
Summary
Add a
nav agentcommand that runs an interactive REPL loop on the client, communicates with LLMs through a newChatgRPC service on the Navigator gateway server, and executes tools by shelling out tonavCLI commands to operate the cluster.Design
Chatservice handles single LLM calls only (server-streaming RPC). The client orchestrates multi-turn tool-calling loops.nav <subcommand> <args>as subprocesses (e.g.,nav sandbox list,nav cluster status). No direct gRPC calls for tools.std::env::current_exe()by default, withNAV_AGENT_CLIenv var override for dev.--cluster <name>so subcommands target the correct cluster.Tool,ToolCall,ChatMessagemessages. The server translates to/from OpenAI or Anthropic format based on the resolved inference route's protocol.routing_hint = "agent". The operator creates an inference route withrouting_hint: agent. The Chat service filters routes using the existinglist_sandbox_routesfunction.ChatStreamis a server-streaming RPC that streamsContentDeltaevents as tokens arrive via SSE, then sends a finalChatMessagewith anytool_calls.Components
Proto (
proto/chat.proto)Chatservice withChatStreamserver-streaming RPCChatStreamRequest,ChatMessage,Tool,ToolCall,ChatStreamEvent,ContentDelta,ChatErrormessagesServer (
crates/navigator-server/src/chat/)ChatService— gRPC implementation with route resolution, protocol detection, spawned streaming taskopenai.rs— OpenAI format translation and SSE streaming with tool call accumulationanthropic.rs— Anthropic format translation (system prompt extraction, tool_use/tool_result blocks) and SSE streamingCLI (
crates/navigator-cli/src/agent.rs)cluster_status,sandbox_list,sandbox_get,sandbox_create,sandbox_delete,sandbox_logs,provider_list,provider_get,inference_route_listNAV_AGENT_CLIenv var overriderustylinewith tool-calling loop--routing-hintand--system-promptCLI argumentsAcceptance Criteria
nav agentstarts an interactive REPLcargo check,cargo clippy,cargo test, andcargo fmtall pass