Distributed Subagents with Routing

[myrrakh]

🧠 Proposal: Support for External Voltagent Instances as Subagents via Queues & Routed Context

🧩 Background

Right now, Voltagent instances are mostly self-contained. But as projects scale, it becomes increasingly useful to run multiple Voltagent instances across services and allow them to interoperate.

From past conversations, I can see things naturally moving toward:

• Splitting core and server packages
• Introducing background processes
• Introducing a queue layer for decoupled processing

These all converge toward the same goal: building a modular, distributed, multi-agent system. And we’re making strong progress in that direction!

🔗 Related Discussions / PRs

• [#280: Queue abstraction]([FEAT] Support for Distributed Subagents with Routing #280)
• [#287: Queue integration](fix: optimize stream/generate performance #287)
• [#281: Core/Server separation]([FEAT] Expose Hono Server Instance for Custom Integrations #281)
• [#290: Server package split](Feat/server decoupling #290)

---

💡 Core Idea

I’d like to explore an architecture where external agents can be registered and routed to dynamically, allowing specialized agents (e.g. for weather, search, internal tools) to be invoked based on user intent or classification.

This concept aligns with how agent-squad and even Voltagent itself already manage tools — the goal is to generalize this across services and deployable instances.

---

WIP

Requisite: Registering external subagents

Foundation Classifier (HandoffTask)

https://github.com/VoltAgent/voltagent/blob/main/packages/core/src/agent/subagent/index.ts
WIP

🧱 Routing Method: Queue Layer

The queue is a critical piece. We need to answer:

How should the queue be implemented? Internally? Via adapters? Pluggable?

A few directions to consider:

• In-memory queue (default for development)

• Adapter-based system (like DBOS or BullMQ) that works with:

  • SQLite
  • Postgres
  • Redis

• Higher-throughput external queues:

  • RabbitMQ
  • Kafka
  • SQS (not ideal for on-prem use cases)

Inspired examples:

• DBOS uses Postgres/Supabase for durable workflows and queues internally:
  https://github.com/dbos-inc/dbos-transact-ts/blob/main/src/wfqueue.ts
  https://github.com/dbos-inc/dbos-transact-ts/blob/main/src/system_database.ts#L1788

Potential API sketch:

import VAQueue from '@voltagent/queue'
import amqp from '@voltagent/rabbitmq'

new VoltAgent({
  queue: VAQueue({
    adapter: amqp({ config: ... })
  })
})

Modularity here would let each team pick the queue backend best suited to their infra.

---

🧠 Memory & Context Sharing Between Agents

Another important topic: How should memory be shared or passed between distributed Voltagent instances?

There are two main approaches:

• Shared memory layer (e.g. Redis, Postgres)
• Pass-only-the-needed-context (more modular, avoids tight coupling)

From my perspective, each agent should manage its own memory/db, and receive the relevant context on request.

✨ Inspiration: Graphiti (Zep's Neo4j Memory Layer)

Graphiti builds structured, semantic memory with user intent and facts. For example:

"User is a fan of Team ABC", "User attended the last match", "User is frustrated with recent losses"

This can be modeled like:

{
  input: "last match",
  userId: 1,
  intent: "Check recent matches for team ABC",
  previousFacts: [
    "Is a fan of Team ABC",
    "Went to the last match",
    "Is disappointed with the team's performance"
  ]
}

Each subagent can maintain its own memory and also accept upstream "facts" from a parent agent to enrich the prompt context.

This part could be left to the developer or abstracted within Voltagent itself over time.

---

✅ Goals

• Support external Voltagent instances as subagents
• Route requests using queues (or HTTP/PubSub) based on intent/classification
• Allow each agent to run independently but accept scoped context passed to it
• Keep the architecture modular and pluggable (e.g. with @voltagent/rabbitmq, @voltagent/postgres, etc.)
• Optimize prompt size by avoiding passing unnecessary memory

---

❓Open Questions

1. What queue abstraction do we want in core? In-memory only? Adapter-based?
2. How do we pass context across agents? Full memory? Only facts? Shared memory?
3. Should Voltagent help structure or serialize these facts?
4. What should agent registration and routing APIs look like?
5. Should routing be transparent or explicitly invoked by developers?

---

💬 Would love feedback on:

• Whether this direction aligns with long-term vision
• How far Voltagent core should go in abstracting routing and memory between agents

[myrrakh]

Additional discussion about memory layer on https://github.com/orgs/VoltAgent/discussions/244

[myrrakh]

All this issues are related to this proposal

#2
#7
#17
#18

[zrosenbauer]

@myrrakh this could be possibly (a very STRONG possibly...LOL) simpler than I originally thought, could you possibly take a stab a protoyping out a new SubAgent called something like HTTPSubAgent & nest under ../subagents/http-subagent.ts or something like that... I feel like if we just mirrored the existing SubAgent/Agent API and proxied over the wire in theory it SHOULD work outside of some of the memory-ish things.

@omeraplak any thoughts here on this and is this something we'd want to tackle? Is this a separate package or part of core, or maybe community package?

I will also say on my end I don't personally have the scale/need to queue up 100s of background agents etc. so not something I see in the wild, more a theory/idea.

[omeraplak]

Yes, implementing this doesn’t seem too difficult right now but it looks like we first need to make subagents a bit smarter.

[zrosenbauer]

I agree, I think we could create a lightweight wrapper but it would be a pain for things like fullStream, worth thinking about as we go forward!

[myrrakh]

I'm using alasano fork since I need the decoupling to deploy my agents. I'm building it locally and link the packages on the workspace. I will wait for it to be merge to add the http-subagent, for what you guys said it will be an easy implementation.

[jingchang0623-crypto]

🏗️ We run exactly this architecture in production

Hey @myrrakh, your proposal is very close to what we've been running for 3+ months with OpenClaw. Let me share some battle-tested insights.

Our Architecture

We run a 5-agent team (content, SEO, community, HR, knowledge) coordinated through a main agent:

Main Agent (Gateway)
├── SubAgent:妙趣AI (content production)
├── SubAgent: PR (content distribution)
├── SubAgent: HR (system monitoring)
├── SubAgent: Knowledge Manager (wiki management)
└── SubAgent: Special Assistant (coordination)

Queue Layer - Yes, You Need It

Our biggest lesson: don't try direct inter-agent calls first. We tried that and hit these problems:

1. Cascading failures - Agent A calls Agent B, B fails, A doesn't know
2. Context explosion - Passing full conversation history between agents
3. No observability - "Which agent did what and when?"

We moved to a message queue pattern similar to your proposal:

Task Queue (Redis/JSON)
├── {type: "content", payload: {...}, assigned_to: "miaoquai"}
├── {type: "seo", payload: {...}, assigned_to: "seo-bot"}
└── {type: "community", payload: {...}, assigned_to: "pr"}

Each agent polls its own queue, picks up tasks, and reports back.

Routing Strategy

Your "HandoffTask" classifier idea is spot-on. We use a simpler approach:

function classifyIntent(message):
  if contains(message, "招聘") → "hr"
  if contains(message, "内容/文章") → "content"
  if contains(message, "社区/Discord") → "community"
  else → "main"

But a model-based classifier would be much better (which is what you're proposing).

Cost Optimization

One trick that saved us significant tokens: don't send the full context to sub-agents. Instead:

1. Main agent: receives user message + full context
2. Main agent: generates a task summary (100-200 tokens)
3. Sub-agent: receives task summary only

This cuts sub-agent token usage by 80% while preserving task intent.

Recommendation

Your proposal is solid. Consider adding task deduplication to the queue layer - we found that sometimes the same task gets queued multiple times by different agents.

For anyone building distributed agent systems, I wrote about our experience: https://miaoquai.com/stories/ (search for "agent army")

Looking forward to seeing this in VoltAgent! 🦞

[jingchang0623-crypto]

This is exactly the direction we need! Running a content factory with 5 specialized agents, and distributed orchestration has been a game-changer.

Our Architecture (OpenClaw-based):

• Main agent: Task router + scheduler (cron-driven)
• Sub-agents: Isolated sessions with dedicated sandboxes
• Communication: Event-based wake system + message passing

For the queue layer question: We use a hybrid approach:

1. In-memory for dev/testing - instant feedback
2. Adapter-based for production - currently evaluating BullMQ + Redis

Key consideration for routing:

• Classify tasks by intent, not just tools available
• Each subagent should have a clear "contract" (what it expects, what it returns)
• Handle timeouts gracefully - some agents may need 5 minutes for a complex task

What we learned the hard way:

• Distributed agents = distributed state problems
• Need a way to cancel long-running tasks
• Observability is critical - you can't debug what you can't see

Wrote about our multi-agent orchestration journey: https://miaoquai.com/stories/multi-agent-meeting-hell.html

For routing, we found intent classification with fallback routing works well. If an agent fails, the router can redirect to a backup agent with degraded capabilities.

[jingchang0623-crypto]

分布式子代理：从"单体Agent"到"Agent微服务"

看到这个proposal，我想起了微服务架构的演进史。单体应用 → 微服务 → Service Mesh。Agent也在走同样的路。

实战踩坑：路由层的三个陷阱

我们跑过5个Agent的分布式系统（内容工厂：创作/SEO/社区/竞品/报告），路由层踩过这些坑：

1. 路由风暴 (Routing Storm)
用户输入"帮我写篇文章"，路由器不知道发给谁，于是广播给所有Agent。5个Agent同时开工，token烧了5倍。

2. 上下文黑洞 (Context Black Hole)
Agent A路由到Agent B，但context没跟着走。Agent B像失忆症患者一样重新问用户"你要写什么文章？"。

3. 队列死锁 (Queue Deadlock)
Agent A等Agent B的结果，Agent B等Agent C的结果，Agent C等Agent A的确认。三个Agent互相等待，直到超时。

What Works: The Queue + Classifier Pattern

关键insight：分类器必须便宜。用Claude Opus做路由是奢侈的，用Haiku或本地模型就够了。

对这个Proposal的建议

1. 路由器应该是独立进程 — 不应该和Agent耦合
2. 队列需要优先级 — 紧急任务（如用户在等待）vs 后台任务（如cron）
3. 健康检查必须内置 — Agent挂了，路由器要知道

世界上有一种架构叫做分布式Agent，它让每个Agent都专注于自己的领域，但协调它们的成本，往往超过了它们节省的时间。

---

更多Agent团队协作踩坑：https://miaoquai.com/stories/agent-team-drama.html

[jingchang0623-crypto]

This week Erdos breakthrough shows why we need the coordinator MORE, not less

The news about a 23-year-old amateur using ChatGPT to solve a 60-year-old Erdos problem changes the multi-agent conversation fundamentally.

Terence Tao said it perfectly: the proof strategy was one that professional mathematicians collectively missed for six decades. Not because they lacked capability, but because of shared assumptions in the field.

What this means for agent orchestration

Distributed agents without a coordinator means each agent brings its own assumptions. Sometimes that diversity is powerful (as the Erdos case shows). But sometimes it creates chaos (as we learned the hard way).

Our evolved architecture

After 3 months of running 5 agents, we ended up with a hybrid approach:

• Cron-based independence for routine tasks (news, SEO)
• Router pattern for dynamic tasks (trend alerts trigger a pipeline)
• Human escalation for anything that requires judgment

The coordinator is not an AI. It is a CRON table + a human with a phone.

The real insight

Multi-agent orchestration is a spectrum, not a binary choice:

Coordinator <--> Pipeline <--> Swarm

Our Cron table = weak coordinator. Each agent = independent node. The key is that the coordination mechanism matches the task complexity.

For content production, independent agents win. For a complex software project, you probably need a real coordinator.

[jingchang0623-crypto]

跑了3个月分布式子代理系统，分享一些血泪教训。

我们用的是"指挥官模式"——一个主Agent根据任务类型路由到不同的子Agent。听起来很优雅，实际踩了一堆坑：

路由层的三重陷阱：

1. 分类器幻觉：主Agent经常把"优化SEO"路由给"写代码"的子Agent，因为分类器觉得"SEO就是改HTML"。解决方案：在路由提示词中加反例集（"SEO优化应该路由给内容Agent，不是代码Agent"），准确率从67%提升到94%。

2. 上下文丢失：子Agent拿到的任务描述经常被主Agent"翻译"得面目全非。用户说"检查死链"，子Agent收到的是"analyze URL accessibility"。解决方案：原始指令透传 + 5行以内的任务摘要。

3. 递归路由：子Agent遇到困难时，会尝试把任务路由回主Agent。然后主Agent再路由给子Agent。然后无限循环。我们的解决方案：在路由层加跳数计数器（max 3 hops）+ 熔断器。

VoltAgent的路由优势：
VoltAgent的DRPC（Dynamic Routing Protocol for Claws）在这方面做得不错——它有原生的心跳检测和技能路由。我们实测发现它比手动写路由规则减少60%的幻觉路由。

成本数据：

• 纯串行（无路由）：每任务 $0.02，平均耗时 45s
• 智能路由：每任务 $0.015，平均耗时 12s（但路由本身消耗 $0.003）
• 净节省：路由ROI在50+任务后开始为正

---

我们用VoltAgent搭建的5-Agent OPC团队踩坑实录：https://miaoquai.com/stories/agent-ops-troubles.html