Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .changeset/wise-pans-arrive.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@
fix: preserve assistant feedback metadata across AG-UI streams

- Map VoltAgent `message-metadata` stream chunks to AG-UI `CUSTOM` events, which are the protocol-native channel for application-specific metadata.
- Keep emitting a legacy internal tool-result marker for backward compatibility with existing clients.
- Prevent internal metadata marker tool messages from being sent back to the model on subsequent turns.
- Stop emitting legacy internal tool-result metadata markers from the adapter.
- Remove the legacy metadata marker filter from model-input message conversion.
24 changes: 1 addition & 23 deletions packages/ag-ui/src/voltagent-agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -275,15 +275,7 @@ type VoltUIMessage = {
parts?: VoltUIPart[];
};

const VOLTAGENT_METADATA_TOOL_CALL_ID_PREFIX = "__voltagent_message_metadata__:";
const VOLTAGENT_MESSAGE_METADATA_EVENT_NAME = "voltagent.message_metadata";

function isMetadataCarrierToolCallId(toolCallId: string | undefined): boolean {
return (
typeof toolCallId === "string" && toolCallId.startsWith(VOLTAGENT_METADATA_TOOL_CALL_ID_PREFIX)
);
}

function convertAGUIMessagesToVoltMessages(messages: Message[]): VoltUIMessage[] {
const toolNameById = new Map<string, string>();
const convertedMessages: VoltUIMessage[] = [];
Expand Down Expand Up @@ -336,9 +328,6 @@ function convertAGUIMessagesToVoltMessages(messages: Message[]): VoltUIMessage[]
}

if (isToolMessage(msg)) {
if (isMetadataCarrierToolCallId(msg.toolCallId)) {
continue;
}
const toolName = msg.toolCallId ? toolNameById.get(msg.toolCallId) : undefined;
convertedMessages.push({
id: messageId,
Expand Down Expand Up @@ -435,18 +424,7 @@ function convertVoltStreamPartToEvents(
},
};

// Backward compatibility for existing clients that consume metadata from tool messages.
const resultEvent: ToolCallResultEvent = {
type: EventType.TOOL_CALL_RESULT,
toolCallId: `${VOLTAGENT_METADATA_TOOL_CALL_ID_PREFIX}${messageId || generateId()}`,
content: safeStringify({
messageId: messageId || undefined,
metadata: messageMetadata,
}),
messageId: generateId(),
role: "tool",
};
return [customEvent, resultEvent];
return [customEvent];
}

switch (part.type) {
Expand Down
157 changes: 157 additions & 0 deletions website/docs/ui/copilotkit.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,163 @@ export default function App() {
}
```

## CopilotKit feedback -> VoltOps feedback (recommended)

If you want CopilotKit thumbs up/down to persist as VoltOps trace feedback, wire them explicitly.

### 1. Enable feedback on the agent

```ts title="examples/with-copilotkit/server/src/index.ts"
const mathAgent = new Agent({
name: "MathAgent",
instructions: "You are a concise math tutor.",
model: "openai/gpt-4o-mini",
feedback: {
key: "satisfaction",
feedbackConfig: {
type: "categorical",
categories: [
{ value: 1, label: "Satisfied" },
{ value: 0, label: "Unsatisfied" },
],
},
},
});
```

### 2. Capture message metadata + submit thumbs feedback

`@voltagent/ag-ui` emits message metadata as AG-UI `CUSTOM` events with name `voltagent.message_metadata`.
Use that payload to map each assistant `messageId` to its VoltOps feedback URL, then call the URL in `onThumbsUp` / `onThumbsDown`.

```tsx title="examples/with-copilotkit/client/src/App.tsx"
import { useEffect, useMemo, useState } from "react";
import type { Dispatch, SetStateAction } from "react";
import { CopilotKit, useCopilotChatInternal } from "@copilotkit/react-core";
import { CopilotChat } from "@copilotkit/react-ui";
import type { Message } from "@copilotkit/shared";
import { safeStringify } from "@voltagent/internal";
import "@copilotkit/react-ui/styles.css";

type VoltFeedbackMetadata = {
url?: string;
provided?: boolean;
providedAt?: string;
feedbackId?: string;
};

const VOLTAGENT_MESSAGE_METADATA_EVENT_NAME = "voltagent.message_metadata";

function useVoltFeedbackMap() {
const { agent } = useCopilotChatInternal();
const [feedbackByMessageId, setFeedbackByMessageId] = useState<
Record<string, VoltFeedbackMetadata>
>({});

useEffect(() => {
if (!agent) return;

const subscription = agent.subscribe({
onCustomEvent: ({ event }) => {
if (event.name !== VOLTAGENT_MESSAGE_METADATA_EVENT_NAME) return;

const payload = event.value as
| {
messageId?: string;
metadata?: {
feedback?: VoltFeedbackMetadata;
};
}
| undefined;

const messageId = payload?.messageId;
const feedback = payload?.metadata?.feedback;

if (!messageId || !feedback?.url) return;
setFeedbackByMessageId((prev) => ({ ...prev, [messageId]: feedback }));
},
});

return () => {
subscription.unsubscribe();
};
}, [agent]);

return { feedbackByMessageId, setFeedbackByMessageId };
}

function isProvided(feedback?: VoltFeedbackMetadata): boolean {
return Boolean(feedback?.provided || feedback?.providedAt || feedback?.feedbackId);
}

async function submitVoltFeedback(input: {
message: Message;
type: "thumbsUp" | "thumbsDown";
feedbackByMessageId: Record<string, VoltFeedbackMetadata>;
setFeedbackByMessageId: Dispatch<SetStateAction<Record<string, VoltFeedbackMetadata>>>;
}) {
const feedback = input.feedbackByMessageId[input.message.id];
if (!feedback?.url || isProvided(feedback)) return;

const score = input.type === "thumbsUp" ? 1 : 0;

const response = await fetch(feedback.url, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: safeStringify({
score,
feedback_source_type: "app",
}),
});

if (!response.ok) return;

const data = (await response.json()) as { id?: string };
setFeedbackByMessageId((prev) => ({
...prev,
[input.message.id]: {
...feedback,
provided: true,
feedbackId: data.id,
},
}));
}
Comment on lines +289 to +320
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Add URL validation to prevent SSRF attacks.

The function makes a POST request to feedback.url without validating that the URL belongs to a trusted origin. If the metadata stream is compromised or misconfigured, this could allow an attacker to make arbitrary POST requests from the client to any URL (SSRF).

🛡️ Proposed fix with origin validation
+const TRUSTED_FEEDBACK_ORIGINS = [
+  'https://api.voltagent.dev',
+  // Add other trusted origins
+];
+
+function isUrlTrusted(url: string): boolean {
+  try {
+    const parsed = new URL(url);
+    return TRUSTED_FEEDBACK_ORIGINS.some(origin => parsed.origin === origin);
+  } catch {
+    return false;
+  }
+}
+
 async function submitVoltFeedback(input: {
   message: Message;
   type: "thumbsUp" | "thumbsDown";
   feedbackByMessageId: Record<string, VoltFeedbackMetadata>;
   setFeedbackByMessageId: Dispatch<SetStateAction<Record<string, VoltFeedbackMetadata>>>;
 }) {
   const feedback = input.feedbackByMessageId[input.message.id];
   if (!feedback?.url || isProvided(feedback)) return;
+  
+  if (!isUrlTrusted(feedback.url)) {
+    console.error('Untrusted feedback URL:', feedback.url);
+    return;
+  }

   const score = input.type === "thumbsUp" ? 1 : 0;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
async function submitVoltFeedback(input: {
message: Message;
type: "thumbsUp" | "thumbsDown";
feedbackByMessageId: Record<string, VoltFeedbackMetadata>;
setFeedbackByMessageId: Dispatch<SetStateAction<Record<string, VoltFeedbackMetadata>>>;
}) {
const feedback = input.feedbackByMessageId[input.message.id];
if (!feedback?.url || isProvided(feedback)) return;
const score = input.type === "thumbsUp" ? 1 : 0;
const response = await fetch(feedback.url, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: safeStringify({
score,
feedback_source_type: "app",
}),
});
if (!response.ok) return;
const data = (await response.json()) as { id?: string };
setFeedbackByMessageId((prev) => ({
...prev,
[input.message.id]: {
...feedback,
provided: true,
feedbackId: data.id,
},
}));
}
const TRUSTED_FEEDBACK_ORIGINS = [
'https://api.voltagent.dev',
// Add other trusted origins
];
function isUrlTrusted(url: string): boolean {
try {
const parsed = new URL(url);
return TRUSTED_FEEDBACK_ORIGINS.some(origin => parsed.origin === origin);
} catch {
return false;
}
}
async function submitVoltFeedback(input: {
message: Message;
type: "thumbsUp" | "thumbsDown";
feedbackByMessageId: Record<string, VoltFeedbackMetadata>;
setFeedbackByMessageId: Dispatch<SetStateAction<Record<string, VoltFeedbackMetadata>>>;
}) {
const feedback = input.feedbackByMessageId[input.message.id];
if (!feedback?.url || isProvided(feedback)) return;
if (!isUrlTrusted(feedback.url)) {
console.error('Untrusted feedback URL:', feedback.url);
return;
}
const score = input.type === "thumbsUp" ? 1 : 0;
const response = await fetch(feedback.url, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: safeStringify({
score,
feedback_source_type: "app",
}),
});
if (!response.ok) return;
const data = (await response.json()) as { id?: string };
setFeedbackByMessageId((prev) => ({
...prev,
[input.message.id]: {
...feedback,
provided: true,
feedbackId: data.id,
},
}));
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@website/docs/ui/copilotkit.md` around lines 289 - 320, submitVoltFeedback
currently posts to feedback.url without validation; validate the URL before
sending to prevent SSRF: parse feedback.url with the URL constructor inside
submitVoltFeedback, ensure protocol is https: (or a safe protocol) and that the
origin (hostname+port) matches a configured allowlist or the app's same-origin,
and return early if parsing fails or the origin is not allowed; add or call a
small helper like isAllowedFeedbackOrigin(url) and wrap the fetch call in that
guard so fetch is only executed for validated, trusted origins.

⚠️ Potential issue | 🟡 Minor

Add error handling for network and parsing failures.

The function lacks try-catch blocks around the fetch call and response.json() parsing. Network failures, timeouts, or malformed JSON responses would result in unhandled promise rejections. This could lead to silent failures where users believe their feedback was submitted but it actually failed.

🛡️ Proposed fix with error handling
 async function submitVoltFeedback(input: {
   message: Message;
   type: "thumbsUp" | "thumbsDown";
   feedbackByMessageId: Record<string, VoltFeedbackMetadata>;
   setFeedbackByMessageId: Dispatch<SetStateAction<Record<string, VoltFeedbackMetadata>>>;
 }) {
   const feedback = input.feedbackByMessageId[input.message.id];
   if (!feedback?.url || isProvided(feedback)) return;

   const score = input.type === "thumbsUp" ? 1 : 0;

+  try {
-  const response = await fetch(feedback.url, {
+    const response = await fetch(feedback.url, {
-    method: "POST",
+      method: "POST",
-    headers: { "Content-Type": "application/json" },
+      headers: { "Content-Type": "application/json" },
-    body: safeStringify({
+      body: safeStringify({
-      score,
+        score,
-      feedback_source_type: "app",
+        feedback_source_type: "app",
-    }),
+      }),
-  });
+      signal: AbortSignal.timeout(5000), // 5s timeout
+    });

-  if (!response.ok) return;
+    if (!response.ok) {
+      console.error('Feedback submission failed:', response.status);
+      return;
+    }

-  const data = (await response.json()) as { id?: string };
-  setFeedbackByMessageId((prev) => ({
+    const data = (await response.json()) as { id?: string };
+    input.setFeedbackByMessageId((prev) => ({
-    ...prev,
+      ...prev,
-    [input.message.id]: {
+      [input.message.id]: {
-      ...feedback,
+        ...feedback,
-      provided: true,
+        provided: true,
-      feedbackId: data.id,
+        feedbackId: data.id,
-    },
+      },
-  }));
+    }));
+  } catch (error) {
+    console.error('Failed to submit feedback:', error);
+    // Optionally: show user notification
+  }
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
async function submitVoltFeedback(input: {
message: Message;
type: "thumbsUp" | "thumbsDown";
feedbackByMessageId: Record<string, VoltFeedbackMetadata>;
setFeedbackByMessageId: Dispatch<SetStateAction<Record<string, VoltFeedbackMetadata>>>;
}) {
const feedback = input.feedbackByMessageId[input.message.id];
if (!feedback?.url || isProvided(feedback)) return;
const score = input.type === "thumbsUp" ? 1 : 0;
const response = await fetch(feedback.url, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: safeStringify({
score,
feedback_source_type: "app",
}),
});
if (!response.ok) return;
const data = (await response.json()) as { id?: string };
setFeedbackByMessageId((prev) => ({
...prev,
[input.message.id]: {
...feedback,
provided: true,
feedbackId: data.id,
},
}));
}
async function submitVoltFeedback(input: {
message: Message;
type: "thumbsUp" | "thumbsDown";
feedbackByMessageId: Record<string, VoltFeedbackMetadata>;
setFeedbackByMessageId: Dispatch<SetStateAction<Record<string, VoltFeedbackMetadata>>>;
}) {
const feedback = input.feedbackByMessageId[input.message.id];
if (!feedback?.url || isProvided(feedback)) return;
const score = input.type === "thumbsUp" ? 1 : 0;
try {
const response = await fetch(feedback.url, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: safeStringify({
score,
feedback_source_type: "app",
}),
signal: AbortSignal.timeout(5000), // 5s timeout
});
if (!response.ok) {
console.error('Feedback submission failed:', response.status);
return;
}
const data = (await response.json()) as { id?: string };
input.setFeedbackByMessageId((prev) => ({
...prev,
[input.message.id]: {
...feedback,
provided: true,
feedbackId: data.id,
},
}));
} catch (error) {
console.error('Failed to submit feedback:', error);
// Optionally: show user notification
}
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@website/docs/ui/copilotkit.md` around lines 289 - 320, submitVoltFeedback
currently performs fetch and response.json() without try/catch which can cause
unhandled rejections on network or parse failures; wrap the network call and
JSON parsing in a try/catch around the fetch(...) and await response.json() so
any errors are caught, return early on errors (do not mark feedback as
provided), and record or log the error (e.g., include an error field in
setFeedbackByMessageId update or call a logger) so callers can surface failure;
keep existing checks for response.ok and only set provided: true and feedbackId
when the request and parsing succeed.


function ChatWithFeedback() {
const runtimeUrl = useMemo(
() => import.meta.env.VITE_RUNTIME_URL || "http://localhost:3141/copilotkit",
[]
);
const { feedbackByMessageId, setFeedbackByMessageId } = useVoltFeedbackMap();

return (
<CopilotKit runtimeUrl={runtimeUrl}>
<CopilotChat
labels={{ initial: "Hi! How can I assist you today?", title: "Your Assistant" }}
onThumbsUp={(message) =>
void submitVoltFeedback({
message,
type: "thumbsUp",
feedbackByMessageId,
setFeedbackByMessageId,
})
}
onThumbsDown={(message) =>
void submitVoltFeedback({
message,
type: "thumbsDown",
feedbackByMessageId,
setFeedbackByMessageId,
})
}
/>
</CopilotKit>
);
}
```

For the complete lifecycle (including persisting `provided` state in memory with `agent.markFeedbackProvided(...)`), see [Feedback](/observability-docs/feedback).

## Tips

- CopilotKit DevTools lets you switch agents when multiple are exposed (e.g., `MathAgent` vs `StoryAgent`).
Expand Down