Add expressive expr marker dialect#1988
Conversation
🦋 Changeset detectedLatest commit: 650417e The changes in this PR will be included in the next version bump. This PR includes changesets to release 36 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| (_match, _tag, _attrs, inner: string | undefined) => inner ?? '', | ||
| ); | ||
| clean = clean.replace(new RegExp(`<\\/(?:${tagPattern})\\s*>`, 'g'), ''); | ||
| return clean.replace(/\[[^\]]+\]/g, ''); |
There was a problem hiding this comment.
🔍 stripAllMarkup removes all square-bracket content, which may strip legitimate text
The stripAllMarkup function at agents/src/tts/provider_format.ts:238 ends with .replace(/\[[^\]]+\]/g, '') which strips ALL [...] content from the text. This is used to clean assistant messages before storing them in chat context (agent_activity.ts:2652, agent_activity.ts:3102, agent_activity.ts:3148). If the LLM produces legitimate square-bracket content (e.g. [1] citations, [link text](url) markdown, or literal brackets), that content will be silently removed from the conversation history. This only affects users who enable the expressive option, since stripExpressiveMarkup gates on expressiveInstructions returning a truthy value. The risk is low since markdown filtering is typically already applied, but worth being aware of.
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
<expr/>expression markersexpressiveoptions onAgentSession/AgentVerification
pnpm build:agentspnpm exec eslint "agents/src/tts/provider_format.ts" "agents/src/inference/tts.ts" "agents/src/voice/agent.ts" "agents/src/voice/agent_activity.ts" "agents/src/voice/agent_session.ts" "agents/src/voice/index.ts" "agents/src/voice/turn_config/utils.ts" "plugins/cartesia/src/tts.ts" "plugins/inworld/src/tts.ts"(warnings only from existing rules/patterns)cue-clivoice-mode runtime check with temporary agentexpr-marker-js, asserting assistant framework event did not contain raw<expr/>markersNotes
pnpm --filter @livekit/agents-plugin-cartesia --filter @livekit/agents-plugin-inworld buildbuilt JS bundles; Inworld completed fully, Cartesia declaration generation failed on existing missing test-only workspace deps (@livekit/agents-plugin-silero,@livekit/agents-plugins-test,@livekit/agents-plugin-openai).pnpm --filter @livekit/agents api:checkfailed before API analysis because the current API Extractor does not support the existingexport * as ...syntax inagents/dist/index.d.ts.Ported from livekit/agents#6347
Original PR description
Replace the provider-native markup dialects with a single marker tag — the only dialect the LLM is ever taught, everywhere (llm_instructions() and the expressive preset bodies alike):
The marker syntax is shared, but each provider gets its own instruction block advertising only the types and label vocabularies it actually supports:
The provider-native instruction blocks are deleted; the six preset bodies are rewritten in expr. The native tag tables remain solely as tolerance — a hallucinated native tag is still stripped from the transcript and converted for the TTS instead of leaking.
convert_markup lowers expr to each provider's native syntax before synthesis and drops types a provider doesn't support — the words always survive, the marker never leaks. normalize_markup closes unclosed self-closing markers, and the transcript strippers remove expr in a dedicated pre-pass so the type/label pair surfaces as an ExpressiveTag.