-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Hotel receptionist polish #6299
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ShayneP
wants to merge
3
commits into
main
Choose a base branch
from
hotel-receptionist-polish
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| # Project tests | ||
| test/ | ||
| tests/ | ||
| eval/ | ||
| evals/ | ||
|
|
||
| # Python bytecode and artifacts | ||
| __pycache__/ | ||
| *.py[cod] | ||
| *.pyo | ||
| *.pyd | ||
| *.egg-info/ | ||
| dist/ | ||
| build/ | ||
|
|
||
| # Virtual environments | ||
| .venv/ | ||
| venv/ | ||
|
|
||
| # Caches and test output | ||
| .cache/ | ||
| .pytest_cache/ | ||
| .ruff_cache/ | ||
| coverage/ | ||
|
|
||
| # Logs and temp files | ||
| *.log | ||
| *.gz | ||
| *.tgz | ||
| .tmp | ||
| .cache | ||
|
|
||
| # Environment variables | ||
| .env | ||
| .env.* | ||
|
|
||
| # VCS, editor, OS | ||
| .git | ||
| .gitignore | ||
| .gitattributes | ||
| .github/ | ||
| .idea/ | ||
| .vscode/ | ||
| .DS_Store | ||
|
|
||
| # Project docs and misc | ||
| README.md | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
|
|
||
| # Coding agent files | ||
| .claude/ | ||
| .codex/ | ||
| .cursor/ | ||
| .windsurf/ | ||
| .gemini/ | ||
| .cline/ | ||
| .clinerules | ||
| .clinerules/ | ||
| .aider* | ||
| .cursorrules | ||
| .cursorignore | ||
| .cursorindexingignore | ||
| .clineignore | ||
| .codeiumignore | ||
| .geminiignore | ||
| .windsurfrules | ||
| CLAUDE.md | ||
| AGENTS.md | ||
| GEMINI.md | ||
| .github/copilot-instructions.md | ||
| .github/personal-instructions.md | ||
| .github/instructions/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,3 +3,4 @@ fake_data/hotel.db-shm | |
| fake_data/hotel.db-wal | ||
| __pycache__/ | ||
| .env.local | ||
| livekit.toml | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,146 @@ | ||
| """A sidecar that suggests what the *caller* could say next. | ||
|
|
||
| Built specifically for the frontend of the demo at http://livekit.com/agents/hotel-receptionist | ||
|
|
||
| After each receptionist turn, a small LLM proposes one to three short replies the | ||
| caller might give - so a guest who isn't sure how to respond has a ready line to | ||
| speak. The suggestions are published on the agent participant as the | ||
| ``lk.agent.suggestions`` attribute, so the web frontend can render | ||
| clickable nudge chips. | ||
|
|
||
| The whole feature is one self-contained object attached in a single line: | ||
|
|
||
| SuggestedReplies(session, ctx.room, llm=inference.LLM("google/gemma-4-31b-it")).attach() | ||
|
|
||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import asyncio | ||
| import json | ||
| import logging | ||
|
|
||
| from livekit import rtc | ||
| from livekit.agents import AgentSession, ConversationItemAddedEvent, llm | ||
|
|
||
| logger = logging.getLogger("hotel-receptionist.suggestions") | ||
|
|
||
| SUGGESTIONS_ATTRIBUTE = "lk.agent.suggestions" | ||
| MAX_SUGGESTIONS = 4 | ||
|
|
||
| _SYSTEM_PROMPT = """\ | ||
| You help a guest who is ON A PHONE CALL with a hotel receptionist. Given the \ | ||
| receptionist's most recent line, suggest 1-3 short things the guest could say next. \ | ||
| This is a demo, so the guest should NEVER be left without something to say - always \ | ||
| return at least one ready reply. | ||
|
|
||
| Return ONLY JSON, no markdown fences, in exactly this shape: | ||
| {"suggestions": [{"label": "<=3 words for a button", "value": "the exact words the guest speaks"}]} | ||
|
|
||
| How to choose replies: | ||
| - Phrase "value" as a natural first-person guest utterance (e.g. "Non-smoking, please."). | ||
| - If the receptionist offered explicit choices (yes/no, listed room types, listed \ | ||
| times), suggest from those. | ||
| - If the receptionist asks for the guest's OWN details - name, email, phone, card - \ | ||
| just make one up so the guest can say it. Use obviously-fake demo values: a name like \ | ||
| "Alex Morgan", an address at example.com for email, a 555 phone number like \ | ||
| "415-555-0142", and the test card "4242 4242 4242 4242". | ||
| - Do NOT invent facts the HOTEL controls that weren't stated - real prices, room types \ | ||
| not offered, availability, or confirmation codes. If unsure there, suggest a neutral \ | ||
| reply like "Yes, that works" or a short clarifying question instead. | ||
| - Always return at least one suggestion; never return an empty list. | ||
| """ | ||
|
|
||
|
|
||
| class SuggestedReplies: | ||
| """Watches an ``AgentSession`` and publishes per-turn reply suggestions. | ||
|
|
||
| Attach it once after building the session; it does the rest for the life of | ||
| the call. | ||
| """ | ||
|
|
||
| def __init__(self, session: AgentSession, room: rtc.Room, *, llm: llm.LLM) -> None: | ||
| self._session = session | ||
| self._room = room | ||
| self._llm = llm | ||
| self._task: asyncio.Task[None] | None = None | ||
|
|
||
| def attach(self) -> SuggestedReplies: | ||
| """Subscribe to the session. Returns self so it reads as one expression.""" | ||
| self._session.on("conversation_item_added", self._on_item) | ||
| return self | ||
|
|
||
| def _on_item(self, ev: ConversationItemAddedEvent) -> None: | ||
| item = ev.item | ||
| # React only to the receptionist's own turns; ignore the caller's. | ||
| if not isinstance(item, llm.ChatMessage) or item.role != "assistant": | ||
| return | ||
| # The newest turn wins: drop any suggestion still being generated for a | ||
| # previous line so we can never publish stale chips. | ||
| if self._task and not self._task.done(): | ||
| self._task.cancel() | ||
| self._task = asyncio.create_task(self._run(item.text_content or "")) | ||
|
|
||
| async def _run(self, assistant_line: str) -> None: | ||
| try: | ||
| await self._suggest(assistant_line) | ||
| except asyncio.CancelledError: | ||
| raise | ||
| except Exception: | ||
| logger.exception("suggested-reply generation failed") | ||
| await self._publish([]) # fail safe to no chips, never stale ones | ||
|
|
||
| async def _suggest(self, assistant_line: str) -> None: | ||
| # Clear last turn's chips immediately; we'll set this turn's below. | ||
| await self._publish([]) | ||
|
|
||
| ctx = llm.ChatContext() | ||
| ctx.add_message(role="system", content=_SYSTEM_PROMPT) | ||
| ctx.add_message(role="user", content=f'The receptionist just said: "{assistant_line}"') | ||
|
|
||
| stream = self._llm.chat(chat_ctx=ctx) | ||
| raw = "" | ||
| try: | ||
| async for chunk in stream: | ||
| if chunk.delta and chunk.delta.content: | ||
| raw += chunk.delta.content | ||
| finally: | ||
| await stream.aclose() | ||
|
|
||
| await self._publish(_parse(raw)) | ||
|
|
||
| async def _publish(self, items: list[dict[str, str]]) -> None: | ||
| # set_attributes merges keys, so this leaves lk.agent.state etc. untouched. | ||
| await self._room.local_participant.set_attributes( | ||
| {SUGGESTIONS_ATTRIBUTE: json.dumps(items)} | ||
| ) | ||
|
|
||
|
|
||
| def _parse(raw: str) -> list[dict[str, str]]: | ||
| """Pull a clean [{label, value}] list out of the model's reply, or [] on any doubt.""" | ||
| data = _loads(raw) | ||
| items = data.get("suggestions") if isinstance(data, dict) else data | ||
| if not isinstance(items, list): | ||
| return [] | ||
| clean: list[dict[str, str]] = [] | ||
| for it in items: | ||
| if not isinstance(it, dict): | ||
| continue | ||
| label, value = it.get("label"), it.get("value") | ||
| if isinstance(label, str) and isinstance(value, str) and label.strip() and value.strip(): | ||
| clean.append({"label": label.strip(), "value": value.strip()}) | ||
| return clean[:MAX_SUGGESTIONS] | ||
|
|
||
|
|
||
| def _loads(raw: str) -> object: | ||
| try: | ||
| return json.loads(raw) | ||
| except (ValueError, TypeError): | ||
| # Salvage a JSON object if the model wrapped it in prose or ``` fences. | ||
| start, end = raw.find("{"), raw.rfind("}") | ||
| if start != -1 and end > start: | ||
| try: | ||
| return json.loads(raw[start : end + 1]) | ||
| except (ValueError, TypeError): | ||
| pass | ||
| return {} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🚩 TTS-dependent behavior change in _speak_code removes explicit character spelling
The old
_speak_codespelled codes character by character with dashes spoken as "dash" (e.g., "H, T, L, dash, X, Q, 7, Z"). The new version just returnscode.upper()and relies entirely on the TTS engine's built-in parser to read alphanumeric codes correctly. This is used in ~20 places (tools_rooms.py,tools_services.py,tools_restaurant.py) for confirmation codes, case numbers, and reference codes. If the TTS engine doesn't handle mixed alphanumeric codes well (e.g., reads "HTL-XQ7Z" as a word rather than spelling it), callers won't be able to note down their codes. The persona instructions atexamples/hotel_receptionist/persona.py:27now explicitly say the TTS handles this, so this is intentional - but it's a significant behavioral dependency on the TTS provider.Was this helpful? React with 👍 or 👎 to provide feedback.