feat(app): desktop companion UI — Tauri bridge, overlay, settings (#1909)#2246
Merged
senamakel merged 6 commits intoMay 19, 2026
Merged
Conversation
…inyhumansai#1909) Socket.IO + Tauri hotkey bridge, Redux companion slice, overlay companion mode, settings panel with session controls, status badge.
- companion_commands: use crate::AppRuntime (tauri::Cef, not stale Wry) - CompanionPanel.test: replace require() with ESM import + mock cast - socketService: merge duplicate companionSlice imports
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (14)
✅ Files skipped from review due to trivial changes (3)
🚧 Files skipped from review as they are similar to previous changes (3)
📝 WalkthroughWalkthroughAdds a desktop companion subsystem: Tauri hotkey commands and shared hotkey state, Redux companion slice and selectors with tests, Socket.IO bridging, Settings UI with session controls and tests, overlay visuals including a companion mode and pointer overlay, and i18n strings. ChangesDesktop Companion Feature
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint skipped: no ESLint configuration detected in root package.json. To enable, add Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/src-tauri/src/companion_commands.rs`:
- Around line 111-120: The guard.clear() is called before attempting OS
unregisters, causing in-memory state to be lost if any unregister fails; move
the mutation so the in-memory registry is only cleared/updated after successful
unregisters — e.g., iterate over a cloned list (the existing let old =
guard.clone();) and call app.global_shortcut().unregister(...) for each
shortcut, and only clear the original guard (or remove each shortcut from guard)
after all unregister calls succeed; adjust error handling so failures return
without mutating guard (referencing guard, old, and the for shortcut in old
loop).
In `@app/src/components/settings/panels/CompanionPanel.tsx`:
- Line 3: Replace direct callCoreRpc usage in CompanionPanel.tsx with the
in-process relay invocation; any calls that currently pass RPC names like
"openhuman.companion_status" or companion control RPCs should instead call the
Tauri/IPC relay via invoke('core_rpc_relay', { method: '<rpc_name>', params:
<params> }). Locate each occurrence of callCoreRpc in this file (the status
fetch and the companion control handlers) and swap the call to
invoke('core_rpc_relay', ...) preserving the same method name and parameters
structure, and ensure the calling code still handles the returned promise/result
the same way.
In `@app/src/overlay/CompanionPointer.tsx`:
- Around line 21-24: The effect currently calls setVisible(true) synchronously
which violates the react-hooks/set-state-in-effect rule; change it to schedule
the show update asynchronously (e.g., call setVisible(true) inside a short timer
or requestAnimationFrame) and manage timers in the same effect so they are
cleared on cleanup. In the CompanionPointer useEffect (references: targets,
setVisible, dismissMs) replace the direct setVisible(true) with an async
showTimer (e.g., window.setTimeout(() => setVisible(true), 0) or
requestAnimationFrame) and keep/clear the existing dismiss timer (and any
created timers) in the cleanup to avoid leaks.
In `@app/src/overlay/OverlayApp.tsx`:
- Around line 688-690: The ARIA label for the companion branch in OverlayApp
(the ternary that checks mode === 'companion') is hardcoded as "Companion
active"; replace that literal with a localized string using the existing i18n
function (t) — e.g., t('overlay.companionActive') — and add the matching
key/value to the localization resources so the companion branch uses the same
translation mechanism as the other branches.
In `@app/src/pages/Settings.tsx`:
- Around line 246-252: The new settings entry with id 'companion' currently
hardcodes English strings for title and description; update the Settings
component to use the i18n translation function (the same t(...) used by other
items) so both title and description call t with appropriate keys (e.g.,
t('settings.desktopCompanion.title') and
t('settings.desktopCompanion.description') and ensure keys are added to the
locale files), and keep the other properties (id, route, icon) unchanged so the
item integrates with the existing settings array.
In `@app/src/services/socketService.ts`:
- Around line 295-300: The companion event handler currently only checks for a
'state' key and may dispatch malformed data; create and use a strict type-guard
(e.g., isCompanionStateChangedEvent) that validates the shape and types of
CompanionStateChangedEvent (verify event is object, event.state is one of the
allowed values/enums, and event.sessionId is a non-empty string or other
expected type) inside the socket.on('companion:state_changed', ...) callback,
log an error via socketLog when validation fails, and only call
store.dispatch(setCompanionState(event)) when the guard returns true to prevent
invalid sessionId/state from being written to Redux.
In `@app/src/store/companionSlice.ts`:
- Around line 76-84: The reducer setCompanionState currently only assigns
state.lastError on error events, allowing stale errors to persist; update
setCompanionState (handling CompanionStateChangedEvent) to clear state.lastError
when event.state is not 'error' (e.g., set it to undefined or null) so that once
the companion recovers the UI no longer shows a previous error.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 45d91403-dce4-496b-a48d-6cec20572cde
📒 Files selected for processing (14)
app/src-tauri/src/companion_commands.rsapp/src-tauri/src/lib.rsapp/src/components/BottomTabBar.tsxapp/src/components/settings/panels/CompanionPanel.tsxapp/src/components/settings/panels/__tests__/CompanionPanel.test.tsxapp/src/overlay/CompanionPointer.tsxapp/src/overlay/OverlayApp.tsxapp/src/pages/Settings.tsxapp/src/services/socketService.tsapp/src/store/companionSlice.test.tsapp/src/store/companionSlice.tsapp/src/store/index.tsapp/src/test/test-utils.tsxsrc/core/socketio.rs
coderabbitai
Bot
added
the
feature
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/src/overlay/OverlayApp.tsx`:
- Around line 99-112: The companionStateLabel helper currently returns hardcoded
English strings; update it so labels are localized by either returning
translation keys or calling the i18n translator directly (use
companionStateLabel to call t(...) instead of returning raw strings, or have it
return keys like 'companion.state.listening' and translate at the call site).
Locate the function companionStateLabel and replace each hardcoded string (cases
'listening','thinking','speaking','pointing', default) with calls to the
translation function t(...) or their corresponding translation keys, and ensure
the file imports the translator (or the caller performs t(...) if you choose to
return keys). Keep the same wording/ellipsis in translations and preserve the
default behavior by mapping unknown states to a translated template (e.g., a key
that accepts the state as a variable).
- Around line 311-316: When handling the 'error' state in the OverlayApp (the
block that calls setMode('companion') and setBubble with id
`companion-error-...`), trim payload.message before deciding whether to use it;
i.e., compute a trimmed string from payload?.message and if that trimmed value
is non-empty use it inside the quoted text, otherwise fall back to the literal
"Error" so whitespace-only messages don't render as an empty quoted bubble.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: c28e603c-10f4-4935-b336-2f27f00c87ce
📒 Files selected for processing (7)
app/src/components/__tests__/BottomTabBar.test.tsxapp/src/components/settings/panels/__tests__/CompanionPanel.test.tsxapp/src/overlay/OverlayApp.tsxapp/src/overlay/__tests__/CompanionPointer.test.tsxapp/src/overlay/__tests__/companionStateLabel.test.tsapp/src/services/__tests__/companionPayload.test.tsapp/src/services/socketService.ts
✅ Files skipped from review due to trivial changes (1)
- app/src/overlay/tests/companionStateLabel.test.ts
Contributor
Author
|
Hey Steve (@senamakel) — UI for the companion core from #2025. Closing the loop on #1909 properly. |
mtkik
pushed a commit
to mtkik/openhuman-meet
that referenced
this pull request
May 21, 2026
CodeGhost21
pushed a commit
to CodeGhost21/openhuman
that referenced
this pull request
May 22, 2026
AusAgentSmith
pushed a commit
to AusAgentSmith/openhuman
that referenced
this pull request
May 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wires the UI/bridge layer for the desktop companion domain landed in #2025.
register_companion_hotkey,unregister_companion_hotkey,companion_activate— same managed-state + rollback pattern asdictation_hotkeys. No JS injection.desktop_companion::bus::subscribe_state_changed()and emitscompanion:state_changedover the web channel.companionSlice(non-persisted) +socketServicelistener dispatching on inbound events.OverlayAppextended with'companion'mode rendering idle / listening / thinking / speaking / pointing / error. Label-onlyCompanionPointer(full desktop-absolute pointing requires a separate transparent window — deferred).Problem
The core domain in #2025 has no entry point into the desktop product surface — no hotkey, no Socket.IO state into the React app, no settings affordance, no overlay rendering. This PR closes those gaps so the #1909 acceptance criteria around desktop entry, state visibility, and frontend tests are actually met.
Solution
Stacked on #2025 (squash-merged as
504d10e9onmain). A single fixup commit reconciles withmainpost-rebase:crate::AppRuntimeinstead of staletauri::Wry, ESM-pattern Vitest mock forcoreRpcClient, deduplicatedcompanionSliceimports.Reuses the RPC and event surface from #2025 —
openhuman.companion_start_session/companion_stop_session/companion_status/companion_config_get/companion_config_set, and theCompanionSessionStarted/CompanionStateChanged/CompanionSessionEndedevents.Submission Checklist
companionSlice.test.ts(15 cases),CompanionPanel.test.tsx(6 cases incl. "shows error when start session fails")dictation_hotkeyspattern## Related## RelatedImpact
companion:state_changed+ underscore alias).Related
Refs #1909 — completes the UI/bridge half deferred from #2025.
PS: #1909 shows as closed because GitHub's
Closeskeyword in #2025's body auto-closed it on merge — the feature itself is only complete once this PR ships.Test plan
cargo check(core lib + Tauri shell) — cleanpnpm typecheck/lint/format:check— clean (0 lint errors)pnpm debug unit companionSlice— 15/15pnpm debug unit CompanionPanel— 6/6Summary by CodeRabbit
New Features
Tests
Documentation