🦆 Duckyd - Voice Mentor for Claude Code

Duckyd is a voice-enabled teaching assistant that observes your Claude Code sessions and provides real-time guidance through natural speech. It watches what Claude does, explains concepts, and answers your questions - all without interrupting your workflow.

Features

• Voice Teaching: Real-time spoken explanations while you code with ElevenLabs TTS
• Natural Conversation: Interrupt anytime with voice to ask questions (barge-in support)
• Smart Cadence: Knows when to speak based on work phases, developer state, and importance scoring
• Arc Tracking: Understands your work cycles (prompt → tools → completion) for contextual comments
• Conversation Memory: Maintains dialogue history with automatic summarization
• Privacy First: No data stored by default, configurable PII masking
• Customizable Mentor: Adjust teaching style, verbosity, voice, and speaking cadence
• Observer Only: Never controls your Claude Code session - pure teaching layer
• Fail-Soft Design: Daemon failure doesn't affect Claude Code; voice failure falls back to text
• Profile Inheritance: Global → Project → Session configuration cascade
• Keyboard Shortcuts: Quick controls (V: toggle mentor, M: mic, S: speaker)

Quick Start

Prerequisites

• Node.js 20+ and pnpm 10+
• Claude Code CLI installed and configured
• ElevenLabs API key for voice functionality (Get one here)

Installation

# Clone the repository
git clone https://github.com/dhofheinz/duckyd.git
cd duckyd

# Install dependencies
pnpm install

# Build all packages
pnpm build

# Start the daemon
node packages/cli/dist/index.js start

# Open the web UI
open http://localhost:5173

Configure Claude Code Hooks

Duckyd needs to receive events from Claude Code. Add the following to your Claude Code settings file (~/.claude/settings.json):

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "",
        "hooks": [
          {
            "type": "command",
            "command": "node /absolute/path/to/duckyd/packages/hooks/dist/pre-tool-use.js"
          }
        ]
      }
    ],
    "PostToolUse": [
      {
        "matcher": "",
        "hooks": [
          {
            "type": "command",
            "command": "node /absolute/path/to/duckyd/packages/hooks/dist/post-tool-use.js"
          }
        ]
      }
    ],
    "UserPromptSubmit": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "node /absolute/path/to/duckyd/packages/hooks/dist/user-prompt-submit.js"
          }
        ]
      }
    ],
    "Stop": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "node /absolute/path/to/duckyd/packages/hooks/dist/stop.js"
          }
        ]
      }
    ]
  }
}

Important: Replace /absolute/path/to/duckyd with the actual absolute path to your Duckyd installation.

Note: Hooks are designed to never block Claude Code. They have a 100ms timeout and always exit successfully, even if the daemon is unreachable.

Set Up ElevenLabs

1. Get an API key from ElevenLabs
2. Set the environment variable:

export ELEVENLABS_API_KEY=your_key_here

3. (Optional) Get a voice ID from ElevenLabs or use a default:
   • Go to ElevenLabs Voice Library
   • Pick a voice and copy its ID
   • Configure it in your profile (see Configuration section)

Architecture

┌──────────────────┐     ┌─────────────────┐     ┌──────────────────┐
│   Claude Code    │────>│  Duckyd Daemon  │────>│    Web UI        │
│   (with hooks)   │     │  (Fastify + WS) │     │  (React + Vite)  │
└──────────────────┘     └────────┬────────┘     └──────────────────┘
                                  │
                         ┌────────┴────────┐
                         │                 │
                    ┌────▼────┐      ┌─────▼─────┐
                    │   TTS   │      │    STT    │
                    │ElevenLabs      │ ElevenLabs│
                    └─────────┘      └───────────┘

Data Flow

1. Claude Code invokes hook scripts with JSON on stdin
2. Hook scripts compact events and POST to daemon (100ms timeout, always exit 0)
3. Daemon stores events in per-session ring buffer (last 500 events)
4. Daemon notifies WebSocket clients and triggers cadence controller
5. Mentor Brain assembles teaching prompt from recent events using LLM
6. TTS streams audio to browser; user can barge-in to interrupt
7. STT captures user voice questions and feeds them back to mentor

Package Structure

• @duckyd/shared - Zod schemas and TypeScript types (dependency for all other packages)
• @duckyd/daemon - Fastify 5 server with WebSocket, event ingestion, session state
• @duckyd/web - React 19 + Vite + Tailwind v4 dashboard
• @duckyd/cli - Commander-based mentorctl for daemon management
• @duckyd/hooks - Claude Code hook emitters (PreToolUse, PostToolUse, UserPromptSubmit, Stop, Statusline)

Configuration

Environment Variables

Variable	Description	Default
ELEVENLABS_API_KEY	ElevenLabs API key for voice (required for voice features)	—
ANTHROPIC_API_KEY	Anthropic API key (optional, for direct API backend)	—
CLAUDE_COMMAND	Path to Claude CLI executable	claude
CLAUDE_MODEL	Model for mentor responses (opus, sonnet, haiku)	sonnet
CLAUDE_MAX_TURNS	Max agentic turns for mentor responses	3
DUCKYD_PORT	Daemon HTTP/WebSocket port	7777
DUCKYD_HOST	Daemon bind address	127.0.0.1
DUCKYD_URL	Daemon URL (used by CLI and hooks)	http://127.0.0.1:7777

Profile Settings

Profiles control how the mentor behaves. They cascade from global → project → session, with each level able to override specific fields.

Profile Locations:

• Global: ~/.duckyd/profile.json
• Project: .duckyd/profile.json (in project root)
• Session: Runtime override via Web UI

Example Profile:

{
  "id": "default",
  "name": "Balanced Mentor",
  "teaching": {
    "style": "balanced",
    "verbosity": 0.5,
    "interrupt_tolerance": "medium",
    "ask_before_lecturing": true,
    "focus": []
  },
  "cadence": {
    "min_seconds_between_spoken_turns": 20,
    "speak_on_stop_hook": true,
    "speak_on_errors": true,
    "batch_window_ms": 1200
  },
  "voice": {
    "provider": "elevenlabs",
    "voice_id": "your_voice_id_here",
    "model": "eleven_flash_v2_5"
  },
  "stt": {
    "provider": "elevenlabs",
    "mode": "realtime_ws",
    "vad": true,
    "sample_rate_hz": 16000
  },
  "privacy": {
    "store_audio": false,
    "store_transcripts": false,
    "pii_masking": "on",
    "redaction_rules": ["emails", "api_keys", "paths_home"]
  },
  "llm": {
    "backend": "claude_code_headless",
    "command": "claude",
    "args": ["-p"],
    "allowedTools": [],
    "permissionMode": "default"
  }
}

Configuration Options Explained

Teaching Style

Option	Type	Description
style	"socratic" | "explanatory" | "balanced"	Teaching approach: Socratic asks guiding questions, explanatory tells directly, balanced mixes both
verbosity	0.0 - 1.0	How verbose responses are (0 = terse, 1 = detailed)
interrupt_tolerance	"low" | "medium" | "high"	How easily mentor can be interrupted mid-explanation
ask_before_lecturing	boolean	Ask permission before launching into long explanations
focus	string[]	Topics to focus on (e.g., ["security", "performance"])

Cadence (Speaking Timing)

Option	Type	Description
min_seconds_between_spoken_turns	number	Minimum seconds between spoken turns (prevents spam)
speak_on_stop_hook	boolean	Speak when Claude Code session stops
speak_on_errors	boolean	Speak when errors are detected in events
batch_window_ms	number	Milliseconds to batch events before deciding to speak

Voice (TTS)

Option	Type	Description
provider	"elevenlabs"	TTS provider (currently only ElevenLabs supported)
voice_id	string	ElevenLabs voice ID
model	string	ElevenLabs model (default: eleven_flash_v2_5 for low latency)

STT (Speech-to-Text)

Option	Type	Description
provider	"elevenlabs"	STT provider (currently only ElevenLabs supported)
mode	"realtime_ws"	STT mode (WebSocket real-time)
vad	boolean	Voice Activity Detection (auto-detect speech boundaries)
sample_rate_hz	number	Audio sample rate (default: 16000)

Privacy

Option	Type	Description
store_audio	boolean	Store audio recordings (default: false)
store_transcripts	boolean	Store text transcripts (default: false)
pii_masking	"off" | "on" | "aggressive"	PII masking level before sending to LLM
redaction_rules	string[]	Categories to redact: "emails", "api_keys", "paths_home", "urls_with_tokens", "phone_numbers"

LLM Backend

Option	Type	Description
backend	"claude_code_headless" | "anthropic_api" | "openai" | "local"	Which LLM backend to use
command	string	Command to execute (for claude_code_headless)
args	string[]	Command arguments
allowedTools	string[]	Allowed tools (empty = none; mentor is observer-only)
permissionMode	string	Permission mode for Claude Code headless

CLI Commands

The mentorctl CLI manages the daemon:

# Check daemon status and health
mentorctl status

# Start daemon in background
mentorctl start

# Start daemon in foreground (useful for debugging)
mentorctl start --foreground

# Start daemon on custom port
mentorctl start --port 8888

# Stop daemon gracefully
mentorctl stop

# Force kill daemon if graceful shutdown fails
mentorctl stop --force

# Restart daemon
mentorctl restart

# List active Claude Code sessions
mentorctl sessions

# View daemon logs (last 50 lines)
mentorctl logs

# Follow daemon logs in real-time
mentorctl logs --follow

# Show configuration paths
mentorctl config

Daemon Files

• Pidfile: ~/.duckyd/daemon.pid
• Logs: ~/.duckyd/daemon.log
• Global profile: ~/.duckyd/profile.json
• Project profile: .duckyd/profile.json (in repository root)

Development

# Run all packages in development mode (hot reload)
pnpm dev

# Run specific package in development
pnpm --filter @duckyd/daemon dev
pnpm --filter @duckyd/web dev

# Type checking across all packages
pnpm typecheck

# Build all packages for production
pnpm build

# Run tests
pnpm test

# Lint code
pnpm lint

# Format code
pnpm format

# Clean all build artifacts
pnpm clean

Project Structure

duckyd/
├── packages/
│   ├── daemon/     # Fastify server with WebSocket support
│   │   └── src/
│   │       ├── mentor/    # Brain, cadence controller, conversation memory
│   │       ├── voice/     # TTS/STT managers, ElevenLabs client
│   │       ├── privacy/   # PII masking
│   │       └── routes/    # HTTP and WebSocket endpoints
│   ├── web/        # React UI for dashboard and configuration
│   │   └── src/
│   │       ├── components/  # UI components including voice overlay
│   │       ├── hooks/       # WebSocket, audio capture, VAD
│   │       └── design-system/  # Tokens, typography, colors
│   ├── cli/        # mentorctl command-line interface
│   ├── hooks/      # Claude Code hook emitters
│   ├── shared/     # Shared Zod schemas and TypeScript types
│   └── e2e/        # End-to-end tests
├── docs/           # Architecture documentation
└── CLAUDE.md       # Development guidance for Claude Code

Troubleshooting

Daemon won't start

Symptoms: mentorctl start fails or daemon exits immediately

Solutions:

1. Check if port 7777 is already in use: lsof -i :7777
2. Check daemon logs for errors: mentorctl logs
3. Try starting in foreground mode to see errors: mentorctl start --foreground
4. Verify Node.js version: node --version (must be 20+)
5. Ensure all packages are built: pnpm build

No voice output

Symptoms: Mentor doesn't speak, no audio in browser

Solutions:

1. Verify ELEVENLABS_API_KEY environment variable is set
2. Check daemon health status: mentorctl status
3. Verify TTS connection in status output shows "✓ Connected"
4. Ensure browser allows audio playback (check browser console)
5. Check that profile has valid voice_id configured
6. Test ElevenLabs API directly to verify quota/billing

Hooks not triggering

Symptoms: No events showing in dashboard, mentor seems unaware of Claude Code activity

Solutions:

1. Verify hook paths in ~/.claude/settings.json are absolute paths
2. Test hook manually: echo '{"session_id":"test"}' | node /path/to/hooks/dist/post-tool-use.js
3. Check that hooks have been built: ls packages/hooks/dist/
4. Verify daemon is running and reachable: curl http://localhost:7777/api/health
5. Check Claude Code is using the correct settings file: claude --help (shows settings path)

WebSocket disconnects

Symptoms: Web UI loses connection, shows "Disconnected" status

Solutions:

1. Check daemon is running: mentorctl status
2. Look for WebSocket errors in daemon logs: mentorctl logs
3. Restart the daemon: mentorctl restart
4. Check browser console for WebSocket error messages
5. Verify firewall isn't blocking localhost connections

Voice is laggy or choppy

Symptoms: Audio playback stutters, high latency between events and speech

Solutions:

1. Check your internet connection speed
2. Reduce verbosity setting in profile (0.3 or lower)
3. Increase min_seconds_between_spoken_turns to reduce frequency
4. Verify ElevenLabs service status
5. Consider switching to eleven_flash_v2_5 model (lowest latency)

Barge-in not working

Symptoms: Can't interrupt mentor while speaking

Solutions:

1. Ensure browser has microphone permission (check browser settings)
2. Verify STT connection in mentorctl status shows "✓ Connected"
3. Check that interrupt_tolerance isn't set to "low" in profile
4. Test microphone in browser console: check for AudioWorklet support
5. Try refreshing the web UI

PII not being masked

Symptoms: Sensitive data visible in transcripts or mentor responses

Solutions:

1. Verify privacy.pii_masking is set to "on" or "aggressive" in profile
2. Check that relevant redaction_rules are enabled
3. Add custom patterns to profile if needed
4. Remember: masking happens before LLM calls, not retroactively
5. Review privacy.store_transcripts setting (should be false for maximum privacy)

"Daemon already running" but status shows not running

Symptoms: mentorctl start says daemon is running but mentorctl status says it's not

Solutions:

1. Check for stale pidfile: cat ~/.duckyd/daemon.pid
2. Kill any stale process: kill $(cat ~/.duckyd/daemon.pid)
3. Remove stale pidfile: rm ~/.duckyd/daemon.pid
4. Try starting again: mentorctl start

Privacy & Security

Duckyd is designed with privacy as a core principle:

Data Retention

• Audio recordings: Not stored by default (store_audio: false)
• Transcripts: Not stored by default (store_transcripts: false)
• Event data: Kept in memory only, ring buffer of last 500 events per session
• Session data: Cleared when Claude Code session ends

PII Masking

• Before LLM calls: All event data is passed through PII masking before being sent to the LLM
• Configurable rules: Email, API keys, file paths, URLs with tokens, phone numbers
• Masking levels: Off, On (standard patterns), Aggressive (broader patterns)
• Local processing: Masking happens on your machine before data leaves

Network Communication

• TTS/STT: Audio streams directly between browser and ElevenLabs
• LLM: Uses Claude Code headless by default (local CLI), or optional Anthropic API
• Daemon: Runs locally on 127.0.0.1, not exposed to network
• No telemetry: Duckyd doesn't phone home or collect analytics

Observer-Only Design

• No tool calls: Mentor never executes tools in the active Claude Code session
• No file writes: Mentor cannot modify your code or files
• Read-only access: Only receives event notifications, doesn't control Claude Code

Use Cases

Learning a new codebase

Configure the mentor to be more verbose and explanatory while you explore an unfamiliar project. The mentor will explain patterns, architectural decisions, and guide you through the code.

Code review companion

Set teaching style to "socratic" and the mentor will ask probing questions about your changes, helping you think critically about edge cases and design decisions.

Junior developer pairing

Use a balanced, verbose mentor that explains not just what Claude Code is doing, but why, teaching best practices and patterns along the way.

Focus mode

Set high min_seconds_between_spoken_turns and low verbosity for minimal interruption. The mentor will only speak for important events like errors or session stops.

Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Make your changes and add tests
4. Run type checking and tests: pnpm typecheck && pnpm test
5. Commit with clear messages: git commit -m 'Add amazing feature'
6. Push to your fork: git push origin feature/amazing-feature
7. Open a Pull Request

Development Guidelines

• Follow existing code style (use pnpm format)
• Add TypeScript types for all new code
• Update Zod schemas in @duckyd/shared for data changes
• Write tests for new features
• Update documentation for user-facing changes
• Test hooks with real Claude Code sessions before committing

Tech Stack

• Runtime: Node.js 20+, TypeScript 5.7+, ESM modules
• Monorepo: Turborepo for build orchestration, pnpm workspaces
• Backend: Fastify 5 with @fastify/websocket for HTTP/WS server
• Frontend: React 19, Vite 6, Tailwind v4
• Validation: Zod for schema validation
• Voice: ElevenLabs Flash v2.5 for STT/TTS (75ms latency)
• LLM: Claude Code headless or Anthropic API

Roadmap

• Claude Code statusline integration
• Additional TTS/STT providers (OpenAI, Google, Azure)
• More LLM backends (OpenAI, local models via Ollama)
• Visual code annotations in web UI
• Session replay and analysis
• Custom redaction patterns via regex
• Mobile app for remote monitoring
• Multi-project dashboard
• Mentor personality customization
• Integration with git hooks for commit guidance

License

MIT

Acknowledgments

• Built on Claude Code by Anthropic
• Voice powered by ElevenLabs
• Inspired by pair programming and teaching assistants everywhere

Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Documentation: docs/

---

Happy coding with your voice mentor! 🦆