Tabby

A system-wide AI keyboard layer that transforms your input device into a real-time AI collaborator.

Features · Screenshots · Tech Stack · Getting Started · Architecture · Docs · Contributing · License

---

Features

Tabby lives at the point of input - no more switching between apps for AI help.

Feature	Description
Interview Copilot	Real-time coding interview assistance with screen capture and multi-tab analysis
Context-Aware Autocomplete	Inline AI suggestions based on what you're typing, anywhere on your system
Desktop Automation	Full Windows MCP integration for system-level control
Persistent Memory	Remembers your preferences, coding style, and past interactions via Mem0
Invisible Typing	AI types directly into any application, character-by-character
Voice Agent	Voice-to-text, text-to-voice, and live conversational agent
Action Menu	Quick AI actions on selected text — fix grammar, change tone, expand, summarize

---

Screenshots

Interview Copilot

Interview Ghost Text

Action Menu

Voice Agent

Knowledge Graph

Memories Dashboard

Settings

Architecture

---

Tech Stack

Layer	Technology
Desktop App	Electron 38
Frontend	Next.js 15, React 19, Tailwind CSS
AI	Vercel AI SDK, OpenAI / Groq / Cerebras / Google Gemini
Memory	Mem0 (Supabase vector store + Neo4j graph)
Desktop Automation	nut-js, node-window-manager, Windows MCP
Database	Supabase (Local Docker)

---

Keyboard Shortcuts

Global Shortcuts

Shortcut	Action
Ctrl+\	Open / close action menu
Ctrl+Space	Get AI suggestion
Ctrl+Shift+B	Toggle brain panel
Ctrl+Alt+I	Interview ghost text
Ctrl+Alt+J	Voice Agent
Ctrl+Shift+X	Stop autotyping
Ctrl+Shift+T	Cycle transcribe modes
Ctrl+Alt+T	Toggle voice transcription

Interview Copilot

Shortcut	Action
Alt+X	Capture screen & analyze coding problem
Alt+Shift+X	Update analysis with new constraints
Alt+N	Get code suggestions / improvements
Ctrl+1-6	Switch tabs (Chat, Idea, Code, Walkthrough, Test Cases, Memories)

Navigation

Shortcut	Action
Ctrl+Arrow	Move floating window
Esc	Back / close
Enter	Accept & paste

---

Getting Started

Prerequisites

• Node.js 18+
• Python 3.12+ (for memory backend)
• uv — Python package manager
• pnpm — JavaScript package manager
• Docker Desktop — for local Supabase
• An OpenAI API key

Optional API keys

• Google Generative AI API key
• XAI API key
• Groq API key
• Cerebras API key
• OpenRouter API key
• Tavily API key (web search)
• Neo4j instance (knowledge graph)

1. Clone & Install

git clone https://github.com/TabbyAIKeyboard/tabby.git
cd tabby

# Frontend
cd frontend && pnpm install

# Next.js Backend
cd ../nextjs-backend && pnpm install

# Memory Backend
cd ../backend && uv sync

2. Database Setup

We use a local Supabase instance running in Docker.

# Start Docker Desktop first, then:
npx supabase init     # first time only
npx supabase start    # starts all services (~10 s after first run)

After startup, note the API URL, anon key, and service_role key printed in the terminal.

Supabase Quick Reference

Action	Command
Start	npx supabase start
Stop	npx supabase stop
Status	npx supabase status
Admin UI	http://localhost:54323
Reset DB	npx supabase db reset

Note: Docker Desktop must be running before npx supabase start.

Neo4j (Knowledge Graph — Optional)

1. Create a free instance at Neo4j AuraDB.
2. Save the credentials text file (URI, username, password).
3. Note the Instance ID and Instance Name from the dashboard.

3. Environment Variables

Copy the example env files, then fill in your credentials:

cp frontend/env.example frontend/.env.local
cp nextjs-backend/env.example nextjs-backend/.env.local
cp backend/env.example backend/.env

Frontend - frontend/.env.local

NEXT_PUBLIC_SUPABASE_URL="http://127.0.0.1:54321"
NEXT_PUBLIC_SUPABASE_ANON_KEY="<ANON_KEY from supabase status>"
SUPABASE_ADMIN="<SERVICE_ROLE_KEY from supabase status>"

NEXT_PUBLIC_APP_NAME="Tabby"
NEXT_PUBLIC_APP_ICON="/logos/tabby-logo.png"

NEXT_PUBLIC_API_URL="http://localhost:3001"
NEXT_PUBLIC_MEMORY_API_URL="http://localhost:8000"

Next.js Backend - nextjs-backend/.env.local

NEXT_PUBLIC_SUPABASE_URL="http://127.0.0.1:54321"
NEXT_PUBLIC_SUPABASE_ANON_KEY="<ANON_KEY from supabase status>"
SUPABASE_ADMIN="<SERVICE_ROLE_KEY from supabase status>"

RESEND_API_KEY=""
RESEND_DOMAIN=""

NEXT_PUBLIC_APP_NAME=Tabby
NEXT_PUBLIC_APP_ICON='/logos/tabby-logo.png'

# AI Providers
OPENAI_API_KEY=""
GOOGLE_GENERATIVE_AI_API_KEY=""
GROQ_API_KEY=""
CEREBRAS_API_KEY=""
OPENROUTER_API_KEY=""

TAVILY_API_KEY=""

MEMORY_API_URL="http://localhost:8000"

Backend - backend/.env

OPENAI_API_KEY=
SUPABASE_CONNECTION_STRING="postgresql://postgres:postgres@127.0.0.1:54322/postgres"

# Neo4j (optional)
NEO4J_URL=
NEO4J_USERNAME=
NEO4J_PASSWORD=

4. Run

# Start Supabase first (Docker Desktop must be running)
npx supabase start

# Start all services in dev mode (backend + frontend + nextjs-backend)
pnpm dev

All three services start in parallel with color-coded logs.

Run services individually

# Terminal 1 — Memory backend
cd backend && uv run main.py

# Terminal 2 — Next.js backend
cd nextjs-backend && pnpm dev

# Terminal 3 — Windows MCP server (optional)
cd frontend && pnpm run windows-mcp

# Terminal 4 — Electron app
cd frontend && pnpm dev

Production mode

# Build and start everything
pnpm prod

# Or step-by-step:
pnpm build    # builds frontend + nextjs-backend
pnpm start    # starts all services in production mode

Once running:

Service	URL
Supabase API	http://127.0.0.1:54321
Supabase Studio	http://localhost:54323
Frontend (Electron)	http://localhost:3000
Next.js Backend	http://localhost:3001
Memory API	http://localhost:8000
Windows MCP	http://localhost:8001

5. System Tray

Tabby runs in the system tray. Right-click the icon for:

• Show Actions Menu
• Brain Panel
• Settings
• Quit

---

Building & Releasing

Local Build

cd frontend
pnpm run dist

The .exe will be in frontend/dist.

GitHub Releases (CI)

Automated Windows releases via GitHub Actions.

1. GitHub Secrets — Add to repository settings:
   • GH_TOKEN — Personal Access Token (classic) with repo scope
   • All NEXT_PUBLIC_* and SUPABASE_* variables from .env.local
2. Trigger a release:

   cd frontend && pnpm run release

   This creates a git tag from package.json version, pushes it, and triggers a GitHub Action to build and publish.

Python Backend — Azure

• Workflow: .github/workflows/backend-deploy.yml
• Trigger: Push to backend/ on main
• URL: tabby-backend.azurecontainerapps.io
• Builds Docker image → pushes to Docker Hub (thecubestar/tabby-backend) → updates Azure Container App

Next.js Backend — Vercel

• Deployment: Automatic from main
• URL: tabby-api-psi.vercel.app

---

Project Structure

tabby/
├── frontend/                   # Electron + Next.js desktop app
│   ├── electron/src/           # Electron main process
│   │   ├── main.ts             # Window management, shortcuts
│   │   ├── text-handler.ts     # Clipboard, typewriter mode
│   │   └── context-capture.ts  # Periodic screenshot capture
│   └── src/
│       ├── app/                # Next.js pages
│       └── components/         # React components
│           ├── action-menu/    # Main AI menu, copilot, chat
│           ├── brain-panel/    # Memory dashboard
│           └── ai-elements/    # Message rendering
├── nextjs-backend/             # Shared API backend (Next.js)
│   └── src/app/api/            # AI and auth API routes
├── backend/                    # FastAPI memory server
│   └── main.py                 # Mem0 API endpoints
└── supabase/                   # Database migrations & config

---

Contributing

Contributions are what make the open-source community amazing. Any contributions you make are greatly appreciated.

Check out our Documentation for detailed guides on architecture, features, and APIs.

Please read our Contributing Guide to get started. In short:

1. Fork the repo
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes using Conventional Commits
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

---

License

Distributed under the MIT License. See LICENSE for more information.