diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..208e4df --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,34 @@ +--- +name: Bug Report +about: Report a bug to help us improve DiffKit +title: "" +labels: bug +assignees: "" +--- + +## Description + +A clear and concise description of the bug. + +## Steps to Reproduce + +1. +2. +3. + +## Expected Behavior + +What you expected to happen. + +## Actual Behavior + +What actually happened. + +## Screenshots + +If applicable, add screenshots to help explain the problem. + +## Environment + +- Browser: +- OS: diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000..0086358 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1 @@ +blank_issues_enabled: true diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 0000000..f4c6167 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,23 @@ +--- +name: Feature Request +about: Suggest a new feature or enhancement for DiffKit +title: "" +labels: enhancement +assignees: "" +--- + +## Problem + +What problem are you trying to solve? + +## Proposed Solution + +Describe the solution you'd like. + +## Alternatives Considered + +Any alternative solutions or features you've considered. + +## Additional Context + +Any other context, mockups, or screenshots. diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000..450b0d8 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,19 @@ +## Summary + + + +## Changes + +- + +## Test Plan + +- [ ] + +## Screenshots + + + +| Before | After | +|--------|--------| +| Previous UI | After UI | diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 88694fd..7793e4d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -6,6 +6,8 @@ Thanks for your interest in contributing to DiffKit! This guide will help you ge Follow the [Getting Started](README.md#getting-started) section in the README to set up your local environment. You'll need both a GitHub OAuth App and a GitHub App configured. +For webhook testing during local development, use an ngrok tunnel (or similar) and set `DEV_TUNNEL_URL` in your `.dev.vars`. + ## Project Architecture DiffKit is a **pnpm monorepo** managed with **Turborepo**: @@ -22,20 +24,29 @@ diffkit/ │ │ ├── api/ # API routes (auth callbacks, webhooks) │ │ └── _protected/ # Auth-gated routes │ └── drizzle/ # SQL migration files +├── extensions/ +│ └── diffkit-redirect/ # Browser extension (GitHub → DiffKit redirects) ├── packages/ │ ├── ui/ # Shared UI components (Radix UI + Tailwind CSS) │ ├── icons/ # Icon wrapper package │ └── typescript-config/ # Shared TypeScript configurations -└── scripts/ # Migration runner and dev utilities +├── scripts/ # Migration runner and dev utilities +└── docs/ # Architecture documentation ``` ### Key Technologies +- **TanStack Start** — Full-stack React 19 framework on Cloudflare Workers - **TanStack Router** — File-based routing in `apps/dashboard/src/routes/` - **TanStack Query** — Server state management and caching - **Drizzle ORM** — Database schema and migrations in `apps/dashboard/src/db/` and `apps/dashboard/drizzle/` - **Better Auth** — Authentication with a GitHub OAuth App, plus GitHub App user and installation tokens for installed repos -- **Cloudflare D1** — SQLite database at the edge +- **Cloudflare D1** — SQLite database at the edge (auth data, cache control state) +- **Cloudflare KV** — Hot payload cache for GitHub API responses (`GITHUB_CACHE_KV` binding) +- **Cloudflare Durable Objects** — `SignalRelay` for real-time webhook-to-client revalidation over WebSocket +- **Vite** — Build tooling via `@cloudflare/vite-plugin` +- **Vitest** — Test framework (`pnpm --filter dashboard test`) +- **Biome** — Linting and formatting ### GitHub Integration @@ -43,15 +54,40 @@ DiffKit uses a hybrid GitHub auth model: - The **GitHub OAuth App** signs users in and powers broad user-context reads, including public or external repositories where the GitHub App is not installed. - The **GitHub App user token** (`ghu_` prefix) powers installation discovery via `GET /user/installations`. -- The **GitHub App installation token** is preferred for repo-scoped reads and writes when the app is installed for that owner. +- The **GitHub App installation token** is preferred for repo-scoped reads and writes when the app is installed for that owner. Tokens are cached in KV (with in-memory fallback) and reused until five minutes before expiry. Auth callbacks: - OAuth App: `/api/auth/callback/github` - GitHub App user authorization: `/api/github/app/callback` -- GitHub App setup URL: `/?show-org-setup=true` (with **Redirect on update** enabled) +- GitHub App setup URL: `/setup` (with **Redirect on update** enabled) Environment variables are documented in `apps/dashboard/.dev.vars.example`. Do not commit real `.dev.vars` values or private keys. If a private key is exposed, revoke it in GitHub App settings and generate a replacement. +### Caching & Revalidation + +DiffKit uses a split-cache architecture to minimize GitHub API calls while keeping data fresh. For a deep dive, see [`docs/github-cache-architecture.md`](docs/github-cache-architecture.md). Here's the overview: + +**Split KV/D1 cache** — GitHub API responses are cached in Cloudflare KV for fast reads, with D1 as the authoritative control plane for invalidation state. Key files: + +- `apps/dashboard/src/lib/github-cache.ts` — Core cache read/write logic +- `apps/dashboard/src/lib/github-revalidation.ts` — Signal key definitions and webhook-to-signal mapping +- `apps/dashboard/src/lib/github.functions.ts` — GitHub API operations with cache mode opt-ins + +**Invalidation flow** — When a GitHub webhook arrives or a mutation runs: + +1. Affected signal keys are resolved (e.g. `pull:{owner}/{repo}#{number}`, `pulls.mine`) +2. D1 revalidation signal timestamps and namespace versions are bumped +3. Future cache reads build a different KV key from the new namespace version, naturally bypassing stale entries +4. Connected clients are notified in real time via the `SignalRelay` Durable Object + +**Real-time revalidation** — The `SignalRelay` Durable Object (`apps/dashboard/src/lib/signal-relay.server.ts`) maintains WebSocket connections per user. When the webhook handler broadcasts signal keys, subscribed clients receive them instantly. Detail routes use a one-shot signal check (`apps/dashboard/src/lib/use-github-signal-stream.ts`) to invalidate only the active query when a newer server-side signal exists. + +**Rate-limit resilience** — The cache layer extends freshness when GitHub quota is low (≤100 remaining: 2-min floor; ≤25 remaining: 5-min floor or until reset). If GitHub returns a rate-limit error and a cached payload exists, the stale payload is served instead of failing. + +### Webhook Handler + +The webhook endpoint at `apps/dashboard/src/routes/api/webhooks/github.ts` verifies the HMAC-SHA256 signature, maps events to cache signal keys, writes invalidation state to D1, and broadcasts to connected WebSocket clients. Supported events include `pull_request`, `issues`, `issue_comment`, `check_run`, `check_suite`, `pull_request_review`, and more. + ### Adding a New Route Routes live in `apps/dashboard/src/routes/`. TanStack Router uses file-based routing — create a new file and the route is automatically registered. @@ -62,6 +98,17 @@ Protected routes go under `_protected/` which enforces authentication. Shared components go in `packages/ui/src/components/`. App-specific components go in `apps/dashboard/src/components/`. +### Database Migrations + +Migration files live in `apps/dashboard/drizzle/`. To run migrations: + +```bash +pnpm --filter dashboard migrate # Local D1 +pnpm --filter dashboard migrate:remote # Remote D1 +``` + +If you add a new table or column, create a new numbered SQL file in `apps/dashboard/drizzle/`. + ## Workflow 1. **Fork the repo** and create your branch from `main` @@ -72,7 +119,11 @@ Shared components go in `packages/ui/src/components/`. App-specific components g pnpm lint # Linting pnpm format # Formatting ``` -4. **Open a pull request** against `main` +4. **Run tests** if you touched caching or GitHub integration: + ```bash + pnpm --filter dashboard test + ``` +5. **Open a pull request** against `main` ## Code Style diff --git a/README.md b/README.md index 677bfb5..4032084 100644 --- a/README.md +++ b/README.md @@ -68,7 +68,7 @@ The GitHub App provides installation tokens for repo-scoped access, webhook deli - **GitHub App name**: DiffKit Dev (must be globally unique) - **Homepage URL**: `http://localhost:3000` - **Callback URL**: `http://localhost:3000/api/github/app/callback` - - **Setup URL**: `http://localhost:3000/?show-org-setup=true` + - **Setup URL**: `http://localhost:3000/setup` - Check **Redirect on update** - Leave **Request user authorization (OAuth) during installation** **unchecked** - **Webhook URL**: leave blank for now (see [Local webhook testing](#local-webhook-testing) below) @@ -272,23 +272,23 @@ Events marked "Later" are harmless to enable now — the app will ignore them un - [x] Submit reviews (approve, request changes, comment) - [x] Update branch with base - [ ] Create new pull requests -- [ ] Merge pull requests (merge, squash, rebase) -- [ ] Close / reopen pull requests +- [x] Merge pull requests (merge, squash, rebase) +- [x] Close / reopen pull requests - [ ] Edit PR title, body, and metadata -- [ ] Add / remove reviewers -- [ ] Add / remove labels +- [x] Add / remove reviewers +- [x] Add / remove labels - [ ] Link issues to pull requests ### Issues - [x] List issues by role (assigned, authored, mentioned) - [x] Issue detail view with metadata, body, and comments -- [ ] Create new issues +- [x] Create new issues - [ ] Close / reopen issues -- [ ] Comment on issues +- [x] Comment on issues - [ ] Edit issue title, body, and metadata -- [ ] Assign / unassign users -- [ ] Add / remove labels +- [x] Assign / unassign users +- [x] Add / remove labels - [ ] Set milestones ### Code Reviews @@ -310,14 +310,14 @@ Events marked "Later" are harmless to enable now — the app will ignore them un ### Repositories -- [ ] Repository list and search -- [ ] Repository file browser +- [x] Repository list and search +- [x] Repository file browser - [ ] Branch and tag management -- [ ] README preview +- [x] README preview ### Search -- [ ] Global search across PRs, issues, and repos +- [x] Global search across PRs, issues, and repos - [ ] Saved searches and filters - [ ] Advanced query syntax @@ -326,10 +326,10 @@ Events marked "Later" are harmless to enable now — the app will ignore them un - [x] GitHub App authentication - [x] Dark mode with system preference - [x] Response caching with ETags -- [ ] Keyboard shortcuts -- [ ] Command palette -- [ ] User settings and preferences -- [ ] Mobile-responsive layout +- [x] Keyboard shortcuts +- [x] Command palette +- [x] User settings and preferences +- [x] Mobile-responsive layout ## Contributing