feat(034): distinguish ingestion types (invoice vs license request)

[studert]

Why

ingestion_log was built for invoices, then license-request ingestion (the "new request for a user" flow from Microsoft Forms) was squeezed into the same table by overloading invoice columns — invoice_number held a form-response GUID, linked_invoice_id (a real FK to invoices.id) held a license_requests.id, and the "filtered" outcome was reused to mean "deduped". The Settings → Ingestion table, the dashboard activity feed, and the settings copy all rendered invoice semantics, so a license request showed up as a half-empty, partly-wrong invoice row — and the overloaded FK was a latent bug (silent audit-insert rejection, or a cross-wired "Download document" link).

This PR makes the two ingestion kinds first-class and distinguished end-to-end, and is built to make future ingestion types cheap to add. Design docs live in specs/034-ingestion-types-distinction/ (proposal.html, implementation-plan.html — Nothing-styled, implementation-notes.html).

What changed

Schema (P0 — additive, migration 0023)

• New ingestion_kind + ingestion_source_type enums.
• ingestion_log gains kind (NOT NULL DEFAULT 'invoice'), source_type, label, entity_type, entity_id, details (jsonb) + a kind index. Deprecated invoice columns retained (expand step).
• IngestionDetails discriminated union in src/types/ingestion.ts.

Write path (P2)

• logIngestion() is the canonical discriminated logger; logIngestionAttempt() kept as a dual-writing deprecated shim so the ~22 invoice call sites are untouched and legacy columns stay populated during the window.
• License route stops overloading linked_invoice_id and "filtered"; dedup is now success + details.deduped. Closes the latent cross-type FK bug.

Read path + UI (P3)

• getIngestionHistory returns kind/details/entity.
• Ingestion-type registry (registry.tsx) + server-safe label builder (labels.ts).
• History table rebuilt: Kind pill, Summary, per-kind drill-through (invoice → PDF; license request → /requests/:id), and All / Invoices / License requests sub-tabs — using the in-repo Nothing design primitives.
• Generalised settings copy and the dashboard activity feed (now kind-aware: "License request · {requester}" vs "Invoice from {vendor}").

Backfill (P1)

• scripts/backfill-ingestion-kind.ts — idempotent, dry-run by default, classifies legacy rows by invoice_number ∈ license_requests.form_response_id.

Open questions

All five from the proposal resolved with the plan defaults (sync stays separate; filter + sub-tabs; no new outcome enum; user_import reserved; JSONB details).

Test plan

• ✅ pnpm typecheck clean · pnpm lint 0 warnings · pnpm test 404 pass (incl. 10 new unit tests for the label builder + registry).
• ✅ Migration 0023 generated and reviewed (drizzle-migration-reviewer → PASS, purely additive).
• ⏳ Not yet run (no DB in the build env): apply 0023 and run scripts/backfill-ingestion-kind.ts --apply against the Neon preview branch, then exercise invoice + license-request ingestion end-to-end. Planned against this PR's Vercel/Neon preview.

Deferred

• P4 (contract): dropping the deprecated columns + the shim — a later, separate destructive migration after a soak window.

https://claude.ai/code/session_018kwPp9j8BHkKQwFSRfvvvR

---

Generated by Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
ai-developer-hub	Ready	Preview, Comment	Jun 3, 2026 9:08am

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[Copilot]

Pull request overview

This PR makes ingestion events first-class by distinguishing invoice ingestion vs. license-request ingestion throughout the schema, write/read paths, and admin UI. It introduces a kind discriminator plus JSONB details, replaces the overloaded linked_invoice_id usage with a polymorphic (entity_type, entity_id) drill-through reference, and updates Settings → Ingestion + the dashboard activity feed to render kind-appropriate summaries and links.

Changes:

• Add new ingestion enums/columns (kind, source_type, label, entity_*, details) via additive migration 0023.
• Introduce a canonical logIngestion() (with an invoice-shaped deprecated shim) + label/registry helpers and unit tests.
• Update license-request ingest logging, ingestion history table UI, and dashboard activity feed to be kind-aware.

Reviewed changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/unit/ingestion/labels-and-registry.test.ts	Adds unit tests for label building, registry drill-through, and present-kinds ordering.
src/types/ingestion.ts	Defines ingestion type system (kinds/source/outcome/channel + details union).
src/types/index.ts	Re-exports ingestion types from the central types barrel.
src/lib/ingestion/registry.tsx	Adds kind registry mapping kinds to icons and drill-through behavior.
src/lib/ingestion/labels.ts	Adds server-safe kind labels and buildIngestionLabel() used to populate ingestion_log.label.
src/lib/ingestion-logger.ts	Implements canonical discriminated ingestion logger + deprecated invoice shim.
src/lib/db/schema.ts	Adds new ingestion enums/columns and index; keeps legacy invoice columns for migration window.
src/lib/db/migrations/meta/_journal.json	Registers migration 0023_perfect_runaways.
src/lib/db/migrations/0023_perfect_runaways.sql	Adds enums + new ingestion_log columns + kind index (additive migration).
src/components/dashboard/admin/activity-timeline.tsx	Generalizes the activity card description copy to cover non-invoice ingestions.
src/app/settings/ingestion/page.tsx	Updates ingestion settings page copy to mention license requests.
src/app/settings/ingestion/ingestion-history-table.tsx	Rebuilds ingestion history UI to show Kind pill + Summary + per-kind drill-through + kind sub-tabs.
src/app/api/license-requests/ingest/route.ts	Switches license-request ingestion logging to logIngestion() with correct kind/details/entity and dedup semantics.
src/actions/ingestion-log.ts	Updates ingestion-history query/result shape to include kind/source/label/details/entity fields.
src/actions/dashboard.ts	Makes recent dashboard ingestion activity kind-aware using kind/details/label with legacy fallbacks.
specs/034-ingestion-types-distinction/proposal.html	Adds design proposal document for ingestion type distinction.
specs/034-ingestion-types-distinction/implementation-plan.html	Adds implementation plan document.
specs/034-ingestion-types-distinction/implementation-notes.html	Adds implementation notes / decisions log.
scripts/backfill-ingestion-kind.ts	Adds idempotent backfill script (dry-run by default) to classify legacy rows and populate new columns.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[studert]

Thanks — all four addressed in ef7dfb4:

1. kind vs details.kind drift — logIngestion now derives the stored kind from details.kind (single source of truth) and the redundant kind param is dropped, so they can't disagree.
2. Missing "other" details variant — added OtherIngestionDetails ({ kind: "other"; description? }) so IngestionDetails covers every ingestion_kind enum value, plus the buildIngestionLabel case.
3. Search broken for legacy label=null rows — the table now normalises label ?? vendor before passing rows to DataTable, so legacy rows stay searchable during the migration window.
4. Empty-state copy — restored explicit "bulk upload".

typecheck + lint clean, 404 tests pass.

---

Generated by Claude Code