Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
3f68952
docs(spec): gap-close design for mockups/flow.html → impl
T-rav May 12, 2026
6440a71
docs(plan): phase 1 plan — schema + ConferenceMigration
T-rav May 12, 2026
ef15085
feat(ios): add Conference entity and Note.conference relationship
T-rav May 13, 2026
03ef2a9
feat(ios): add ChatThread and ChatMessage SwiftData entities
T-rav May 13, 2026
fd86496
feat(ios): add ConferenceMigration to backfill Conference records
T-rav May 13, 2026
239500c
feat(ios): run ConferenceMigration on app launch
T-rav May 13, 2026
731e18d
feat(ios): seed two conferences in sample data
T-rav May 13, 2026
0d2f1f8
fix(ios): correct duration-default assertion in NoteModelTests
T-rav May 13, 2026
569f59a
refactor(ios): introduce hex-arch ports + World composition root
T-rav May 13, 2026
9085fe3
test(ios): add fakes + TestWorld for hex-arch port injection
T-rav May 13, 2026
3465bf5
test(ios): route network-touching tests through TranscriptionPort
T-rav May 13, 2026
7f6dc83
fix(ios): summary fallback for short transcripts; word-count test data
T-rav May 13, 2026
262c6e8
refactor(ios): route production callsites through World ports
T-rav May 13, 2026
63d38b0
docs(plan): phase 2 plan — chat backend
T-rav May 13, 2026
c5d9521
feat(api): chat backend — chatService + talk and multi-session routes
T-rav May 13, 2026
472971f
fix: review findings on Phase 1 + hex arch + Phase 2
T-rav May 13, 2026
f6958f8
docs(plan): phase 3 plan — BlendRenderer + AugmentedNoteView
T-rav May 13, 2026
15da3b6
feat(ios): Phase 3 — BlendRenderer + AugmentedNoteView
T-rav May 13, 2026
7fce1ab
fix(ios): Phase 3 review findings — UTF-16 offsets, logging, error st…
T-rav May 13, 2026
408420c
docs(plan): phase 4 plan — MainView + ConferenceDetailView
T-rav May 13, 2026
690c8be
feat(ios): Phase 4 — MainView + ConferenceDetailView + NoteRow
T-rav May 13, 2026
518ded1
fix(ios): Phase 4 review findings — header tap, archive consistency, …
T-rav May 13, 2026
5f08471
docs(plan): phase 5 plan — ChapteredPlaybackView
T-rav May 13, 2026
c59b0de
feat(ios): Phase 5 — ChapteredPlaybackView
T-rav May 13, 2026
1ec4e2b
fix(ios): Phase 5 review findings — tap-to-seek, audio path, autoplay
T-rav May 13, 2026
8336443
docs(plan): phase 6 plan — ChatView (iOS)
T-rav May 13, 2026
e52fbea
feat(ios): Phase 6 — ChatView (iOS) wired against Phase 2 backend
T-rav May 13, 2026
f10229a
fix(ios): Phase 6 review findings — chat is now actually functional e…
T-rav May 13, 2026
c0a68d4
docs(plan): phases 7-10 combined — polish + Live Activity + cleanup
T-rav May 13, 2026
ebf0a83
feat(ios): Phase 7+8 — recording polish + Live Activity scaffolding
T-rav May 13, 2026
a98d205
chore(ios): Phase 9+10 — delete orphaned views; seed backendSessionId
T-rav May 13, 2026
f9dd0ac
fix(ios): final review findings — chat-button gate + blend status pil…
T-rav May 13, 2026
e71f05f
chore(ios): post-PR follow-ups — BlendingOverlay, renames, tap-to-see…
T-rav May 13, 2026
5a2a331
feat: close all out-of-scope follow-ons (auth, widget target, sendable)
T-rav May 13, 2026
4664226
feat: dev sign-in + auto-refresh — complete the auth loop end-to-end
T-rav May 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,109 changes: 1,109 additions & 0 deletions docs/superpowers/plans/2026-05-12-phase-1-schema-migration.md

Large diffs are not rendered by default.

683 changes: 683 additions & 0 deletions docs/superpowers/plans/2026-05-12-phase-2-chat-backend.md

Large diffs are not rendered by default.

670 changes: 670 additions & 0 deletions docs/superpowers/plans/2026-05-12-phase-3-blend-renderer.md

Large diffs are not rendered by default.

550 changes: 550 additions & 0 deletions docs/superpowers/plans/2026-05-12-phase-4-main-and-conference.md

Large diffs are not rendered by default.

663 changes: 663 additions & 0 deletions docs/superpowers/plans/2026-05-12-phase-5-chaptered-playback.md

Large diffs are not rendered by default.

589 changes: 589 additions & 0 deletions docs/superpowers/plans/2026-05-12-phase-6-chat-view.md

Large diffs are not rendered by default.

65 changes: 65 additions & 0 deletions docs/superpowers/plans/2026-05-12-phase-7-10-polish-and-cleanup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Phases 7-10 (Combined): Polish, Live Activity, Cleanup, Sample Data

> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:executing-plans.

**Goal:** Land the final four phases as one PR-end push:
- **Phase 7** — `NewNoteView` polish + `WaveformView` rework so the recording screen matches the mockup.
- **Phase 8** — ActivityKit Live Activity scaffolding for background recording / Dynamic Island. Includes the `ActivityAttributes` type, `AudioRecordingManager` hooks, and Info.plist background mode. The Live Activity widget extension target itself must be added in Xcode (cannot be created from code); this PR ships the in-app scaffolding so Adding That Target is the only remaining manual step.
- **Phase 9** — Delete the orphaned views replaced by Phases 3-4 (`SimpleMainView`, `SimpleNoteDetailView`, `AISummaryEditorView`, `EnhancedNoteEditorView`, `MyNotesView`). Their tests come along.
- **Phase 10** — Refresh `SampleDataManager` so seeded notes carry a `backendSessionId`. Manual smoke checklist.

**Spec reference:** `docs/superpowers/specs/2026-05-12-gap-close-design.md` § Scenes ii, iii, and § Salvage, Cleanup, Testing.

---

## Phase 7 — Recording polish

**WaveformView:** widen from 5 bars to a denser 24-bar set with mockup-matching heights derived from `audioLevel`. Replace the solid-green fill with a colour that adapts to the system theme.

**NewNoteView controls:** the existing record button is fine; promote the stop affordance to a square `stop.fill` icon for the mockup look. Confirm Pause is still reachable.

Both edits are visual; no unit-test additions (the existing live-update logic is unchanged).

## Phase 8 — Live Activity scaffolding

**Files (production):**
- `src/mobile/Muesli/LiveActivity/RecordingActivityAttributes.swift` — `ActivityAttributes` shared with the widget extension once it's added.
- `src/mobile/Muesli/LiveActivity/LiveActivityController.swift` — `@MainActor` controller that `Activity<RecordingActivityAttributes>.request` on start, `update` every second, `end` on stop. Wraps `if #available(iOS 16.2, *)` and `ActivityAuthorizationInfo().areActivitiesEnabled` so the integration is a no-op when the user has disabled them.
- `src/mobile/Muesli/AudioRecordingManager.swift` — call `LiveActivityController.shared.start/update/end` from the recording lifecycle. Guarded for DEBUG and gracefully degraded if `areActivitiesEnabled == false`.

**Info.plist:** add `UIBackgroundModes` with `audio` so the recording survives backgrounding.

**Cannot do from code:** adding the Widget Extension target itself. The actual `RecordingActivity` Live Activity UI lives in that target; we drop a placeholder doc-comment in `RecordingActivityAttributes.swift` pointing the reader at how to add the target. Until then the controller's `start()` returns gracefully (`areActivitiesEnabled` returns false without a hosting extension).

## Phase 9 — Salvage cleanup

**Delete (after confirming nothing references them):**
- `src/mobile/Muesli/Views/SimpleMainView.swift` (replaced by `MainView`)
- `src/mobile/Muesli/Views/SimpleNoteDetailView.swift` (replaced by `AugmentedNoteView`)
- `src/mobile/Muesli/Views/AISummaryEditorView.swift` (no consumer)
- `src/mobile/Muesli/Views/EnhancedNoteEditorView.swift` (no consumer)
- `src/mobile/Muesli/Views/MyNotesView.swift` (no consumer)
- Matching tests: `MuesliTests/Views/AISummaryEditorViewTests.swift`, `MuesliTests/Views/EnhancedNoteEditorViewTests.swift`, `MuesliTests/Views/NewNoteViewFallbackTests.swift`'s SimpleMain references, `MuesliTests/Views/SimpleMainViewFallbackTests.swift`

After deletion: `grep -r SimpleMainView src/` to confirm no dangling references. Build + run tests to confirm the suite still passes.

## Phase 10 — Sample data + smoke

`SampleDataManager.generateSampleNotes` now sets `backendSessionId = note.id` for every seeded talk so chat works in debug builds against a backend that has matching seed sessions (or just to verify the iOS-side flow with the API stubbed/down).

Manual smoke checklist:
- Cold launch shows `MainView` with `DataSummit 2026` and `DevWorld 2026` sections.
- Tap a talk → AugmentedNoteView renders. (Sample data has no `blendedMarkdown`, so the blend-status fallback shows.)
- Tap conference header → ConferenceDetailView shows hero + talks list + active Chat button.
- Tap Chat → ChatView opens with the scope chip.
- Tap Listen on a note with audio → ChapteredPlaybackView appears.
- Background the app while recording → Dynamic Island banner (only if widget extension target was added; otherwise the app keeps recording but the banner doesn't appear).

---

## Done when

- All four phases committed.
- iOS test suite passes (the deleted view tests should be the only delta).
- Build + lint clean.
- Final cross-task review captures any drift.
287 changes: 287 additions & 0 deletions docs/superpowers/specs/2026-05-12-gap-close-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,287 @@
# Gap Close: Mockup → Implementation

**Date:** 2026-05-12
**Author:** Travis Frisinger (with Claude)
**Status:** Approved for planning
**Delivery shape:** Single PR (per user direction). Internal commit stacking by phase for review legibility.

## Goal

Close the gap between `mockups/flow.html` and the current SwiftUI/Node implementation so all nine scenes of the design ship.

Today the skeleton (notes list, recording, basic detail, image capture) is present, but the high-value scenes (augmented note rendering, conference grouping, chaptered scrubber, chat) are missing or stubbed. The backend AI pipeline (`blendService`, `chapterizeService`, `anthropic.js`) is largely built and `Note` already persists the structured outputs (`blendedMarkdown`, parallel citation arrays, chapters JSON). The remaining work is mostly iOS rendering plus a new chat backend.

## Scope

In scope (every scene in `mockups/flow.html`):

1. Scene i, Notes list grouped by conference
2. Scene ii, Recording UI polish
3. Scene iii, Background recording + Dynamic Island Live Activity
4. Scene iv, In-app camera with captured-slide strip
5. Scene v, Processing/blending overlay state
6. Scene vi, Augmented note view (the flagship)
7. Scene vii, Conference detail
8. Scene viii, Chat sheet (talk + conference scopes, with citations)
9. Scene ix, Chaptered playback scrubber

Out of scope (explicit non-goals, follow-on candidates):

- iCloud sync, multi-day folders, calendar import
- Embedding-based retrieval for chat (token-budget heuristic is the v1)
- Streaming chat responses (single-shot in v1)
- Server-side chat history persistence (client-only via SwiftData)
- User-visible chat cost display (logged server-side only)
- iOS test coverage gate (no gate added in this PR)

## Architecture Overview

### Data model (iOS / SwiftData)

New entities:

```swift
@Model final class Conference {
var id: UUID
var name: String
var location: String?
var startDate: Date?
var endDate: Date?
var conferenceDescription: String?
var createdAt: Date
@Relationship(deleteRule: .nullify, inverse: \Note.conference)
var notes: [Note] = []
}

@Model final class ChatThread {
var id: UUID
var scopeKind: String // "talk" | "conference"
var scopeId: UUID
var createdAt: Date
var updatedAt: Date
@Relationship(deleteRule: .cascade, inverse: \ChatMessage.thread)
var messages: [ChatMessage] = []
}

@Model final class ChatMessage {
var id: UUID
var role: String // "user" | "assistant"
var content: String
var citationsJSON: Data?
var createdAt: Date
var thread: ChatThread?
}
```

Changes to `Note`:

- Add `var speaker: String?`
- Add `var conference: Conference?` relationship
- Keep `var conferenceName: String?` for one release as a fallback. New reads go through `conference?.name`. Removable in a follow-on.

### Schema versioning + migration

Use SwiftData `VersionedSchema` chain `SchemaV1 → SchemaV2`. New entities and new optional fields are additive (lightweight migration).

`ConferenceMigration.swift` runs once at app launch:

1. Query `Note` where `conference == nil && conferenceName != nil`.
2. Group by trimmed, case-insensitive `conferenceName`.
3. For each group, find-or-create a `Conference` with that name. Backfill `startDate` and `endDate` from min/max `note.timestamp`. Other metadata stays nil.
4. Attach `note.conference = conference` for each note in the group.
5. Save. Mark migration complete via `UserDefaults` keyed by schema version so it does not re-run.

Idempotent and additive. The original `conferenceName` stays intact so a botched migration is recoverable.

`SampleDataManager` updated to seed two conferences with multi-talk groupings so debug builds exercise the new screens.

### Backend additions

Two new routes, mounted under existing JWT auth:

```
POST /sessions/:sessionId/chat
POST /conferences/:conferenceId/chat
```

Request: `{ messages: [{ role, content }, ...] }` (full thread, server is stateless).
Response: `{ message, citations[], usage }`.

Citation shape:

```json
{ "kind": "transcript", "talkId": "uuid", "startSec": 612.4, "endSec": 624.1, "label": "10:12" }
{ "kind": "note", "noteId": "uuid", "title": "The three pillars" }
```

`chatService.js` exports `chatTalk(...)` and `chatConference(...)` which delegate to a shared `runChat({ context, messages })`. Context assembly:

- Talk scope: transcript, userNotes, blendedMarkdown, photo OCR summaries for one session.
- Conference scope: for each session under the conference, include a compact summary (title, speaker, date, aiSummary, top OCR snippets). For the 3 most-recent talks (or whatever fits the token budget), include full `blendedMarkdown`. Older talks degrade to summary-only.

System prompt instructs Sonnet to:

- Answer only from supplied context. If unknown, say so.
- Emit citations inline as `[[c:N]]` tokens referencing entries in a `references[]` array it also returns.
- Output strict JSON: `{ answer, references }`.

Server post-processing strips `[[c:N]]` tokens from `answer`, resolves references into the `citations[]` returned to the client (formatting `startSec` as `mm:ss`, resolving note titles), and drops references that fail to resolve.

Guardrails:

- Token budget cap at ~150k input tokens for conference scope.
- `max_tokens` 2000 per turn.
- Rate limit via a new `chatLimiter` key reusing the transcription-tier rate (20 / 15min).
- Authz check confirms the requesting user owns the session or conference before responding.
- Cost tracking via existing `ledgerService` pattern. Not surfaced to users in v1.

### iOS view restructuring

Delete after salvage:

- `SimpleMainView.swift`, `SimpleNoteDetailView.swift`
- `AISummaryEditorView.swift` (salvage edit UX into augmented note overflow sheet)
- `EnhancedNoteEditorView.swift` (salvage formatting toolbar into inline user-notes editor)
- `MyNotesView.swift` (fold pattern into augmented note)

Rename and restyle:

- `SimpleArchiveView` → `ArchiveView`
- `SimpleSettingsView` → `SettingsView`

New views:

- `MainView` (Scene i)
- `ConferenceDetailView` (Scene vii)
- `AugmentedNoteView` (Scene vi)
- `ChapteredPlaybackView` (Scene ix)
- `ChatView` (Scene viii)
- `BlendingOverlay` (Scene v overlay)

Polished in place:

- `NewNoteView` (Scenes ii, iv)
- `WaveformView` (animated bars)

New components:

- `SlideCard`, `ScopeChip`, `CitationChip`, `ChapterScrubber`

Dev-only views (`DebugMenuView`, `DeveloperSettingsView`, `PerformanceView`, `TranscriptView`) stay, but navigation entry points get wrapped in `#if DEBUG`.

### Background recording (Scene iii)

ActivityKit Live Activity with `RecordingAttributes { startedAt, sessionId, title }`.

- Compact leading: red dot. Compact trailing: `mm:ss`.
- Expanded: title, elapsed time, Stop button.
- `AudioRecordingManager` calls `Activity.update(...)` every 1s while recording is active.
- Info.plist gains `UIBackgroundModes: audio` so recording survives backgrounding.
- Tap on Dynamic Island deep-links back to the active `NewNoteView`.

## Component design: Augmented note renderer

This is the flagship and the most subtle piece. Detailed because the renderer is the most testable and most error-prone new code.

Inputs (already persisted on `Note`):

- `blendedMarkdown: String?`
- `blendCitationsJSON: Data?` decodes to `BlendCitations { userNoteSpans, quoteSpans, imagePlacements, citations }`. All four are parallel arrays of char ranges into `blendedMarkdown`.
- `photos: [Photo]` for thumbnail lookup keyed by `photoId` referenced in `imagePlacements`.
- `chaptersJSON: Data?` for "Listen" jump targets.

Render strategy:

1. Parse `blendedMarkdown` into a base `AttributedString` via SwiftUI's markdown initializer.
2. Walk `userNoteSpans` and apply a `.bold()` + accent foreground attribute over each range.
3. Walk `quoteSpans` and apply a quote-block paragraph style (left bar, italic). Attach a custom attribute carrying `startSec` so a tap gesture can read it.
4. Walk `citations` and apply a subtle underline + custom attribute carrying the cited transcript range.
5. For rendering photos at `imagePlacements`, split the `blendedMarkdown` at each offset (ascending) into text segments. Build a `[BlendSegment]` list alternating `.text(AttributedString)` and `.photo(Photo, caption)`. The view body iterates segments in a `VStack` so photos render as full-width cards between paragraphs rather than as inline runs.

Tap handling:

- Tap on a `quoteSpan` or `citation` opens `ChapteredPlaybackView` for this note at `startSec`.
- Photos open `FullscreenImageViewer` (existing component).

Edge cases the renderer must handle (each becomes a unit test):

- `blendedMarkdown` missing or `blendStatus != .complete`: show `BlendingOverlay` or failed state with retry.
- Empty arrays for any of the four parallel structures: render bare markdown without overlays.
- Out-of-range char offsets (defensive against bad model output): clamp and log, do not crash.
- Image placements pointing at a `photoId` no longer in `note.photos`: skip the photo card silently.
- Multiple spans overlapping: last-applied wins; document this in renderer.

## Salvage map

| From | Take | Into |
|---|---|---|
| `AISummaryEditorView` | Summary edit field + save flow | `AugmentedNoteView` overflow → "Edit AI summary" sheet |
| `EnhancedNoteEditorView` | Formatting toolbar component | `AugmentedNoteView` inline `userNotes` editor |
| `MyNotesView` | Personal-notes edit affordance pattern | folded into `AugmentedNoteView` |

## Testing strategy

### API (Jest, existing infra, 70% gate per commit 3a5a0b9)

Unit:

- `chatService.test.js`: context assembly per scope; citation post-processing (token strip + reference resolution); JSON parse failure handling; reference-resolve failure drops the citation; token budget fallback for conference scope. Mock anthropic client like existing `blendService.test.js`.

Integration:

- `chat.routes.test.js`: auth gating (no token → 401), ownership check (wrong user → 403), rate limit hit, happy path with mocked anthropic for both scopes.

### iOS (XCTest)

Unit:

- `BlendRendererTests`: synthetic `blendedMarkdown` + parallel arrays, assert ranges are styled and segment list inserts photos at correct offsets. Covers all edge cases listed above.
- `ConferenceMigrationTests`: fixture of notes with `conferenceName` strings, assert correct `Conference` records created and notes attached. Run migration twice, assert idempotency.
- `ChatServiceTests`: request shape, citation decoding, thread persistence. Mock URLSession.

UI:

- Smoke test through main → conference → note → scrubber → chat sheet. Navigation graph only, no visual assertions.

## Ready-to-ship checklist

Before marking the PR ready for review:

- `./scripts/lint.sh` clean
- `./scripts/test.sh all` green
- `cd src/api && npm test` green at 70%+ coverage
- Manual smoke on simulator covering: record, blend pipeline runs to `.complete`, augmented note renders with overlays, conference grouping shows on main, scrubber seeks and respects chapters, chat works for both scopes with tappable citations, background recording survives backgrounding with Dynamic Island

## Risks and mitigations

| Risk | Mitigation |
|---|---|
| Single mega-PR is hard to review and revert | Stack commits inside the PR by phase so reviewers can walk scene-by-scene. Feature-flag chat routes so prod can disable without revert. |
| SwiftData migration on real user data is destructive if wrong | Migration is additive (keeps `conferenceName` field), idempotent (guarded by UserDefaults key), and write-on-launch (no migration during normal use). |
| Sonnet returning malformed JSON for chat | `chatService` mirrors `blendService` parse-or-throw pattern. Route returns 502 with a user-facing message; iOS surfaces a retry. |
| Conference-scope chat exceeds token budget | Heuristic (summaries + N recent full blends) implemented in `chatService` with explicit cap. Embedding retrieval is the v2 path. |
| Live Activity not supported on older devices | Feature-gate on `ActivityKit.Activity.activitiesEnabled`. Graceful fallback to a plain in-app banner. |
| Augmented note renderer crashes on bad span offsets | Defensive clamping + logging. Unit tests cover overlapping/out-of-range spans. |

## Phased implementation order (within the single PR)

This order minimizes time spent on broken intermediate states.

1. **Schema + migration** (additive, no UI yet)
2. **Backend `chatService` + routes + tests** (independently testable; feature-flagged)
3. **`BlendRenderer` + `AugmentedNoteView`** (highest test value, unblocks most scenes). Salvaged UX from `AISummaryEditorView` and `EnhancedNoteEditorView` lands here.
4. **`MainView` + `ConferenceDetailView`** (depends on schema). Renames of `Simple*Archive`/`Settings` happen here.
5. **`ChapteredPlaybackView`** (depends on renderer's citation taps)
6. **`ChatView` + iOS `ChatService`** (depends on backend)
7. **`NewNoteView` polish + `WaveformView` rework**
8. **Live Activity / Dynamic Island**
9. **Delete orphaned view files** (`AISummaryEditorView`, `EnhancedNoteEditorView`, `MyNotesView`, `SimpleNoteDetailView`, `SimpleMainView`) once their salvage targets are in place
10. **Sample data refresh, manual smoke, ship**

## Open follow-ons (post-merge)

- Remove `Note.conferenceName` after one release.
- Embedding-based retrieval for conference-scope chat as corpus grows.
- Streaming chat responses if perceived latency is an issue.
- iOS code coverage gate.
- Server-side chat persistence + sync.
Loading
Loading