Add the AI-editorialized journal (mechanical day layer + cached LLM digests)#213
Merged
Conversation
…igests Implements the two-layer journal (ADR-0023 / SPEC-0016), replacing the `msgbrowse journal` stub. A deterministic per-day mechanical rollup (counts, per-source counts, top senders) is always built from local messages with no egress; an optional LLM digest per day is cached and versioned by (day, model, prompt_version), so a model swap or a journal.digest_prompt edit re-derives the affected days. It mirrors the internal/facts feature. Surfaces: - `msgbrowse journal` (--since/--backfill/--regenerate/--dry-run), incremental and resumable, honoring journal.max_days_per_run and exclude_conversations. - GET /journal: days newest-first with each day's digest or its mechanical summary, keyset infinite scroll, empty state. - Home quick-links restyled into four compact icon tiles (Search, Media, Journal, Status), 4-across desktop / 2x2 mobile. Implementation: - schemaV11 adds day-keyed, FK-less journal_days + journal_digests (same re-ingest rationale as embeddings/contact_facts). Day bucketing is UTC via date(ts_unix,'unixepoch') so legacy iMessage timestamp strings bucket correctly. - internal/journal orchestrates the run; the digest pass is the only network egress and applies the exclude denylist before any transcript is assembled. - --dry-run makes no LLM calls; it estimates input tokens with a char/4 heuristic (no tokenizer/price table exists in the codebase). - Docs: ADR-0023, SPEC-0016 spec pair, README/ARCHITECTURE/SECURITY. Tests cover the migration, the store layer (incl. UTC + legacy-timestamp bucketing), the orchestration (cache hit/miss/stale/regenerate/dry-run/cap/ exclude), the CLI surface, and the web page (full vs boosted render, pagination, empty state, digest escaping). Part of SPEC-0016 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
joestump
force-pushed
the
feature/journal
branch
from
9050d8e to
1b59aa0
Compare
…hlights) Upgrade the journal's LLM layer from a prose body to a STRUCTURED digest. The digest prompt now returns a JSON object (summary / people / themes / mood / highlights / standout_media / notable_links), parsed tolerantly — mirroring the facts JSON contract: coerce every field, unknown mood -> neutral, skip a malformed/empty response like an empty body rather than wedging the run. - schemaV11 amended IN PLACE (safe while unmerged): journal_digests gains `structured` (canonical JSON) + `mood` (denormalized so the calendar/heatmap tint up to 366 day cells with one bounded query). `body` kept as the plain-text summary/fallback + empty-response guard. - config.DefaultDigestPrompt rewritten to the JSON contract. This bumps prompt_version, so every existing prose digest is marked stale and re-derives into the structured shape on the next `msgbrowse journal` run — no --regenerate, no schema bump for the data migration. - internal/journal: Moods enum + isKnownMood + parseDigest / extractJSONObject + Digest/Highlight types; Run stores structured+mood; digest MaxTokens 1024->2048 and Temperature 0.3->0.2 for stable JSON. - store: JournalDigest / PutDayDigest / JournalDayView / ListJournalDays widened. Tests: parseDigest (fences/prose/truncated/unknown-mood/empty-summary), structured round-trip, and a malformed-response run -> Skipped. Green across store/journal/config/web/cli. Next on this PR: calendar store reads (month/heatmap/streak) + the mood calendar navigator + editorial day card UI (Phase 2b/2c). Part of SPEC-0016 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reshape /journal into the redesign's mood-tinted month calendar plus an editorial reading card, consuming the structured digest from Phase 2a. - store (journal_calendar.go): JournalMonth (mood-tinted month grid), JournalStats (longest streak + days-with-entries from the journal_days key set in Go; most-active weekday + peak hour as year-bounded argmax GROUP BYs over messages via a sargable ts_unix range), GetJournalDay, JournalYears, LatestJournalDay(+InYear), longestStreak. - web: /journal is now year tabs + a mood-tinted month grid (tint + count, prev/next, legend) + stat tiles + the editorial day card (summary / highlights / people / themes / standout media / notable links). Navigation is boosted query-param links (?year&month&day); the old /journal/items pagination is gone. Structured fields render html/template-escaped; notable links are text (never a raw href); mood tints are CSS classes (CSP-safe). - stat tiles honor journal.exclude_conversations (wired through the Server). High-effort /code-review fixes folded in: stat tiles now honor the exclude denylist; a year tab opens on that year's latest active month (not an empty January); argmax uses the sargable ts_unix index; the prose fallback keeps its paragraph breaks; dropped the unwired year-heatmap dead code. Verified live against the fixture archive with a real local LLM producing structured digests — the mood tints + full editorial card render end to end. Part of SPEC-0016 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ndar Add REQ-0016-011..016 (structured JSON digest + tolerant parse; mood enum + denormalization; in-place schemaV11 amendment as a free prompt_version-driven migration + the amend caveat; mood-calendar navigator + editorial day card; the calendar read surface; untrusted-field escaping), a supersession note on REQ-0016-010, and a "Phase 2 design decisions" section in design.md. No renumbering. Part of SPEC-0016 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The AI-editorialized journal (ADR-0023 / SPEC-0016), shipped in its final 1.0 form — a mechanical day layer, structured LLM digests, and a mood calendar. Built across three commits on this branch (foundation → structured digest → calendar UI).
Two layers
summary·people·themes·mood·highlights[{text,time}]·standout_media·notable_links), parsed tolerantly (coerce every field, unknown mood →neutral, skip malformed responses). Cached in SQLite, versioned by(day, model, prompt_version)— editing the prompt re-derives every day for free.Surfaces
msgbrowse journal— flags--since/--backfill/--regenerate/--dry-run; incremental + resumable; honorsjournal.exclude_conversations.GET /journal— a mood-tinted month calendar navigator (year tabs · day cells tinted by mood with a message-count subscript · prev/next · legend), stat tiles (days-with-entries · longest streak · most-active weekday · peak hour), and an editorial day card (summary · timed highlights · people · themes · standout media · notable links). Boosted?year&month&daynavigation; the bare page opens on the newest day; a year tab opens on that year's latest active month.Implementation notes
schemaV11: day-keyed, FK-lessjournal_days+journal_digests(bodysummary/fallback +structuredJSON + denormalizedmoodfor the calendar tint). Amended in place (safe while unmerged); see the caveat indesign.md.date(ts_unix,'unixepoch')); the stat argmax uses a sargablets_unixrange (indexed).style=).Review + verification
/code-reviewpasses (structured backend, then calendar UI). They caught and this PR fixes: a stat-tile exclude-list leak, a year-tab-lands-on-empty-January bug, a non-sargable index scan, a lost-paragraph-breaks regression, and unwired dead code — each with a regression test.vet,gofmt, and the CSP no-inline-style guard are green.Note on schema
A DB already at
user_version = 11(a dev who ran an earlier build of this branch) won't retro-gain thestructured/moodcolumns — recreate the derived-cache data dir (the SQLite DB is a rebuildable cache of the read-only archive). Documented indesign.md.Part of SPEC-0016
🤖 Posted on behalf of
@joestumpby Claude.