feat(curator): monitors-based streaming curator (stages 1+2) — #65#92
Draft
ovidb wants to merge 4 commits into
Draft
feat(curator): monitors-based streaming curator (stages 1+2) — #65#92ovidb wants to merge 4 commits into
ovidb wants to merge 4 commits into
Conversation
…ge 1) Adds `rememora curate --stream`, a long-lived streaming curator that incrementally gates + curates transcript deltas from stdin. Intended as the worker for a future Claude Code `monitors` entry; Stop hook remains the default in stage 1. - `src/stream.rs`: `StreamState` state machine (push-line + flush-on- bytes/idle), token bucket rate limiter, watermark persistence, reader thread driver. - `src/curator.rs`: `Subagent` trait + `DefaultSubagent` production impl so tests inject fakes without calling `claude -p`. - CLI: new `--stream` / `--session` / `--stream-flush-ms` / `--stream-notify-secs` flags with clap `conflicts_with` against the existing curate modes. - Plugin: opt-in `plugin/monitors/monitors.json` + `curate-monitor.sh` (gated on `REMEMORA_USE_MONITOR=1`), plugin version bumped to 1.3.0. 8 unit tests cover state machine, token bucket, watermark round-trip, and clean-EOF reader shutdown. Full implementation plan lives at `.claude/plans/65-monitors-curator.md`.
- `stop-curate.sh`: pre-flight pgrep for `rememora curate --stream` on the current session; short-circuits when the monitor is already serving, preventing double-curate during stage-2 dogfood. - `curate --stream --notify-log <path>`: appends `<rfc3339> <outcome>` lines per flush (outcome ∈ no_signal, curated). Stage-2 metric plumbing; removed alongside `stop-curate.sh` in stage 4.
Unified version bump across VERSION / Cargo.{toml,lock} /
plugin.json / marketplace.json. Release workflow now:
- marks pre-release tags (anything with `-` in the tag name) as GitHub
pre-releases via softprops/action-gh-release@v2
- skips the Homebrew tap update on pre-release tags so the tap's single
stable formula isn't overwritten by an RC
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stages 1 + 2 of the migration from per-turn Stop-hook curator to a long-lived
monitors-based streaming curator. Full plan:.claude/plans/65-monitors-curator.md. Architectural doc: issue #65.rememora curate --stream— long-lived producer that reads session JSONL from stdin, gates + curates transcript deltas incrementally, emits rate-limited notifications via a token bucket.plugin/monitors/monitors.json+plugin/scripts/curate-monitor.sh— Claude Codemonitorsentry wired to the above. Opt-in in this PR viaREMEMORA_USE_MONITOR=1.curator::Subagenttrait (+DefaultSubagent) lets tests stubclaude -pcalls without spending tokens. Production path unchanged.stop-curate.shshort-circuits when a streaming monitor is already running the session, preventing double-curate during dogfood.--notify-logappends<rfc3339> <outcome>per flush for post-hoc volume tallying.Why staged (what's NOT in this PR)
Stages 3 (flip monitor to always-on, remove Stop hook) and 4 (delete
stop-curate.sh) land in follow-up PRs after ≥5 days of dogfood withREMEMORA_USE_MONITOR=1.Structural wins vs post-#63 state
tail | rememorais one pipeline), not apgrep/filesystem dance.Test plan
cargo test --lib(53 passing, includes 9 newstream::tests)cargo clippy --lib --testsclean (pre-existingtests/dir noise is unrelated)REMEMORA_USE_MONITOR=1→ expect"rememora curator online"banner, one notification per curate-worthy delta, ≤1 / 30 ssqlite3 ~/.rememora/rememora.db 'select byte_offset from watermarks where file_path like "stream:%"'advancesRisks / open items
CLAUDE_SESSION_ID,CLAUDE_CWD) may differ from hook env — the wrapper falls back to newest-JSONL-by-mtime if the explicit vars are missing. To be verified in dogfood (risk Local agent loop: auto-dispatch issues to Claude CLI #8 in plan).monitorskey) is based on the plan's educated guess; if Claude Code expects a different shape, stage 2 dogfood catches it and we iterate.tail -Fon rotated logs is assumed to handle Claude Code's session JSONL writes. Dogfood confirms.Closes #65 (partial — stages 1 + 2 only; stages 3 + 4 in follow-up PRs).