Skip to content

feat(store): add recall tracking and promoted observations endpoint#96

Open
sergiomarquezdev wants to merge 1 commit intoGentleman-Programming:mainfrom
sergiomarquezdev:feat/recall-tracking-and-promotion
Open

feat(store): add recall tracking and promoted observations endpoint#96
sergiomarquezdev wants to merge 1 commit intoGentleman-Programming:mainfrom
sergiomarquezdev:feat/recall-tracking-and-promotion

Conversation

@sergiomarquezdev
Copy link

Linked Issue

Closes #95

PR Type

  • New feature
  • Bug fix
  • Documentation
  • Refactoring
  • Tests
  • CI/CD
  • Other

Summary

  • Add read-side instrumentation (recall_count, last_recalled_at) to observations, tracking how often each memory is retrieved via Search and GetObservation
  • Expose a GET /promoted HTTP endpoint and mem_promoted MCP tool that surface frequently-recalled observations above a configurable threshold
  • Enable clients to implement "memory promotion" strategies where high-value observations graduate to permanent context (e.g., system prompt injection)

Changes

File Change
internal/store/store.go Add recall_count/last_recalled_at to schema, struct, all 13 scan sites, migration. New incrementRecall() (fire-and-forget), PromotedObservations(), and private getObservation() (no side effects)
internal/store/store_test.go 4 new tests: recall increment on search, recall increment on get, promoted threshold filtering, deleted observation exclusion
internal/server/server.go GET /promoted endpoint with project, scope, min_recalls, limit query params
internal/mcp/mcp.go mem_promoted tool in agent profile (deferred loading), updated tool counts in package doc
internal/mcp/mcp_test.go Updated tool count assertions (14→15 total, 11→12 agent)

Design Decisions

Fire-and-forget recall tracking: incrementRecall() uses _, _ = s.db.Exec(...) — recall tracking must never break reads. Errors are silently ignored.

Public vs private GetObservation: Split into GetObservation() (increments recall, for external callers) and getObservation() (no side effects, for internal use like Timeline()). This prevents navigational reads from inflating counts.

No sync for recall data: recall_count is analytics metadata, not content. No enqueueSyncMutationTx calls — avoids noise in cross-device sync.

Schema migration: Uses existing addColumnIfNotExists pattern. recall_count defaults to 0 for existing rows, last_recalled_at defaults to NULL. Safe for databases with existing data.

Test Plan

  • TestRecallCountIncrementsOnSearch — verifies Search bumps recall_count
  • TestRecallCountIncrementsOnGetObservation — verifies GetObservation bumps recall_count
  • TestPromotedObservationsReturnsHighRecallOnly — verifies threshold filtering
  • TestPromotedObservationsExcludesDeleted — verifies soft-deleted excluded
  • Manual E2E: built binary, swapped on live system with 13 existing observations, verified zero data loss, recall tracking works, /promoted returns correct results
go test ./internal/store/... -v -run "TestRecall|TestPromoted"
go test ./internal/mcp/... ./internal/server/...

Automated Checks

Check Status
Issue reference
Issue approved
Type label
Unit tests
E2E tests

Contributor Checklist

  • Code follows existing patterns and conventions
  • All scan sites (13) updated consistently
  • Migration is idempotent (addColumnIfNotExists)
  • MCP handler returns ToolResultError, not Go errors
  • Error messages follow "Failed to [verb] [noun]: " pattern
  • Tests use white-box style with newTestStore(t)
  • Commit message follows conventional commit format
  • No external test frameworks added

Notes for Reviewers

  • The only failing test on Windows is TestNewErrorBranches/fails_when_migration_encounters_conflicting_object — a pre-existing Windows file-locking issue unrelated to this PR
  • recall_count is monotonically increasing by design. Staleness filtering (e.g., "not recalled in last 30 days") is a natural follow-up but not in scope for v1
  • The mem_promoted tool uses deferred loading to avoid bloating the initial tool context for agents that don't need it

Add read-side instrumentation to observations: recall_count and
last_recalled_at columns track how often each observation is retrieved
via Search and GetObservation. Frequently recalled observations can be
queried through a new /promoted HTTP endpoint and mem_promoted MCP tool.

- Add recall_count and last_recalled_at to observations schema
- Increment recall_count on Search() and GetObservation() (fire-and-forget)
- Split GetObservation into public (with recall) and private (without)
  to prevent Timeline navigation from inflating counts
- Add PromotedObservations() store method with configurable threshold
- Add GET /promoted HTTP endpoint with project/scope/min_recalls/limit params
- Add mem_promoted MCP tool to agent profile (deferred loading)
- Add 4 tests covering recall increment and promotion filtering

Closes Gentleman-Programming#95
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(store): add recall tracking and promoted observations endpoint

1 participant