P1: Consolidate memory implementations into unified interface

[galic1987]

Source

ChatGPT architecture review feedback

Problem

Multiple memory-like implementations with overlapping responsibilities:

• src/cognitive/episodic.rs — episodic memory (past sessions, learnings)
• src/memory/ — vector memory, RAG
• src/session/cache.rs — tool result caching with TTL
• src/session/local_first.rs — response caching, offline support
• src/analysis/vector_store.rs — vector store for code search

This matters because the "local-first, durable, self-improving workshop" story depends on memory being coherent, not fragmented across 5 subsystems.

Proposal

1. Define one MemoryStore trait with clear semantics:

   • store(key, value, metadata)
   • retrieve(query, top_k) (semantic search)
   • get(key) (exact lookup)
   • invalidate(key) / expire()

2. Provide multiple backends behind the trait:

   • JsonMemory — simple JSON file store (current episodic)
   • VectorMemory — HNSW-backed semantic search (current vector_store)
   • CacheMemory — TTL-based with LRU eviction (current cache.rs)

3. Document retention and deletion policies clearly

4. Single entry point: agent.memory.store() / agent.memory.search()

Relevant Code

• src/cognitive/episodic.rs
• src/memory/
• src/session/cache.rs
• src/session/local_first.rs
• src/analysis/vector_store.rs

Priority

P1 — architectural clarity

[m13v]

this consolidation is overdue and the unified MemoryStore trait is the right abstraction. we went through a similar convergence, had embeddings in one place, episodic memory in another, cached results in a third. the pain point was always search - you'd find something via vector search but miss it via keyword, or vice versa.

one thing that simplified our implementation: instead of three separate backends behind the trait, we used a single SQLite database with different tables for each memory type but a unified search path. FTS5 handles keyword search, sqlite-vec handles semantic similarity, and you blend the scores at query time. this way you get hybrid search across all memory types without routing queries to different backends.

for the retention and deletion policies, access-frequency tracking was the biggest win. rather than fixed TTLs per memory type, track how often each entry gets accessed during retrieval. entries that the system actually uses stay warm, unused entries decay naturally. avoids the problem of deleting a cache entry that was actually load-bearing.

[m13v]

our unified memory implementation with a single SQLite database for multiple memory types and hybrid search: https://github.com/m13v/ai-browser-profile/blob/main/ai_browser_profile/db.py - shows the schema design with FTS5 + sqlite-vec in one database. the embedding and search pipeline: https://github.com/m13v/ai-browser-profile/blob/main/ai_browser_profile/embeddings.py