Skip to content

Proposal: a basic conversation Memory component (persistence + recall) for stdlib #1249

@shafiqabedin

Description

@shafiqabedin

Summary

I'd like to contribute a small, dependency-free conversation memory component, and I'm opening
this issue first to get your direction on scope and placement (stdlib vs. mellea-contribs) before
sending a PR.

Today a Mellea session can keep recent turns in ChatContext(window_size=N), but there's no
built-in way to (1) persist a conversation's memory across process restarts, or (2) recall a
relevant fact from earlier than the context window. This proposal adds both, in pure Python, behind
a small interface designed so richer backends can be swapped in later without changing the API.

Motivation / the gaps

  • No cross-session persistence. When the process exits, conversation history is gone — there's no
    save()/load() for a session's accumulated knowledge. (RichDocument has a save/load idiom, but
    nothing equivalent for conversation memory.)
  • No selective recall. ChatContext is a recency window; once a turn slides out, it's
    unrecoverable, even if it's exactly the fact the next prompt needs.

Proposed design

A stable Memory Protocol + a trivial built-in KeywordMemory implementation. retrieve() returns
a mapping that drops directly into instruct(grounding_context=...) — the existing injection seam.

@runtime_checkable
class Memory(Protocol):
    def add(self, text: str, *, role: str = "user", metadata: dict | None = None) -> None: ...
    def ingest_context(self, ctx: Context) -> None: ...          # serialize turns via as_chat_history
    def retrieve(self, query: str, *, k: int = 3) -> dict[str, str | CBlock | Component]: ...
    def save(self, path: str | Path) -> None: ...
    @classmethod
    def load(cls, path: str | Path) -> "Memory": ...

class KeywordMemory:   # V1 default: keyword + recency scoring, no extra dependencies
    ...

Usage (instruct-only, since grounding_context is an instruct() parameter):

mem = KeywordMemory.load(p) if p.exists() else KeywordMemory()
with start_session("ollama", context_type="chat") as s:
    out = s.instruct(user_msg, grounding_context=mem.retrieve(user_msg, k=3))  # recall
    mem.ingest_context(s.ctx)                                                  # capture turns
mem.save(p)                                                                    # persist across restarts

Scope (intentionally minimal for a first PR)

  • In: save/load (JSON, versioned), keyword+recency retrieve() returning a
    grounding_context mapping, ingest_context() using as_chat_history(), unit tests (no LLM),
    one backend-free example.
  • Out / future: embeddings/semantic retrieval, active LLM-driven memory (via @mify/MObject),
    auto-persist via a SESSION_CLEANUP hook, and KV-cache-aware retrieval for LocalHFBackend. The
    retrieve() method is deliberately the single pluggable seam so these can land as follow-ups
    without breaking the API.

Dependencies

None. Pure stdlib (json, pathlib, re). pyproject.toml unchanged.

Compatibility

No changes to existing Mellea APIs or behavior. The component is additive and self-contained
(imports only mellea.core and mellea.stdlib.components.chat).

The question for maintainers

  1. Does this belong in mellea/stdlib, or would you prefer it start in mellea-contribs?
  2. Any preference on naming (Memory/KeywordMemory vs. e.g. ConversationMemory)?
  3. Is the Protocol + swappable-backend shape the right altitude, or do you prefer a single concrete
    class for now?

Happy to follow whichever direction you prefer — the design is self-contained so relocating between
stdlib and contribs is a file move, not a rewrite. I'll open the PR once you've weighed in.

@HendrikStrobelt @nrfulton

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions