diff --git a/.env.example b/.env.example index 3894dca..b415d11 100644 --- a/.env.example +++ b/.env.example @@ -5,8 +5,8 @@ QDRANT_HTTPS=true QDRANT_API_KEY=replace-me # mem0 core -MEM0_COLLECTION=ian_memories -MEM0_DEFAULT_USER_ID=ian +MEM0_COLLECTION=memories +MEM0_DEFAULT_USER_ID=default-user # LLM for fact extraction MEM0_LLM_PROVIDER=anthropic diff --git a/CLAUDE.md b/CLAUDE.md index a114790..4c793b3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -67,6 +67,7 @@ Dependencies are pinned in `requirements.txt` (per PRD Appendix B); CI is in `.g ## Deployment notes -- Deploy is **push-to-`main` → CapRover webhook**, independent of CI status. The main app is stateless (Phase 1); only Phase 2 OAuth uses the `/app/data` persistent volume. +- Two supported deploy paths, same app image: **CapRover** (connects to an external Qdrant) and **Docker Compose** (`docker-compose.yml`, bundles Qdrant + app for non-CapRover hosts). The compose file overrides `QDRANT_HOST`/`QDRANT_PORT`/`QDRANT_HTTPS` to point at the in-stack `qdrant` service — keep that override if you edit it. Document changes to either path in `docs/USER_GUIDE.md`. +- Deploy (CapRover) is **push-to-`main` → CapRover webhook**, independent of CI status. The main app is stateless (Phase 1); only Phase 2 OAuth uses the `/app/data` persistent volume. - `/healthz` does a real 2s-timeout round-trip to Qdrant and returns 503 if unreachable — keep the timeout so CapRover health checks don't hang. - The `mem0-backup` app lives in this same repo as a second CapRover app (separate `captain-definition` / relative path). It snapshots Qdrant, uploads to S3, prunes by retention. diff --git a/README.md b/README.md index eae3941..da9217e 100644 --- a/README.md +++ b/README.md @@ -3,11 +3,12 @@ A self-hosted [mem0](https://github.com/mem0ai/mem0) memory server that exposes one shared memory store over **two protocols from a single process**: -- **REST** (`/api/v1/memories…`) for scripts, n8n, curl, and the Hermes Agent. +- **REST** (`/api/v1/memories…`) for scripts, n8n, curl, and custom agents. - **Streamable HTTP MCP** (`/mcp/`) for Claude Code, Claude Desktop, Claude.ai web, and Cowork. -It uses an existing external **Qdrant** instance as the vector backend, deploys to **CapRover** -on push to `main`, and ships a companion `mem0-backup` app that snapshots Qdrant to S3 nightly. +It uses **Qdrant** as the vector backend and runs two ways: a **Docker Compose** stack that bundles +Qdrant and the app together, or on **CapRover** (auto-deploys on push to `main`) against an existing +external Qdrant, with a companion `mem0-backup` app that snapshots Qdrant to S3 nightly. This is a **single-user** system: `MEM0_DEFAULT_USER_ID` is the only user. **Documentation:** @@ -20,7 +21,7 @@ The sections below are a quick reference; the guides above go deeper. ## Architecture ``` -Clients (Claude Code / Desktop / Hermes / curl) ──Bearer──┐ +Clients (Claude Code / Desktop / agents / curl) ──Bearer──┐ Anthropic cloud (Claude.ai / Cowork) ──OAuth 2.1 + PKCE──┐ │ ▼ ▼ CapRover app: mem0-server (this repo) @@ -61,12 +62,30 @@ the full list. Key ones: | Variable | Notes | |---|---| | `QDRANT_HOST`, `QDRANT_API_KEY` | external Qdrant instance | -| `MEM0_DEFAULT_USER_ID` | the single user (e.g. `ian`) | +| `MEM0_DEFAULT_USER_ID` | the single user (e.g. `default-user`) | | `MEM0_EMBED_DIMS` | **must** match the embedder's output dim (3-small=1536) | | `MEM0_API_KEY` | static bearer token; `openssl rand -hex 32` | | `PUBLIC_BASE_URL` | public URL, used in OAuth metadata | | `OAUTH_SIGNING_KEY` | PEM RSA private key; setting it enables Phase 2 OAuth | +## Deploy with Docker Compose + +The simplest way to self-host if you don't already run CapRover. The bundled +`docker-compose.yml` brings up **Qdrant and the app together** — no external +Qdrant required. + +```bash +cp .env.example .env # fill in ANTHROPIC_API_KEY, OPENAI_API_KEY, MEM0_API_KEY, QDRANT_API_KEY +docker compose up -d +``` + +The compose file points the app at the in-stack Qdrant automatically (you don't +need to touch `QDRANT_HOST`/`QDRANT_PORT`/`QDRANT_HTTPS`). The server comes up at +`http://localhost:8000` — REST under `/api/v1`, MCP at `/mcp`. For Phase 2 OAuth, +set `OAUTH_SIGNING_KEY` and a public `PUBLIC_BASE_URL` in `.env` and put the stack +behind an HTTPS reverse proxy. See the +[User Guide](docs/USER_GUIDE.md#deploying-with-docker-compose) for details. + ## Production deploy (CapRover) 1. Create the `mem0-server` app. Enable **Has Persistent Data** and map `/app/data` @@ -96,17 +115,22 @@ claude mcp add --scope user --transport http mem0-remote \ ```bash curl -X POST https://mem0.your-domain.com/api/v1/memories \ -H "Authorization: Bearer $MEM0_API_KEY" -H "Content-Type: application/json" \ - -d '{"content": "Ian uses CapRover on DO", "agent_id": "n8n-flow"}' + -d '{"content": "We deploy services with CapRover", "agent_id": "n8n-flow"}' curl -X POST https://mem0.your-domain.com/api/v1/memories/search \ -H "Authorization: Bearer $MEM0_API_KEY" -H "Content-Type: application/json" \ - -d '{"query": "where does Ian host things?"}' + -d '{"query": "how do we deploy services?"}' ``` **Claude.ai web / Cowork (Phase 2):** add a custom connector pointing at `https://mem0.your-domain.com/mcp/`, leave client ID/secret blank (DCR registers automatically), and complete the consent redirect. +**Make agents actually use it:** connecting a client only makes the memory tools available — add a +short instruction block to your `CLAUDE.md` / ChatGPT custom instructions / `AGENTS.md` so the +agent recalls and saves memory every session. Copy-paste snippets are in the +[User Guide](docs/USER_GUIDE.md#prompting-agents-to-use-memory). + A smoke test against a live server is in `scripts/smoke.sh`. ## Restore drill @@ -118,11 +142,11 @@ aws s3 cp s3:///mem0-backups/2026-05-20T03-00-00Z.snapshot ./ # 2. Upload to Qdrant curl -X POST -H "api-key: $QDRANT_API_KEY" \ -F "snapshot=@2026-05-20T03-00-00Z.snapshot" \ - "https://qdrant.your-domain.com/collections/ian_memories/snapshots/upload" + "https://qdrant.your-domain.com/collections/memories/snapshots/upload" # 3. Verify curl -H "api-key: $QDRANT_API_KEY" \ - "https://qdrant.your-domain.com/collections/ian_memories" + "https://qdrant.your-domain.com/collections/memories" ``` ## Troubleshooting diff --git a/app/auth.py b/app/auth.py index 308616f..9bc86e3 100644 --- a/app/auth.py +++ b/app/auth.py @@ -47,16 +47,18 @@ def __init__( static_token: str, jwt_public_key: str | None = None, issuer: str | None = None, + static_client_id: str = "default-user", ): super().__init__() self.static_token = static_token self.jwt_public_key = jwt_public_key self.issuer = issuer + self.static_client_id = static_client_id async def verify_token(self, token: str) -> AccessToken | None: if self.static_token and secrets.compare_digest(token, self.static_token): return AccessToken( - token=token, client_id="ian", scopes=["read", "write"] + token=token, client_id=self.static_client_id, scopes=["read", "write"] ) if not self.jwt_public_key: return None @@ -89,6 +91,7 @@ def build_verifier() -> AuthProvider: static_token=s.mem0_api_key, jwt_public_key=public_key_pem(), issuer=base, + static_client_id=s.mem0_default_user_id, ) # Wrap the verifier so the mounted MCP app advertises the protected # resource metadata URL in the 401 WWW-Authenticate header (RFC 9728). @@ -105,5 +108,10 @@ def build_verifier() -> AuthProvider: scopes_supported=SCOPES, ) return StaticTokenVerifier( - tokens={s.mem0_api_key: {"client_id": "ian", "scopes": ["read", "write"]}} + tokens={ + s.mem0_api_key: { + "client_id": s.mem0_default_user_id, + "scopes": ["read", "write"], + } + } ) diff --git a/app/config.py b/app/config.py index dad4c5d..719e4b8 100644 --- a/app/config.py +++ b/app/config.py @@ -16,7 +16,7 @@ class Settings(BaseSettings): qdrant_api_key: str # mem0 core - mem0_collection: str = "ian_memories" + mem0_collection: str = "memories" mem0_default_user_id: str # LLM (fact extraction) diff --git a/app/oauth.py b/app/oauth.py index ad923b9..febf2b2 100644 --- a/app/oauth.py +++ b/app/oauth.py @@ -297,7 +297,7 @@ def _issue_tokens(client_id: str) -> dict: now = int(time.time()) claims = { "iss": s.public_base_url.rstrip("/"), - "sub": "ian", + "sub": s.mem0_default_user_id, "aud": "mem0-server", "scope": " ".join(SCOPES), "client_id": client_id, diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..72cb119 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,46 @@ +# Self-contained mem0-server stack: Qdrant + the app in one Docker Compose project (two containers). +# For users who don't run CapRover. See docs/USER_GUIDE.md → "Deploying with Docker Compose". +# +# cp .env.example .env # then fill in the API keys +# docker compose up -d +# +# The app is reachable at http://localhost:8000 (REST under /api/v1, MCP at /mcp). + +services: + qdrant: + # Pinned for reproducible builds; aligned with the qdrant-client version in + # requirements.txt. Bump both together when upgrading Qdrant. + image: qdrant/qdrant:v1.18.0 + restart: unless-stopped + environment: + # Qdrant enforces this key when set; the app sends it via QDRANT_API_KEY. + QDRANT__SERVICE__API_KEY: ${QDRANT_API_KEY:?set QDRANT_API_KEY in .env} + volumes: + - qdrant_data:/qdrant/storage + # Uncomment to inspect Qdrant from the host (dashboard at :6333/dashboard). + # ports: + # - "6333:6333" + + mem0-server: + build: . + restart: unless-stopped + depends_on: + - qdrant + env_file: + - .env + # These override whatever is in .env so the app always talks to the + # in-stack Qdrant over the internal network, not an external host. + environment: + QDRANT_HOST: qdrant + QDRANT_PORT: "6333" + QDRANT_HTTPS: "false" + PUBLIC_BASE_URL: ${PUBLIC_BASE_URL:-http://localhost:8000} + ports: + - "8000:8000" + volumes: + # Persists the Phase 2 OAuth SQLite DB across restarts. + - mem0_data:/app/data + +volumes: + qdrant_data: + mem0_data: diff --git a/docs/DEVELOPER_GUIDE.md b/docs/DEVELOPER_GUIDE.md index de145a5..5db0774 100644 --- a/docs/DEVELOPER_GUIDE.md +++ b/docs/DEVELOPER_GUIDE.md @@ -84,6 +84,7 @@ scripts/ smoke.sh (REST) and smoke_mcp.py (MCP) against a live server docs/ PRD.md (spec), USER_GUIDE.md, DEVELOPER_GUIDE.md. Dockerfile Main app image; runs uvicorn with --workers 2. captain-definition CapRover build descriptor for the main app. +docker-compose.yml Self-contained stack (Qdrant + app) for non-CapRover hosts. ``` ## Request and data flow @@ -225,8 +226,13 @@ reads the `Authorization` header, so tokens are never logged. - **CI** (`.github/workflows/ci.yml`) runs on pushes to `main` and on all PRs: installs deps, then `ruff check app/` and `pytest -q` on Python 3.12. -- **Deploy** is push-to-`main` → CapRover webhook, independent of CI status. The main app builds - from the root `Dockerfile` / `captain-definition` and runs `uvicorn app.main:app --workers 2`. +- **Deploy** has two supported paths (same app image, different infrastructure): + - **CapRover** — push-to-`main` → CapRover webhook, independent of CI status. The main app builds + from the root `Dockerfile` / `captain-definition` and runs `uvicorn app.main:app --workers 2`. + Connects to an external Qdrant. + - **Docker Compose** — `docker-compose.yml` builds the same `Dockerfile` and brings up the app + alongside a bundled Qdrant service, overriding `QDRANT_HOST`/`QDRANT_PORT`/`QDRANT_HTTPS` to the + in-stack service. See the [User Guide](USER_GUIDE.md#deploying-with-docker-compose). - The **backup app** is a second CapRover app built from `backup/` (separate `captain-definition`). See the [User Guide](USER_GUIDE.md#2-deploy-the-backup-app-mem0-backup). - The main app is stateless in Phase 1; only Phase 2 OAuth uses the `/app/data` persistent volume diff --git a/docs/PRD.md b/docs/PRD.md index 79fc5c5..e3de042 100644 --- a/docs/PRD.md +++ b/docs/PRD.md @@ -1,10 +1,10 @@ # PRD: Self-Hosted mem0 Memory Server **Project name:** `mem0-server` -**Owner:** Ian Monroe -**Status:** Ready for implementation -**Target deployment:** CapRover on Digital Ocean (existing droplet) -**Existing infrastructure:** Qdrant already deployed on CapRover at `qdrant.` with API key auth and HTTPS. +**Owner:** project maintainer +**Status:** Implemented (Phase 1 + Phase 2) +**Target deployment:** CapRover, or Docker Compose on any Docker host +**Vector backend:** An external Qdrant instance (API key auth + HTTPS), or the Qdrant bundled in the Docker Compose stack. --- @@ -15,10 +15,10 @@ Build and deploy a single self-hosted Python service that exposes a shared mem0 ### 1.1 Goals 1. **One memory pool, many agents.** All AI clients write to and read from the same Qdrant collection, scoped by a shared `user_id` and tagged by `agent_id`. -2. **One server, two protocols.** REST for scripts, n8n, Hermes Agent, and ad-hoc curl. Streamable HTTP MCP for Claude Code, Claude Desktop, Claude.ai web, and Cowork. +2. **One server, two protocols.** REST for scripts, n8n, custom agents, and ad-hoc curl. Streamable HTTP MCP for Claude Code, Claude Desktop, Claude.ai web, and Cowork. 3. **Push-to-deploy.** Merges to `main` trigger CapRover to pull, build, and redeploy without manual intervention. -4. **Operational durability.** Nightly Qdrant snapshots are uploaded to S3. Droplet loss does not equal memory loss. -5. **Multi-client, multi-machine.** The server is the single source of truth; clients on any of Ian's machines connect to it over HTTPS. +4. **Operational durability.** Nightly Qdrant snapshots are uploaded to S3. Host loss does not equal memory loss. +5. **Multi-client, multi-machine.** The server is the single source of truth; clients on any of your machines connect to it over HTTPS. ### 1.2 Non-goals @@ -33,7 +33,7 @@ Build and deploy a single self-hosted Python service that exposes a shared mem0 |---|---|---|---| | Claude Code (CLI) | Streamable HTTP MCP | Bearer token (header) | 1 | | Claude Desktop | Streamable HTTP MCP | Bearer token (header, via Advanced Settings) | 1 | -| Hermes Agent | REST or MCP | Bearer token | 1 | +| Custom agents / SDK clients | REST or MCP | Bearer token | 1 | | Direct REST (scripts, n8n, curl) | REST | Bearer token | 1 | | Claude.ai (web) | Streamable HTTP MCP | OAuth 2.1 + PKCE + DCR | 2 | | Cowork | Streamable HTTP MCP | OAuth 2.1 + PKCE + DCR | 2 | @@ -48,7 +48,7 @@ Build and deploy a single self-hosted Python service that exposes a shared mem0 ┌────────────────────────────────────────────────────────────────────┐ │ Client machines │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌────────────┐ │ -│ │ Claude Code │ │Claude Desktop│ │Hermes Agent │ │curl / n8n │ │ +│ │ Claude Code │ │Claude Desktop│ │Custom agent │ │curl / n8n │ │ │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └─────┬──────┘ │ └─────────┼────────────────┼────────────────┼───────────────┼────────┘ │ Bearer │ Bearer │ Bearer │ Bearer @@ -86,15 +86,15 @@ Build and deploy a single self-hosted Python service that exposes a shared mem0 ▼ ┌────────────────────────────────────────────────────────────────────┐ │ CapRover app: qdrant (existing) │ -│ Collection: ian_memories (vectors + payload) │ +│ Collection: memories (vectors + payload) │ └────────────────────────────────┬───────────────────────────────────┘ │ Snapshot API ▼ ┌────────────────────────────────────────────────────────────────────┐ │ CapRover app: mem0-backup (this repo) │ │ Cron @ 03:00 UTC daily │ -│ ├── POST /collections/ian_memories/snapshots (create) │ -│ ├── GET /collections/ian_memories/snapshots/{name} (download) │ +│ ├── POST /collections/memories/snapshots (create) │ +│ ├── GET /collections/memories/snapshots/{name} (download) │ │ └── Upload to s3:///mem0-backups/{date}.snapshot │ │ Retention: keep last 14 days in S3, last 3 on local volume │ └────────────────────────────────────────────────────────────────────┘ @@ -154,20 +154,22 @@ mem0-server/ ## 4. Tech stack & dependencies - **Python 3.12** -- **FastAPI** ≥ 0.115 — HTTP framework -- **FastMCP** ≥ 2.12 — MCP server framework. *Note: this is the PrefectHQ/fastmcp package on PyPI, not the older `mcp.server.fastmcp` module. Import as `from fastmcp import FastMCP`.* -- **uvicorn[standard]** ≥ 0.30 — ASGI server -- **mem0ai** ≥ 0.1.100 — memory layer -- **qdrant-client** ≥ 1.13 — pulled in by mem0ai but pin explicitly -- **openai** ≥ 1.50 — embeddings provider client (used by mem0) -- **anthropic** ≥ 0.39 — LLM provider client (used by mem0 for fact extraction) -- **pydantic** ≥ 2.7 & **pydantic-settings** ≥ 2.4 — config -- **structlog** ≥ 24 — structured logging -- **httpx** ≥ 0.27 — used in tests and OAuth endpoints -- **PyJWT[crypto]** ≥ 2.9 — Phase 2 OAuth token signing +- **FastAPI** — HTTP framework +- **FastMCP** (v3) — MCP server framework. *Note: this is the PrefectHQ/fastmcp package on PyPI, not the older `mcp.server.fastmcp` module. Import as `from fastmcp import FastMCP`.* +- **mcp** — the MCP SDK (provides `AccessToken` and related types used by the auth verifier) +- **uvicorn[standard]** — ASGI server +- **mem0ai** (v2) — memory layer +- **qdrant-client** — pulled in by mem0ai but pin explicitly +- **openai** — embeddings provider client (used by mem0) +- **anthropic** — LLM provider client (used by mem0 for fact extraction) +- **pydantic** & **pydantic-settings** — config +- **structlog** — structured logging +- **prometheus-client** — `/metrics` exposition +- **httpx** — used in tests and the Qdrant health check +- **PyJWT[crypto]** — Phase 2 OAuth token signing - **pytest**, **pytest-asyncio**, **respx**, **ruff** — dev tools -Pin via `requirements.txt` with `==` for runtime, and use `pyproject.toml` for dev tools and project metadata. +Runtime dependencies are pinned with `==` in `requirements.txt` (see Appendix B for the exact versions); dev tools and project metadata live in `pyproject.toml`. --- @@ -183,8 +185,8 @@ All configuration is via environment variables. The `Settings` class in `app/con | `QDRANT_PORT` | no | `443` | | | `QDRANT_HTTPS` | no | `true` | | | `QDRANT_API_KEY` | yes | — | Key configured on Qdrant | -| `MEM0_COLLECTION` | no | `ian_memories` | | -| `MEM0_DEFAULT_USER_ID` | yes | — | e.g. `ian`. Used as fallback when clients don't supply one. | +| `MEM0_COLLECTION` | no | `memories` | | +| `MEM0_DEFAULT_USER_ID` | yes | — | e.g. `default-user`. Used as fallback when clients don't supply one. | | `MEM0_LLM_PROVIDER` | no | `anthropic` | mem0 LLM for fact extraction | | `MEM0_LLM_MODEL` | no | `claude-haiku-4-5-20251001` | | | `ANTHROPIC_API_KEY` | conditional | — | Required if `MEM0_LLM_PROVIDER=anthropic` | @@ -218,7 +220,7 @@ class Settings(BaseSettings): qdrant_api_key: str # mem0 core - mem0_collection: str = "ian_memories" + mem0_collection: str = "memories" mem0_default_user_id: str # LLM (fact extraction) @@ -490,7 +492,7 @@ def build_verifier(): s = get_settings() return StaticTokenVerifier( tokens={ - s.mem0_api_key: {"client_id": "ian", "scopes": ["read", "write"]} + s.mem0_api_key: {"client_id": s.mem0_default_user_id, "scopes": ["read", "write"]} } ) ``` @@ -511,7 +513,7 @@ class CompositeVerifier(TokenVerifier): async def verify_token(self, token: str): if token == self.static_token: - return {"client_id": "ian", "scopes": ["read", "write"]} + return {"client_id": settings.mem0_default_user_id, "scopes": ["read", "write"]} try: payload = jwt.decode( token, self.jwt_public_key, algorithms=["RS256"], @@ -587,7 +589,7 @@ Required by Claude.ai web and Cowork. The MCP server itself acts as the OAuth Au ```json { "iss": "https://mem0.your-domain.com", - "sub": "ian", + "sub": "", "aud": "mem0-server", "scope": "read write", "client_id": "", @@ -830,11 +832,11 @@ aws s3 cp s3:///mem0-backups/2026-05-20T03-00-00Z.snapshot ./ curl -X POST \ -H "api-key: $QDRANT_API_KEY" \ -F "snapshot=@2026-05-20T03-00-00Z.snapshot" \ - "https://qdrant.your-domain.com/collections/ian_memories/snapshots/upload" + "https://qdrant.your-domain.com/collections/memories/snapshots/upload" # 3. Verify curl -H "api-key: $QDRANT_API_KEY" \ - "https://qdrant.your-domain.com/collections/ian_memories" + "https://qdrant.your-domain.com/collections/memories" ``` A monthly restore drill (restore to a test collection, verify count) is strongly recommended but not part of v1 automation. @@ -923,13 +925,13 @@ Verify with `claude mcp list`. The server should show as connected with 6 tools. Same as Claude.ai web — go to Connectors, add the custom URL, complete OAuth. -### 14.5 Hermes Agent +### 14.5 Custom agents / SDK clients Two options: -**a) REST**: configure Hermes with base URL `https://mem0.your-domain.com/api/v1` and bearer header. +**a) REST**: configure the agent with base URL `https://mem0.your-domain.com/api/v1` and a bearer header. -**b) MCP**: configure Hermes' MCP client (Hermes Agent supports Streamable HTTP MCP servers) with the same URL + headers as Claude Code in §14.1. +**b) MCP**: if the agent's framework supports Streamable HTTP MCP servers, point its MCP client at the same URL + headers as Claude Code in §14.1. ### 14.6 Direct REST (curl / scripts / n8n) @@ -941,13 +943,13 @@ export MEM0_API_KEY=... curl -X POST $MEM0_URL/api/v1/memories \ -H "Authorization: Bearer $MEM0_API_KEY" \ -H "Content-Type: application/json" \ - -d '{"content": "Ian uses CapRover on DO", "agent_id": "n8n-flow"}' + -d '{"content": "We deploy with CapRover on DO", "agent_id": "n8n-flow"}' # Search curl -X POST $MEM0_URL/api/v1/memories/search \ -H "Authorization: Bearer $MEM0_API_KEY" \ -H "Content-Type: application/json" \ - -d '{"query": "where does Ian host things?"}' + -d '{"query": "where do we host things?"}' ``` --- @@ -1089,8 +1091,8 @@ QDRANT_HTTPS=true QDRANT_API_KEY=replace-me # mem0 core -MEM0_COLLECTION=ian_memories -MEM0_DEFAULT_USER_ID=ian +MEM0_COLLECTION=memories +MEM0_DEFAULT_USER_ID=default-user # LLM for fact extraction MEM0_LLM_PROVIDER=anthropic @@ -1118,21 +1120,23 @@ LOG_LEVEL=INFO ## Appendix B: `requirements.txt` ``` -fastapi==0.115.0 -uvicorn[standard]==0.30.6 -fastmcp==2.12.5 -mem0ai==0.1.100 -qdrant-client==1.13.0 -openai==1.50.0 -anthropic==0.39.0 -pydantic==2.7.4 -pydantic-settings==2.4.0 -structlog==24.4.0 -httpx==0.27.2 -PyJWT[crypto]==2.9.0 +fastapi==0.136.1 +uvicorn[standard]==0.47.0 +fastmcp==3.3.1 +mcp==1.27.1 +mem0ai==2.0.2 +qdrant-client==1.18.0 +openai==2.38.0 +anthropic==0.104.1 +pydantic==2.13.4 +pydantic-settings==2.14.1 +structlog==25.5.0 +prometheus-client==0.25.0 +httpx==0.28.1 +PyJWT[crypto]==2.13.0 ``` -(Adjust to the latest patch versions at implementation time; the constraints above are the minimum tested.) +(This mirrors the repository's `requirements.txt`, which is the authoritative pin list.) ## Appendix C: Useful CapRover CLI snippets diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md index 2cae0a1..ec49dd2 100644 --- a/docs/USER_GUIDE.md +++ b/docs/USER_GUIDE.md @@ -8,6 +8,8 @@ connect clients to it, and use it day to day. If you want to work on the code it - [How memory works](#how-memory-works) - [Prerequisites](#prerequisites) - [Configuration reference](#configuration-reference) +- [Choosing a deployment method](#choosing-a-deployment-method) +- [Deploying with Docker Compose](#deploying-with-docker-compose) - [Deploying to CapRover](#deploying-to-caprover) - [1. Deploy the main app](#1-deploy-the-main-app-mem0-server) - [2. Deploy the backup app](#2-deploy-the-backup-app-mem0-backup) @@ -17,6 +19,10 @@ connect clients to it, and use it day to day. If you want to work on the code it - [Claude.ai web / Cowork (OAuth)](#claudeai-web--cowork-oauth) - [ChatGPT (OAuth, Developer Mode)](#chatgpt-oauth-developer-mode) - [REST / curl / n8n](#rest--curl--n8n) +- [Prompting agents to use memory](#prompting-agents-to-use-memory) + - [Claude (CLAUDE.md)](#claude-claudemd) + - [ChatGPT (custom instructions)](#chatgpt-custom-instructions) + - [Other agents (AGENTS.md and similar)](#other-agents-agentsmd-and-similar) - [REST API reference](#rest-api-reference) - [Backups and restore](#backups-and-restore) - [Health and monitoring](#health-and-monitoring) @@ -61,12 +67,12 @@ Before deploying you need: | Requirement | Why | |---|---| -| A **CapRover** instance | Hosts the app; deploys on push to `main`. | -| A reachable **Qdrant** instance (with API key) | Vector backend that stores the memories. | +| **Docker** + Docker Compose, **or** a **CapRover** instance | Runs the app. See [Choosing a deployment method](#choosing-a-deployment-method). | +| A reachable **Qdrant** instance (with API key) | Vector backend that stores the memories. The Docker Compose method provides this for you. | | An **Anthropic API key** | Default LLM for fact extraction. | | An **OpenAI API key** | Default embedding model. | -| An **S3 bucket** + AWS credentials | Only for the nightly backup app. | -| A **domain/subdomain** (e.g. `mem0.your-domain.com`) | Public HTTPS URL for clients and OAuth. | +| An **S3 bucket** + AWS credentials | Only for the nightly backup app (CapRover). | +| A **domain/subdomain** (e.g. `mem0.your-domain.com`) | Public HTTPS URL for clients and OAuth. Optional for a local Docker Compose run. | You can swap the LLM/embedder providers (see [Configuration reference](#configuration-reference)), but the defaults above are the supported path. @@ -88,8 +94,8 @@ runs, or set these in the CapRover app's **App Configs** panel for production. | `QDRANT_PORT` | no | `443` | Qdrant port. | | `QDRANT_HTTPS` | no | `true` | Use HTTPS to reach Qdrant. | | `QDRANT_API_KEY` | yes | — | Qdrant API key. | -| `MEM0_COLLECTION` | no | `ian_memories` | Qdrant collection name. | -| `MEM0_DEFAULT_USER_ID` | yes | — | The single user, e.g. `ian`. | +| `MEM0_COLLECTION` | no | `memories` | Qdrant collection name. | +| `MEM0_DEFAULT_USER_ID` | yes | — | The single user, e.g. `default-user`. | | `MEM0_LLM_PROVIDER` | no | `anthropic` | LLM provider for fact extraction. | | `MEM0_LLM_MODEL` | no | `claude-haiku-4-5-20251001` | LLM model. | | `ANTHROPIC_API_KEY` | if provider=anthropic | — | Required when the LLM provider is Anthropic. | @@ -120,8 +126,77 @@ openssl genrsa 2048 When pasting a multi-line PEM into a single env var, replace newlines with `\n` — the app converts `\n` back to real newlines at load time. +## Choosing a deployment method + +There are two supported ways to run mem0-server: + +- **Docker Compose** — the simplest path if you don't already run CapRover. One `docker compose up` + brings up **both Qdrant and the app** on a single host, with persistent volumes for each. You + manage your own HTTPS (typically via a reverse proxy) and your own backups. Best for a single VM, + a homelab, or local use. +- **CapRover** — best if you already operate a CapRover instance and want push-to-`main` + auto-deploy plus the companion nightly S3 backup app. This method connects to an **existing, + external** Qdrant. + +The application is identical in both cases; only the surrounding infrastructure differs. The +sections below cover each. + +## Deploying with Docker Compose + +The repository ships a `docker-compose.yml` that runs Qdrant and the app together. You do **not** +need an external Qdrant for this method. + +1. Copy the example environment file and fill in the secrets: + + ```bash + cp .env.example .env + ``` + + At minimum set: `MEM0_API_KEY` (generate with `openssl rand -hex 32`), `QDRANT_API_KEY` (any + strong secret — the bundled Qdrant is configured to require it), `ANTHROPIC_API_KEY`, + `OPENAI_API_KEY`, and `MEM0_DEFAULT_USER_ID`. + + You can leave `QDRANT_HOST`, `QDRANT_PORT`, and `QDRANT_HTTPS` at their `.env.example` values — + the compose file overrides them to point at the in-stack Qdrant service (`qdrant:6333`, no TLS + on the internal network). + +2. Bring up the stack: + + ```bash + docker compose up -d + ``` + + This builds the app image from the root `Dockerfile`, starts Qdrant with a persistent + `qdrant_data` volume, and starts the app on `http://localhost:8000`. The app's `/healthz` + endpoint round-trips to Qdrant; once it returns `{"ok": true, ...}` the stack is ready. + +3. Verify: + + ```bash + curl http://localhost:8000/healthz + ``` + +**HTTPS and public access.** The compose stack serves plain HTTP on port 8000. MCP clients and +OAuth require HTTPS, so for anything beyond local use put the app behind a reverse proxy +(Caddy, nginx, Traefik) that terminates TLS, and set `PUBLIC_BASE_URL` in `.env` to the public +HTTPS URL (e.g. `https://mem0.your-domain.com`). For Phase 2 OAuth, also set `OAUTH_SIGNING_KEY` +(see [Phases](#phases)). + +**Backups.** The nightly S3 backup app is part of the CapRover setup. With Docker Compose you can +take Qdrant snapshots yourself against the bundled instance — see +[Backups and restore](#backups-and-restore) for the snapshot/restore API; the `qdrant_data` volume +also holds the on-disk data. + +**Updating.** Pull the latest code and rebuild: + +```bash +git pull +docker compose up -d --build +``` + ## Deploying to CapRover +This method connects to an **existing, external** Qdrant (it does not start one for you). Deployment is **push-to-`main` → CapRover webhook**. Merging to `main` triggers a rebuild and redeploy automatically, independent of CI status. @@ -225,6 +300,85 @@ a typo can't accidentally allow a lookalike host such as `chatgpt.com.evil.com`. Send the bearer token as an `Authorization` header. See the [REST API reference](#rest-api-reference) below. +## Prompting agents to use memory + +Connecting a client only makes the memory tools *available* — it does not make the agent *use* +them. Models won't reliably search or save memory on their own; you have to tell them to. The most +durable way is to put a short instruction block in whatever file the agent reads at the start of +every session (`CLAUDE.md`, ChatGPT custom instructions, `AGENTS.md`, a system prompt, etc.). + +A good memory instruction covers four behaviors: + +1. **Recall first** — search memory at the start of a task, before answering, so past context is used. +2. **Save durable facts** — persist preferences, decisions, project conventions, and recurring + context as they come up (not transient chatter). +3. **Update, don't duplicate** — when something changes, update the existing memory instead of + adding a near-duplicate. +4. **Don't store secrets** — never save passwords, API keys, or sensitive personal data. + +The server exposes six tools: `search_memories`, `add_memory`, `list_memories`, `get_memory`, +`update_memory`, `delete_memory`. Adjust the tool/connector names below to match how your client +surfaces them (for example, Claude Code namespaces them like `mcp__mem0-remote__search_memories`). + +### Claude (CLAUDE.md) + +For **Claude Code**, add this to the project's `CLAUDE.md` (or your user-level +`~/.claude/CLAUDE.md` to apply it everywhere). For **Claude Desktop**, paste the same text into a +Project's custom instructions. + +```markdown +## Long-term memory (mem0) + +You have a persistent memory store available through the mem0 MCP server. Use it in every session: + +- **At the start of a task**, call `search_memories` with a query about the topic to recall any + relevant preferences, decisions, or context before you respond. +- **When the user shares** a durable preference, decision, project convention, or fact they'll + likely want recalled later, call `add_memory` to save it. Keep each memory a single clear fact. +- **When something changes**, find the existing memory (`search_memories` / `list_memories`) and + `update_memory` it instead of adding a duplicate. +- Do **not** store secrets, credentials, or sensitive personal data. +- You don't need to announce routine memory operations; just use them naturally. +``` + +### ChatGPT (custom instructions) + +In ChatGPT, open **Settings → Personalization → Custom instructions** (or a Project's +instructions) and add the following to the "How would you like ChatGPT to respond?" box. This +assumes you've connected the mem0 connector in Developer Mode (see +[ChatGPT (OAuth, Developer Mode)](#chatgpt-oauth-developer-mode)). + +```text +I have a personal long-term memory store connected via the mem0 MCP connector. Use it every session: +- Before answering a substantive question, use the connector's search_memories tool to recall any + relevant saved preferences, decisions, or context. +- When I share a durable preference, decision, or fact worth remembering, use add_memory to save it + as a single clear statement. +- If something changes, update the existing memory rather than creating a duplicate. +- Never store passwords, API keys, or sensitive personal data. +``` + +### Other agents (AGENTS.md and similar) + +Many coding agents and frameworks read an `AGENTS.md` (or an equivalent system-prompt/rules file) +at session start. Drop in a tool-agnostic version: + +```markdown +## Memory + +A shared long-term memory store is available via the mem0 MCP server. Behavior: + +1. Recall: at the start of a task, search memory for context relevant to the request before acting. +2. Persist: save durable facts, preferences, decisions, and conventions as they arise. +3. Reconcile: update an existing memory when it changes; avoid near-duplicates. +4. Safety: never store secrets, credentials, or sensitive personal data. + +Tools: search_memories, add_memory, list_memories, get_memory, update_memory, delete_memory. +``` + +If your agent has no instruction file but does take a system prompt, the same four numbered rules +work verbatim there. + ## REST API reference All endpoints live under `/api/v1` and require `Authorization: Bearer `. Request and @@ -238,7 +392,7 @@ Provide **either** `content` (a string) **or** `messages` (a chat transcript). O ```bash curl -X POST https://mem0.your-domain.com/api/v1/memories \ -H "Authorization: Bearer $MEM0_API_KEY" -H "Content-Type: application/json" \ - -d '{"content": "Ian hosts services on CapRover on DigitalOcean", "agent_id": "n8n-flow"}' + -d '{"content": "We host services on CapRover on DigitalOcean", "agent_id": "n8n-flow"}' ``` With a transcript instead of a plain string: @@ -256,7 +410,7 @@ Semantic search. Optional `agent_id`, `run_id`, `user_id`, and `limit` (1–100, ```bash curl -X POST https://mem0.your-domain.com/api/v1/memories/search \ -H "Authorization: Bearer $MEM0_API_KEY" -H "Content-Type: application/json" \ - -d '{"query": "where does Ian host things?"}' + -d '{"query": "where do we host things?"}' ``` ### List memories — `GET /api/v1/memories` @@ -299,11 +453,11 @@ aws s3 cp s3:///mem0-backups/2026-05-20T03-00-00Z.snapshot ./ # 2. Upload it to Qdrant curl -X POST -H "api-key: $QDRANT_API_KEY" \ -F "snapshot=@2026-05-20T03-00-00Z.snapshot" \ - "https://qdrant.your-domain.com/collections/ian_memories/snapshots/upload" + "https://qdrant.your-domain.com/collections/memories/snapshots/upload" # 3. Verify the collection is back curl -H "api-key: $QDRANT_API_KEY" \ - "https://qdrant.your-domain.com/collections/ian_memories" + "https://qdrant.your-domain.com/collections/memories" ``` Run a restore drill periodically so you know the snapshots are usable before you need them. diff --git a/scripts/smoke.sh b/scripts/smoke.sh index 777ae80..8912e69 100755 --- a/scripts/smoke.sh +++ b/scripts/smoke.sh @@ -15,17 +15,17 @@ curl -fsS "$MEM0_URL/healthz" && echo echo "2. Add a memory via REST" curl -fsS -X POST "$MEM0_URL/api/v1/memories" \ -H "$AUTH" -H "$JSON" \ - -d '{"content": "smoke-test: Ian hosts services on CapRover", "agent_id": "smoke"}' && echo + -d '{"content": "smoke-test: the team hosts services on CapRover", "agent_id": "smoke"}' && echo echo "3. Search for it via REST" curl -fsS -X POST "$MEM0_URL/api/v1/memories/search" \ -H "$AUTH" -H "$JSON" \ - -d '{"query": "where does Ian host services?", "agent_id": "smoke"}' && echo + -d '{"query": "where are services hosted?", "agent_id": "smoke"}' && echo echo "4. List memories" curl -fsS "$MEM0_URL/api/v1/memories?agent_id=smoke" -H "$AUTH" && echo echo "5-6. MCP add + search (cross-checks REST-added fact via MCP)" -python3 "$(dirname "$0")/smoke_mcp.py" "where does Ian host services?" +python3 "$(dirname "$0")/smoke_mcp.py" "where are services hosted?" echo "Smoke test complete. Review output above; clean up test memories via DELETE if desired." diff --git a/scripts/smoke_mcp.py b/scripts/smoke_mcp.py index fd6d465..b1307f8 100755 --- a/scripts/smoke_mcp.py +++ b/scripts/smoke_mcp.py @@ -39,12 +39,12 @@ async def main() -> int: await client.call_tool( "add_memory", - {"content": "smoke-test (mcp): Ian deploys via CapRover", "agent_id": "smoke-mcp"}, + {"content": "smoke-test (mcp): the team deploys via CapRover", "agent_id": "smoke-mcp"}, ) print(" add_memory via MCP: ok") res = await client.call_tool( - "search_memories", {"query": "how does Ian deploy?", "agent_id": "smoke-mcp"} + "search_memories", {"query": "how does the team deploy?", "agent_id": "smoke-mcp"} ) print(f"5. search_memories via MCP returned: {res.data}") if not _results(res): diff --git a/tests/conftest.py b/tests/conftest.py index 5f47640..9a2011f 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -10,7 +10,7 @@ "QDRANT_HTTPS": "true", "QDRANT_API_KEY": "test-qdrant-key", "MEM0_COLLECTION": "test_memories", - "MEM0_DEFAULT_USER_ID": "ian", + "MEM0_DEFAULT_USER_ID": "default-user", "ANTHROPIC_API_KEY": "test-anthropic", "OPENAI_API_KEY": "test-openai", "MEM0_API_KEY": "test-bearer-token", diff --git a/tests/test_auth.py b/tests/test_auth.py index 6b343b8..fd36388 100644 --- a/tests/test_auth.py +++ b/tests/test_auth.py @@ -45,7 +45,7 @@ def _make_jwt(priv_pem, *, aud="mem0-server", iss=ISSUER, exp_delta=3600, **extr now = int(time.time()) claims = { "iss": iss, - "sub": "ian", + "sub": "default-user", "aud": aud, "scope": "read write", "client_id": "claude-web", @@ -89,7 +89,7 @@ async def test_composite_accepts_static_token(): v = CompositeVerifier(static_token="abc") token = await v.verify_token("abc") assert token is not None - assert token.client_id == "ian" + assert token.client_id == "default-user" assert "write" in token.scopes diff --git a/tests/test_config.py b/tests/test_config.py index 3b6384d..aac506c 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -7,7 +7,7 @@ def test_settings_load_from_env(): s = Settings() assert s.qdrant_host == "qdrant.test" - assert s.mem0_default_user_id == "ian" + assert s.mem0_default_user_id == "default-user" assert s.mem0_api_key == "test-bearer-token" diff --git a/tests/test_mcp.py b/tests/test_mcp.py index 811ab67..e8d02af 100644 --- a/tests/test_mcp.py +++ b/tests/test_mcp.py @@ -32,7 +32,7 @@ async def test_add_memory_tool(mcp, mem): mem.add.assert_called_once() args, kwargs = mem.add.call_args assert args[0] == "remember this" - assert kwargs["user_id"] == "ian" + assert kwargs["user_id"] == "default-user" async def test_search_memories_tool(mcp, mem): @@ -41,7 +41,7 @@ async def test_search_memories_tool(mcp, mem): await client.call_tool("search_memories", {"query": "what", "limit": 7}) _, kwargs = mem.search.call_args # Reads are never scoped by agent_id: the store is shared across agents. - assert kwargs["filters"] == {"user_id": "ian"} + assert kwargs["filters"] == {"user_id": "default-user"} assert kwargs["top_k"] == 7 @@ -60,7 +60,7 @@ async def test_list_memories_tool(mcp, mem): async with Client(mcp) as client: await client.call_tool("list_memories", {}) _, kwargs = mem.get_all.call_args - assert kwargs["filters"] == {"user_id": "ian"} + assert kwargs["filters"] == {"user_id": "default-user"} async def test_delete_memory_tool(mcp, mem): diff --git a/tests/test_oauth.py b/tests/test_oauth.py index 7fc680f..10301a2 100644 --- a/tests/test_oauth.py +++ b/tests/test_oauth.py @@ -194,7 +194,7 @@ def test_full_authorize_token_flow(oauth_client): jwks = oauth_client.get("/.well-known/jwks.json").json() pub_pem = _pub_from_jwks(jwks) payload = jwt.decode(access_token, pub_pem, algorithms=["RS256"], audience="mem0-server") - assert payload["sub"] == "ian" + assert payload["sub"] == "default-user" assert payload["scope"] == "read write" diff --git a/tests/test_rest.py b/tests/test_rest.py index 3bdda1f..aa2d4b5 100644 --- a/tests/test_rest.py +++ b/tests/test_rest.py @@ -22,7 +22,7 @@ def test_add_memory(app_instance, mem, auth_header): mem.add.assert_called_once() args, kwargs = mem.add.call_args assert args[0] == "hi" - assert kwargs["user_id"] == "ian" + assert kwargs["user_id"] == "default-user" def test_add_memory_requires_content_or_messages(app_instance, mem, auth_header): @@ -40,7 +40,7 @@ def test_search(app_instance, mem, auth_header): assert resp.status_code == 200 _, kwargs = mem.search.call_args assert kwargs["top_k"] == 5 - assert kwargs["filters"]["user_id"] == "ian" + assert kwargs["filters"]["user_id"] == "default-user" def test_list(app_instance, mem, auth_header): @@ -50,7 +50,7 @@ def test_list(app_instance, mem, auth_header): assert resp.status_code == 200 _, kwargs = mem.get_all.call_args assert kwargs["filters"]["agent_id"] == "n8n" - assert kwargs["filters"]["user_id"] == "ian" + assert kwargs["filters"]["user_id"] == "default-user" assert kwargs["top_k"] == 50 # default list limit must reach mem0 as top_k