feat(adapters): MemPalaceDaemonAdapter — read-side adapter for production palaces

[jphein]

First in a three-PR stack pulling in the adapters and corpus work that's been accumulating on my fork. Stack order:

1. This PR — MemPalaceDaemonAdapter (read-side, hits palace-daemon's HTTP API)
2. feat/familiar-adapter-and-jp-realm-corpus — FamiliarAdapter + jp-realm-v0.1 30-question corpus + first live readings + lessons doc
3. feat/rlm-adapter — RlmAdapter + Qwen-7B / Llama-70B baselines (the data behind #3)

Each PR is built on top of the previous; the diffs against main will collapse to the new commits as earlier PRs merge.

Summary

Adds MemPalaceDaemonAdapter, a read-side adapter that talks to a running palace-daemon HTTP API rather than opening the underlying ChromaDB directly. Useful when:

• The palace is hosted on a remote machine (typical production setup) and the eval framework can't reach the DB files.
• You want the eval to exercise the same access path the orchestrator uses in production (auth, kind filtering, daemon-side query handling).
• You want a /graph fast-path snapshot for Cat 8 without round-tripping every drawer through the MCP layer.

Design + plan docs in the PR

• docs/superpowers/specs/2026-04-25-mempalace-daemon-adapter-design.md
• docs/superpowers/plans/2026-04-25-mempalace-daemon-adapter.md

What's exposed

• query() — happy path against /search, --kind filter, --api-key auth resolution from arg → env → config.
• get_graph_snapshot() — fast-path against /graph, MCP fallback when /graph is unavailable.
• ingest_corpus() — explicit error message rather than silent no-op (palace-daemon is read-side).

Testing

• Mocked HTTP via fake_urlopen_factory fixture (tests/conftest.py, tests/test_mempalace_daemon_adapter.py) — 28 tests covering query happy/error paths, warning contracts, kind filter behavioural invariants, graph snapshot fast-path + MCP fallback, ontology/ingest errors.
• Gated live smoke (tests/test_mempalace_daemon_integration.py) — runs against an actual running daemon when PALACE_DAEMON_URL and PALACE_API_KEY are set; skipped in CI.

Test plan

• python -m pytest tests/test_mempalace_daemon_adapter.py -v — 28 mocked tests pass
• sme-eval retrieve --adapter mempalace-daemon --questions tests/fixtures/tiny_questions.yaml --daemon-url http://localhost:8085 --api-key $KEY — CLI smoke against a running daemon
• PALACE_DAEMON_URL=... PALACE_API_KEY=... python -m pytest tests/test_mempalace_daemon_integration.py — live smoke if you have a daemon to point at

Happy to walk through any specific commit if useful — the 17 commits split into design/plan, scaffold + auth, query, graph snapshot, ontology + CLI wiring, tests/docs/ruff, and one Copilot-review fix commit on top (env-file resilience + tighter live-test gate). The unused-import lint fix from the same Copilot review only applies after the shared _graph_mapping module lands, so it ships with PR #6 instead.

[Copilot]

Pull request overview

Adds a new read-side adapter (MemPalaceDaemonAdapter) so SME can query and snapshot MemPalace via the palace-daemon HTTP API (rather than direct Chroma/SQLite access), with CLI wiring plus mocked and live-gated tests and supporting docs.

Changes:

• Introduce MemPalaceDaemonAdapter with /search querying, /graph fast-path snapshot + MCP fallback, env-file/env-var auth resolution, and explicit read-only ingest behavior.
• Wire the adapter into the CLI (mempalace-daemon name, --api-key, --kind) and update user-facing docs/README.
• Add comprehensive mocked HTTP tests, a gated live-daemon smoke test, and a tiny questions YAML fixture for CLI smoke runs.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
sme/adapters/mempalace_daemon.py	New HTTP adapter implementation (query + graph snapshot)
sme/cli.py	CLI wiring for mempalace-daemon, plus --api-key / --kind plumbing
tests/conftest.py	Adds fake_urlopen_factory for urllib-level HTTP mocking
tests/test_mempalace_daemon_adapter.py	Mocked unit tests covering auth/query/snapshot/MCP fallback/ontology/ingest
tests/test_mempalace_daemon_integration.py	Live-daemon smoke tests gated by env
tests/fixtures/tiny_questions.yaml	Minimal question set for end-to-end CLI smoke
docs/superpowers/specs/2026-04-25-mempalace-daemon-adapter-design.md	Design/spec documentation for the adapter
docs/superpowers/plans/2026-04-25-mempalace-daemon-adapter.md	Implementation plan documentation
README.md	Documents the shipped adapter and usage patterns

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

_parse_env_file calls path.read_text() without handling I/O errors. If the env file exists but is unreadable (permissions, broken symlink, etc.), adapter construction will raise and bypass the intended fallback to process env vars. Catch OSError/UnicodeDecodeError around the read and return {} (optionally logging at debug/warn) so resolution can proceed.

Suggested change

	for line in path.read_text().splitlines():
	try:
	text = path.read_text()
	except (OSError, UnicodeDecodeError) as exc:
	log.warning("Failed to read env file %s: %s", path, exc)
	return out
	for line in text.splitlines():

[Copilot]

get_graph_snapshot() falls back to the MCP path on any /graph error (including AUTH/403, DNS, etc.). In those cases MCP will typically fail the same way and the method ends up returning an empty snapshot, which can mask a real configuration/auth outage as “empty graph”. Consider only falling back on HTTP 404 (endpoint absent) and otherwise surfacing the failure more explicitly (e.g., raise RuntimeError/SystemExit, or at least log at error level and return empty without retrying).

Suggested change

	# _http_get returns QueryResult on error; treat 404 specifically
	if isinstance(body, QueryResult):
	if body.error and body.error.startswith("HTTP 404"):
	log.info(
	"/graph endpoint not present (404); falling back to MCP"
	)
	else:
	log.warning(
	"/graph fetch failed: %s; falling back to MCP",
	body.error,
	)
	return self._snapshot_via_mcp()
	# _http_get returns QueryResult on error; only treat 404 as
	# "endpoint absent" and fall back to MCP in that case.
	if isinstance(body, QueryResult):
	if body.error and body.error.startswith("HTTP 404"):
	log.info(
	"/graph endpoint not present (404); falling back to MCP"
	)
	return self._snapshot_via_mcp()
	error_msg = body.error or "unknown /graph fetch error"
	log.error("/graph fetch failed: %s", error_msg)
	raise RuntimeError(
	f"Failed to fetch graph snapshot from /graph: {error_msg}"
	)

[jphein]

Deferred for now — the empty-on-error shape may be load-bearing for existing users who rely on get_graph_snapshot returning ([], []) on transient daemon issues rather than raising. The behavior change merits reviewer signal first. Happy to ship a follow-up that distinguishes by status code (404 → MCP fallback; 401/403/5xx/network → empty + log.error) once you've signaled direction. Tracking in the commit body of bb50736.

[Copilot]

The retrieve --db help text says it’s only optional when --api-url is supplied, but MemPalaceDaemonAdapter can also resolve api_url/api_key from ~/.config/palace-daemon/env (or process env), so users can omit both --db and --api-url. Update the help text to mention env-file/env-var resolution to avoid implying --api-url is required for mempalace-daemon.

Suggested change

	"--api-url is supplied (ladybugdb adapter in API-only mode, or "
	"the mempalace-daemon adapter which never takes a path).",
	"--api-url is supplied, or when the mempalace-daemon adapter "
	"can resolve API settings from ~/.config/palace-daemon/env or "
	"the process environment.",

[Copilot]

PR description says the live integration suite is gated by MEMPALACE_DAEMON_LIVE=1, but this test file is gated on PALACE_DAEMON_URL being set. Either the PR description or the gate condition/docstring here should be updated so contributors know which env var actually enables the live run.

[M0nkeyFl0wer]

Approved. Merging this clears the way for #6 and #7.

Local verification on pr-5:

• pytest tests/test_mempalace_daemon_adapter.py -v — 28/28 pass
• Full suite — 64 passed, 3 skipped (live-gated)
• ruff check — clean

What I liked:

• Auth resolution chain is the right shape and the tests verify the priority order (kwarg → env_file → process env → ValueError). That's the kind of thing that silently swaps the wrong key in production if the order isn't pinned by tests.
• /graph fast-path with MCP fallback distinguishes 404 (endpoint not present) from other failure modes before falling back, with the right log levels for each. The _project_graph projection mirrors MemPalaceAdapter.get_graph_snapshot structurally and the operational reasoning for skipping drawer-level surface ("impractical at 151K-drawer scale through HTTP") is honest and worth keeping in the docstring.
• ingest_corpus raises NotImplementedError with a helpful pointer to the daemon's /memory endpoint instead of silent no-op. Right call.
• Operational comments earn their place — "180s timeout, list_wings on 151K-drawer palace ~30s" is exactly the kind of comment that saves a future debugger an hour.
• The 3946-line diff is mostly design docs + tests; the adapter itself is 524 LOC, healthy ratio.

No requested changes. The unused-import lint that ships with PR #6 is appropriately deferred.

Once the stack is in I'll wire MemPalaceDaemonAdapter into the check test runner alongside MemPalaceAdapter so the CLI path is symmetric.