Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions docs/ideas.md
Original file line number Diff line number Diff line change
Expand Up @@ -430,6 +430,44 @@ shape future adapter work and category implementations:
and `baselines/jp_realm_v0_1_*.json` made it obvious where to
add `jp_realm_v0_2`, `family_palace_v0_1`, etc. Don't refactor.

## RlmAdapter — research scaffold (2026-04-26)

`sme/adapters/rlm_adapter.py` ships an adapter that treats RLM
(jphein/rlm fork of alexzhang13/rlm) as the read-side orchestrator:
the LLM itself decides when to call `mempalace_search`, with what
queries, and how to compose results. familiar's deterministic
retrieve→rerank→decay→compress pipeline becomes the *baseline* this
adapter is benchmarked against.

**Design:** RLM gets `mempalace_search` registered as a
`custom_tools` callable. The adapter wraps that callable to capture
every search result into a per-query buffer. After `rlm.completion()`
returns, the buffer's contents become `context_string` (in tool-call
order) and `retrieved_entities` (one Entity per drawer). The Cat 1 /
retrieve substring scorer measures whether `expected_sources` ended
up in `context_string` — same contract as every other adapter.

**Test coverage** (`tests/test_rlm_adapter.py`, 5 tests): tool-call
aggregation, capture-buffer reset across queries, error-dict
graceful handling on palace network failure, empty graph snapshot
(Cat 8 N/A for RLM), ingest_corpus skipped.

**To benchmark live:**
```bash
PORTKEY_API_KEY=... PALACE_DAEMON_URL=http://disks.jphe.in:8085 \
PALACE_API_KEY=... \
venv/bin/sme-eval retrieve --adapter rlm \
--questions sme/corpora/jp_realm_v0_1/questions.yaml \
--json baselines/jp_realm_v0_1_rlm_$(date +%Y%m%d).json
```

A live benchmark will reveal whether RLM-orchestration recovers
recall on questions familiar misses, plateaus at the same number,
or regresses — the answer determines whether v0.4 should
productionize RLM into familiar's chat path. Without that data,
the design spec's "RLM and familiar are complementary" hypothesis
stays a hypothesis.

## What's next

### Categories that aren't implemented yet
Expand Down
9 changes: 8 additions & 1 deletion docs/sme_spec_v8.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,9 +176,16 @@ class SMEAdapter(ABC):
Returns: {'type': 'declared'|'readme'|'inferred',
'schema': [...], 'documentation': str}"""
return {'type': 'inferred', 'schema': [], 'documentation': ''}

def get_harness_manifest(self) -> list[HarnessDescriptor]:
"""Return invocation surfaces this memory system exposes.
Used by Category 9 (Harness Integration). Adapters that don't
expose a harness surface (pure library APIs) return [] — Cat 9b
reports empty_manifest=True rather than crashing."""
return []
```

Three required methods. That's the minimum viable adapter. `get_flat_retrieval` and `get_ontology_source` have defaults — SME fills in its own flat baseline and infers ontology from the graph if the adapter doesn't provide them.
Three required methods. That's the minimum viable adapter. `get_flat_retrieval`, `get_ontology_source`, and `get_harness_manifest` have defaults — SME fills in its own flat baseline, infers ontology from the graph, and treats an empty harness manifest as "Cat 9 not applicable" when the adapter doesn't provide them.

### Default Adapters

Expand Down
11 changes: 11 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,17 @@ ladybugdb = [
neo4j = [
"neo4j>=5.0",
]
# RlmAdapter — Recursive Language Models. Pinned to jphein/rlm (fork
# of alexzhang13/rlm) because the baselines/jp_realm_v0_1_rlm_*.json
# artifacts were produced against this fork; reproducing them on a
# fresh clone needs the same code path, even when the package surface
# is currently identical to upstream. Distribution name is `rlms`;
# importable module is `rlm`. Requires Python >=3.11 (one minor
# higher than SME's >=3.10 floor), so this extra is opt-in and not
# part of `all`. Tests are guarded with `pytest.importorskip`.
rlm = [
"rlms @ git+https://github.com/jphein/rlm.git",
]
all = [
"sme-eval[topology,viz,ladybugdb,neo4j]",
]
Expand Down
4 changes: 2 additions & 2 deletions sme/adapters/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
never touches a database directly — it talks to this thin interface.

Three required methods: ingest_corpus, query, get_graph_snapshot.
Two optional: get_flat_retrieval, get_ontology_source.
Three optional: get_flat_retrieval, get_ontology_source, get_harness_manifest.
"""

from __future__ import annotations
Expand Down Expand Up @@ -116,7 +116,7 @@ class QueryResult:
class SMEAdapter(ABC):
"""Implement this for your database/memory system.

Three required methods. Two optional.
Three required methods. Three optional.
"""

# --- Required ------------------------------------------------------
Expand Down
Loading
Loading