Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
282 changes: 282 additions & 0 deletions skill/references/catalyst-query-protocol.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,282 @@
# Catalyst Query Protocol

**Status:** Production-verified (2026-03-01). Full round-trip confirmed end-to-end.

The Catalyst Query Protocol is how Claude Code runs arbitrary Datalog queries against a live Logseq DB graph via MCP. It uses Logseq's own DB as the IPC bus — no external infrastructure required.

---

## Protocol Overview

```
Claude Code (MCP) Catalyst Plugin (JS in Logseq)
───────────────── ──────────────────────────────
1. Create page "Catalyst Query {uuid}"
with Datalog query as first block
──────────► 2. Detect page by title pattern
Execute via datascriptQuery()
Write JSON result as second block
addBlockTag(uuid, "Catalyst Query Done")
3. Poll getPage until blocks >= 2
4. Read blocks[1] as JSON result
5. Retitle page to archive (cleanup)
```

---

## MCP Side — Claude Code Workflow

### Step 1: Create the request page

```python
# No tag needed — plugin detects by title pattern
r, sid = tool("upsertNodes", {
"operations": [
{"operation": "add", "entityType": "page", "id": "qp",
"data": {"title": f"Catalyst Query {query_uuid}"}},
{"operation": "add", "entityType": "block", "id": None,
"data": {"page-id": "qp", "title": datalog_string}}
]
}, sid)
```

**Key:** No tag assignment needed. The plugin detects by title prefix `"Catalyst Query "`. MCP's `upsertNodes` with `tags: [uuid]` does NOT write to `:block/tags` in Datascript — tag-based detection is unreliable for MCP-created pages.

### Step 2: Poll for completion

```python
for i in range(30): # 30s timeout
time.sleep(1)
r, sid = tool("getPage", {"pageName": page_title}, sid)
content = json.loads(r["result"]["content"][0]["text"])
blocks = content.get("blocks", [])
if len(blocks) >= 2:
break # Done — result is in blocks[1]
```

**Signal:** `getPage` returns 2 blocks → plugin has written the result. No tag polling needed (and `getPage` doesn't reliably expose page-level tags anyway).

**Timing:** Typically completes within 1–4 seconds. Plugin scans on `DB.onChanged` plus a 3-second interval fallback.

### Step 3: Read the result

```python
result_raw = blocks[1].get("title", "") or blocks[1].get("content", "")
result = json.loads(result_raw)
# result = {"status": "ok", "results": [[...], ...]}
# result = {"status": "error", "message": "..."}
```

### Step 4: Cleanup

MCP has no delete operation. Retitle to archive:

```python
page_uuid = content["entity"]["uuid"]
tool("upsertNodes", {
"operations": [{"operation": "edit", "entityType": "page",
"id": page_uuid,
"data": {"title": f"_Catalyst/Query Archive/{query_uuid}"}}]
}, sid)
```

Pages can also be left as-is — they accumulate under the `Catalyst Query *` namespace.

---

## Result Schema

```json
{"status": "ok", "results": [["value1"], ["value2"], ...]}
{"status": "error", "message": "Datascript error text"}
```

Results are raw Datascript output — nested arrays. Shape depends on the `:find` clause.

---

## Plugin Side — Implementation Notes

**Detection:** `findPendingQueryPages()` queries all pages with `:block/title` and `:block/uuid`, then filters in TypeScript:
```typescript
.filter(([, title]) =>
title.startsWith('Catalyst Query ') &&
title !== 'Catalyst Query Done' &&
title !== 'Catalyst Query'
)
```

**Trigger:** `DB.onChanged` (fires for some MCP writes) plus `setInterval(3000)` fallback (catches all MCP writes reliably).

**Already-processed guard:** `if (blocks.length >= 2) return` — skips pages that already have a result block.

**Tag tracking:** Plugin still calls `addBlockTag(uuid, "Catalyst Query Done")` on completion. This is visible in the Logseq UI but is NOT the polling signal for Claude Code.

---

## MCP Capabilities Discovered (2026-03-01)

| Tool | Notes |
|------|-------|
| `listPages` | Returns all pages; `expand: true` needed for tag names |
| `getPage(pageName)` | Returns entity + blocks array; does NOT include page-level tags |
| `upsertNodes` | Creates/edits pages and blocks; `tags` field does NOT write `:block/tags` |
| `searchBlocks` | Full-text search; returns raw block data including internal IDs |
| `listTags` | `expand: true` needed for title/uuid; bare call returns nulls |
| `listProperties` | Lists all properties |
| **DELETE** | **Not available.** Pages accumulate unless retitled. |

**Known limitation:** `upsertNodes` with `tags: [uuid]` does not write to `:block/tags` in Datascript — the tag appears in MCP's own representation but not in the Logseq UI or plugin Datalog queries. Use title-based conventions instead of tags for MCP-to-plugin signaling.

---

## MCP Transport

- **URL:** `http://127.0.0.1:12315/mcp`
- **Auth:** `Authorization: Bearer <token>` (token in `~/Library/Application Support/Logseq/configs.edn`)
- **Transport:** Streamable HTTP (MCP 2024-11-05) — requires `Accept: application/json, text/event-stream`
- **Session:** `POST /mcp` with `initialize` → session ID in `mcp-session-id` response header (body is empty). Include `Mcp-Session-Id` in all subsequent requests.
- **Response format:** Tool call responses arrive in the **POST response body** as chunked SSE: `event: message\ndata: {...}\n\n`. The `initialize` response body is empty — session ID in header is sufficient.

**Known session issue (observed 2026-03-07):** New sessions initialized while the Logseq graph is loading may not deliver tool call responses (empty POST body). If tool calls return empty responses, verify the target graph is fully loaded and the Logseq window is active. Sessions established while the graph is open and active work reliably. A Logseq GitHub issue has been filed.

**Python transport note:** `urllib.request` blocks on `resp.read()` for chunked SSE streams. Use raw socket reads with short timeouts instead — read until `b"data: "` appears in the buffer, then parse the line. See the working helper below.

---

## Logseq Legacy HTTP API (mutations from Claude Code)

The MCP protocol is read-oriented. For mutations that MCP can't yet handle (class creation, tag assignment, extends relationships, page deletion), use the **legacy HTTP API** at the same port:

- **URL:** `http://127.0.0.1:12315/api`
- **Auth:** Same Bearer token
- **Format:** `POST {"method": "logseq.Editor.methodName", "args": [...]}` → returns JSON directly (no SSE, no session required)

```python
import urllib.request, json

def logseq_api(method, args, token="s7hhri3r2"):
body = json.dumps({"method": method, "args": args}).encode()
req = urllib.request.Request("http://127.0.0.1:12315/api", data=body,
headers={"Authorization": f"Bearer {token}", "Content-Type": "application/json"})
with urllib.request.urlopen(req, timeout=10) as resp:
return json.loads(resp.read().decode())

# Examples
tag = logseq_api("logseq.Editor.createTag", ["My Class"])
_ = logseq_api("logseq.Editor.addTagExtends", [child_uuid, parent_uuid])
_ = logseq_api("logseq.Editor.addBlockTag", [page_uuid, "My Class"])
_ = logseq_api("logseq.Editor.addTagProperty", [tag_uuid, prop_uuid])
items = logseq_api("logseq.Editor.getTagObjects", ["My Class"])
_ = logseq_api("logseq.Editor.deletePage", ["Page Title"])
```

Available methods mirror the plugin JS API (`logseq.Editor.*`, `logseq.DB.*`). Methods that don't exist return `{"error": "MethodNotExist: ..."}` (HTTP 500). The `logseq.api.*` namespace is **not** exposed — use `logseq.Editor.*` equivalents.

---

## Complete Python Helper

```python
import json, socket, uuid as uuidlib, time

HOST, PORT, TOKEN = "127.0.0.1", 12315, "s7hhri3r2"

def _make_mcp_req(method, path, body_dict, sid=None):
body = json.dumps(body_dict).encode()
h = [f"{method} {path} HTTP/1.1", f"Host: {HOST}:{PORT}",
f"Authorization: Bearer {TOKEN}",
"Content-Type: application/json",
"Accept: application/json, text/event-stream",
f"Content-Length: {len(body)}", "Connection: keep-alive"]
if sid: h.append(f"Mcp-Session-Id: {sid}")
return ("\r\n".join(h) + "\r\n\r\n").encode() + body

def _recv_data(s, timeout=15):
s.settimeout(0.3)
raw = b""
deadline = time.time() + timeout
while time.time() < deadline:
try:
c = s.recv(8192)
if not c: break
raw += c
if b"data: " in raw: break
except socket.timeout: pass
return raw

def _parse_chunked_sse(resp_bytes):
body = resp_bytes[resp_bytes.index(b"\r\n\r\n")+4:] if b"\r\n\r\n" in resp_bytes else resp_bytes
decoded = b""
buf = body
while buf:
if b"\r\n" not in buf: break
se = buf.index(b"\r\n")
try: csz = int(buf[:se].decode("ascii","replace").strip(), 16)
except: break
if csz == 0: break
ds = se + 2
if len(buf) < ds + csz: decoded += buf[ds:]; break
decoded += buf[ds:ds+csz]
buf = buf[ds+csz:]
if buf.startswith(b"\r\n"): buf = buf[2:]
for line in decoded.decode("utf-8","replace").split("\n"):
if line.startswith("data: "):
return json.loads(line[6:])
return None

def mcp_init():
"""Initialize MCP session. Returns session ID."""
s = socket.socket()
s.connect((HOST, PORT))
s.sendall(_make_mcp_req("POST", "/mcp", {"jsonrpc":"2.0","id":1,"method":"initialize",
"params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"cc","version":"1.0"}}}))
buf = b""
deadline = time.time() + 5
while time.time() < deadline:
try:
s.settimeout(0.3); c = s.recv(4096); buf += c if c else b""
if b"\r\n\r\n" in buf: break
except socket.timeout: break
s.close()
for line in buf.decode("utf-8","replace").split("\r\n"):
if line.lower().startswith("mcp-session-id:"):
return line.split(":",1)[1].strip()

def mcp_tool(sid, name, args, req_id=2, timeout=15):
"""Call an MCP tool and return the parsed JSON-RPC result."""
s = socket.socket()
s.connect((HOST, PORT))
s.sendall(_make_mcp_req("POST", "/mcp", {"jsonrpc":"2.0","id":req_id,"method":"tools/call",
"params":{"name":name,"arguments":args}}, sid=sid))
resp = _recv_data(s, timeout)
s.close()
return _parse_chunked_sse(resp)

def run_datalog(sid, datalog: str, timeout: int = 30) -> dict:
"""Run a Datalog query via the Catalyst Query Protocol. Returns result dict."""
query_uuid = str(uuidlib.uuid4())
page_title = f"Catalyst Query {query_uuid}"

mcp_tool(sid, "upsertNodes", {"operations": [
{"operation":"add","entityType":"page","id":"qp","data":{"title": page_title}},
{"operation":"add","entityType":"block","id":None,"data":{"page-id":"qp","title": datalog}}
]}, req_id=2)

for i in range(timeout):
time.sleep(1)
r = mcp_tool(sid, "getPage", {"pageName": page_title}, req_id=100+i)
if r is None: continue
content_text = r.get("result",{}).get("content",[{}])[0].get("text","{}")
content = json.loads(content_text)
blocks = content.get("blocks", [])
if len(blocks) >= 2:
result_raw = blocks[1].get("title","") or blocks[1].get("content","")
page_uuid = content.get("entity",{}).get("uuid","")
if page_uuid:
mcp_tool(sid, "upsertNodes", {"operations": [{"operation":"edit","entityType":"page",
"id": page_uuid, "data":{"title": f"_Catalyst/Query Archive/{query_uuid}"}}]}, req_id=200)
return json.loads(result_raw)

raise TimeoutError(f"Query not completed in {timeout}s: {datalog}")
```
42 changes: 40 additions & 2 deletions skill/references/core-apis.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,13 +45,35 @@ const zotItems = await logseq.Editor.getTagObjects('zot')

```typescript
// Add property to tag schema (using parent frame API)
// IMPORTANT: Both arguments require BlockIdentity (UUID string), NOT display names.
// Confirmed in production: manifest-install.ts uses info.logseqUuid for the tag
// and a DataScript-resolved UUID for the property.
const parentLogseq = (window as any).parent?.logseq
await parentLogseq.api.add_tag_property(tagUuid, 'propertyName')
await parentLogseq.api.add_tag_property(tagUuid, propertyUuid)

// Remove property from tag schema
await parentLogseq.api.remove_tag_property(tagUuid, 'propertyName')
await parentLogseq.api.remove_tag_property(tagUuid, propertyUuid)
```

**Resolving the property entity UUID** — `upsertProperty` does not return a UUID.
Query DataScript after the upsert to resolve the property entity's UUID:

```typescript
const results = await logseq.DB.datascriptQuery(`
[:find ?uuid
:where
[?e :block/title "propertyName"]
[?e :block/tags ?t]
[?t :db/ident :logseq.class/Property]
[?e :block/uuid ?uuid]]
`) as [string][] | null
const propertyUuid = results?.[0]?.[0]
```

**Resolving the tag entity UUID** — use the UUID returned by `createTag` or the `uuid` field
from `getTag`. Do not pass a display name string. Confirmed: `info.logseqUuid` (from `createTag`
return value or `existing.uuid` from `getTag`) is the correct form.

## Page & Block Creation

### createPage
Expand Down Expand Up @@ -172,6 +194,20 @@ await logseq.Editor.addTagExtends(childTagId, parentTagId)
await logseq.Editor.removeTagExtends(childTagId, parentTagId)
```

**Multiple inheritance is supported.** `:logseq.property.class/extends` is a cardinality-many attribute — calling `addTagExtends` multiple times with different parents stores all parents as separate datoms. Confirmed 2026-03-07.

```typescript
// Single parent
await logseq.Editor.addTagExtends(gammaTag.uuid, alphaTag.uuid)

// Add a second parent — both are stored, neither overwrites
await logseq.Editor.addTagExtends(gammaTag.uuid, betaTag.uuid)

// Datalog query now returns TWO rows for Test Gamma:
// [?e :logseq.property.class/extends 226] ← Alpha
// [?e :logseq.property.class/extends 227] ← Beta
```

**Example - Creating tag hierarchy**:

```typescript
Expand All @@ -190,6 +226,8 @@ await logseq.Editor.addTagExtends(feedbackTag.id, taskTag.id)
// See queries-and-database.md for how to query tag hierarchies
```

**`getTagObjects` is hierarchy-aware** — it traverses `extends` and returns instances of subclasses. Calling `getTagObjects('Task')` returns items directly tagged `Task` AND items tagged with any class that extends `Task`. The standard `or-join` Datalog pattern handles multi-parent classes without modification (each parent chain resolves independently).

**Querying Tag Hierarchies**:

See [queries-and-database.md](./queries-and-database.md#tag-inheritance-with-or-join) for how to find items tagged with parent OR child tags using `or-join`.
Expand Down
25 changes: 24 additions & 1 deletion skill/references/pitfalls-and-solutions.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,30 @@ If you have existing code using the wrong names, do a find-and-replace:
- Find: `logseq.Editor.removeTag(`
- Replace: `logseq.Editor.removeBlockTag(`

## Pitfall 9: or Clause Variable Mismatch
## Pitfall 9: createPage Does Not Apply Class Tags

**Problem**: Creating a page with `tags: ["MyClass"]` in `createPage()` does not tag the page with the class. The page is only tagged with "Page".

**Example**:
```typescript
// ❌ This does NOT tag the page with "Task"
const page = await logseq.Editor.createPage('My Item', { tags: ['Task'] })
// page will have tag "Page" only — not "Task"
```

**Solution**: Use `addBlockTag()` after page creation:

```typescript
// ✅ CORRECT
const page = await logseq.Editor.createPage('My Item', {})
await logseq.Editor.addBlockTag(page.uuid, 'Task')
```

**Confirmed** in test-load, 2026-03-07.

---

## Pitfall 10: or Clause Variable Mismatch

**Problem**: Query fails with error: "All clauses in 'or' must use same set of free vars"

Expand Down