Release v5.11.0 — security hardening, coverage closure, CI matrix on macOS#8
Merged
Conversation
…(v5.12 Phase A foundation) tool/prompt/resource registrations are now collected as typed thunks and replayed onto a server via registerCapabilities(s) — enabling a fresh McpServer per HTTP session (handlers reference shared module-global state, so no per-session state needed). stdio replays onto the singleton before connect; behavior-identical. Verified: tsc clean, full suite 1118/1118, live stdio smoke lists 51 tools incl gnosys_add.
…(v5.12 Phase A + C) New mcpHttp.ts: Node http server hosting StreamableHTTPServerTransport with stateful per-session McpServers (built via registerCapabilities), /health probe, and a bearer-token auth gate (Phase C). serve --transport http|--host|--port|--token; main() branches on GNOSYS_TRANSPORT; default binds 127.0.0.1 (use a tailnet addr to share). stdio stays the zero-config default. 8 tests (incl auth + concurrent sessions); full suite 1126/1126; live 2-client smoke verified. Phase C (auth + binding) delivered alongside A. Roots-notification auto-discovery is stdio-only for now; HTTP clients pass projectRoot per call (the gnosys pattern).
Adapt the existing Dockerfile/compose to run 'serve --transport http' on :7777: GNOSYS_HOME=/data on a host-local named volume (never SMB), non-root user, /health HEALTHCHECK, EXPOSE 7777, GNOSYS_SERVE_TOKEN for bearer auth. Add docs/network-mcp.md (Mac launchd vs Docker, client config, security, backup). NOTE: image not build-tested locally — Docker daemon was down; compose config validated, directives verified.
…ocal one (v5.12 Phase E) centralize.ts uses SQLite's online backup API to write a consistent gnosys.db into a target dir (handles WAL, safe while in use). 'gnosys centralize --to <dir> [--force]' for seeding a Docker volume / new host. 4 tests.
…5.12 Phase B) mcpClientConfig.ts writes the URL-based MCP entry (with optional bearer header) into Cursor (.cursor/mcp.json) or Claude Desktop config, merging with existing servers. 'gnosys connect --url <url> [--token] [--ide] [--dir] [--print]'. Additive — does not touch the existing local-stdio setup flow. 5 tests.
Route the error paths of 10 central-DB-reading tools (gnosys_reinforce,
gnosys_stale, gnosys_lens, gnosys_timeline, gnosys_stats, gnosys_graph,
gnosys_dream, gnosys_export, gnosys_stores, gnosys_recall) plus the
gnosys_read legacy file-read path through the existing formatMcpError
helper. Previously these tools relied on the raw SDK error wrapper, so a
corrupted central DB surfaced "database disk image is malformed" instead
of the actionable corruptionRecoveryInstructions() — and gnosys_read
could leak an absolute filesystem path on read failure.
Error envelope shape is unchanged ({ content: [{type:"text"}], isError:
true }); only message normalization is added. No tool names, schemas, or
success-path output changed.
Review task 1.1 (review_passed). tsc clean; error tests green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The shipped README previously documented zero tools by name (only a generic "50+ memory tools" claim), leaving 19 of 51 registered tools undiscoverable from the npm page. Adds a "## MCP Tool Reference" table covering all 51 tools, each with the first sentence of its registration description. Registered<->documented diff is now clean in both directions (0 missing, 0 phantom). Review task 1.3 (review_passed). Parity loop emits no MISSING lines. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
main() was invoked unguarded at module top level, so importing the entry
module booted the whole server — the reason the 51-tool surface had zero
schema tests. Guard the call with an ESM script check
(fileURLToPath(import.meta.url) === path.resolve(process.argv[1])) so the
module is importable without side effects; binary behavior is unchanged.
Add src/test/mcp-fuzz.test.ts: connects an in-memory MCP client to a
server built from registerCapabilities and asserts every tool with
required fields rejects {} (missing required) and wrong-typed input.
Runtime rejection itself is SDK-guaranteed (safeParseAsync ->
McpError InvalidParams); this locks it under test.
Oversize-string cases intentionally omitted: content fields
(gnosys_add, ingest, bootstrap) use unbounded z.string() by design.
Review task 1.4 (review_passed). tsc + build clean; npm test green
(1136 tests); importing dist/index.js no longer boots the server.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add src/test/mcp-http-replay.test.ts: starts the HTTP transport with the real registerCapabilities registry, opens two concurrent client sessions, and asserts both list the identical full tool surface (>=51 tools, including gnosys_discover/recall/add/ingest_file). The existing v512-mcpHttp.test.ts only exercised a one-tool stub server and a session count, so the replayable-registration invariant (_registrations -> registerCapabilities per session) was untested. Review task 1.6 (review_passed). tsc + build clean; test green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
history, semantic-search, lens, timeline, links, graph, and ask lacked
a --json mode, so they could not be consumed programmatically. Each now
declares --json and emits structured JSON to stdout via the existing
outputResult() helper; human output stays the default and the upgrade
notice stays on stderr (no banner contamination).
JSON shapes: history {memoryPath,entries|diff}, lens {count,items},
timeline {period,count,entries}, links {memoryPath,outgoing,backlinks},
graph {totalLinks,orphanedLinks,nodes}, semantic-search {query,count,
results}, ask {question,answer,sources,deepQueryUsed} (streaming off).
Review task 2.4 (review_passed). tsc + build clean; cli-json and
cli-parity tests green; npm test 1137 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
search.db (persistent), the centralize backup source, and the two readonly opens (legacy-store check in cli.ts, embeddings copy in migrate.ts) opened without a busy_timeout, so a concurrent writer would raise SQLITE_BUSY immediately instead of waiting. Add busy_timeout = 5000 to each. Journal mode is left unchanged — search.db intentionally avoids WAL for sandbox/network-FS portability — and the :memory: opens are untouched. The central gnosys.db already sets WAL + busy_timeout=10000. Review task 3.4 (review_passed). tsc clean; search + db-recovery green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
getIdsModifiedSince() runs SELECT ... WHERE modified > ? OR created > ? for multi-machine sync deltas, which was a full table scan — memories is the one growing table whose WHERE columns weren't fully indexed. Add idx_memories_modified and idx_memories_created to SCHEMA_SQL (run on every open via IF NOT EXISTS, so existing DBs pick them up with no migration bump). EXPLAIN QUERY PLAN now shows MULTI-INDEX OR using both indexes instead of SCAN. Review task 3.5 (review_passed). tsc clean; central-db + db-recovery green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New src/test/db-recovery-extended.test.ts covers four realistic failure modes the existing db-recovery.test.ts did not: - SIGKILL mid-transaction: forked child holds an open WAL transaction, parent SIGKILLs; reopen asserts integrity_check=ok and the uncommitted rows are rolled back. - Full disk: injects SQLITE_FULL and asserts a clear error that isCorruptionError() correctly classifies as non-corruption. - Corrupted FTS index: drops memories_fts; searchFts falls back to LIKE (db.ts FTS catch path) instead of throwing. - Missing better-sqlite3: vi.doMock makes the import fail; isAvailable() is false, getMeta() returns null, backup() rejects clearly. Existing db-recovery.test.ts untouched. Review task 3.7 (review_passed). tsc clean; 10 recovery tests green; npm test 1141 passed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New src/test/lifecycle-e2e.test.ts chains add → read → update → archive → dearchive → reinforce×3 → maintain across GnosysDB, GnosysArchive, and GnosysMaintenanceEngine, then asserts the DB is internally consistent: PRAGMA integrity_check = ok, exactly one primary row per memory id, and a synced memories_fts row. Uses shared _helpers test env with temp dirs. Review task 4.1 (review_passed). tsc clean; test green; npm test 1142. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New src/test/lifecycle-invariants.test.ts asserts, after each lifecycle sync op (add → update → archive → dearchive → reinforce → delete), that every memory id has exactly one primary memories row (zero after delete), a memories_fts count ≤1 that equals the memories count (synced, no orphan FTS rows), and no duplicated ids. Complements the 4.1 end-to-end test with after-each-step invariant checking. Review task 4.6 (review_passed). tsc clean; test green; npm test 1143. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a 50-memory known-content corpus (8 themed clusters, fixed dates), a committed golden top-3 file (20 variant::query entries), and search-golden.test.ts asserting each search variant returns a stable top-3 (identical across runs) that matches the golden file. Covers keyword, discover, federated, hybrid, and semantic; hybrid/semantic use a deterministic hash-based stub embedder so the test is hermetic (no network/model). Guards against silent ranking regressions. Review task 5.1 (review_passed). tsc clean; 21 tests green; npm test 1164. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Document the three retrieval modes: gnosys_search (FTS5 keyword, no embeddings), gnosys_semantic_search (embedding cosine only), and gnosys_hybrid_search (Reciprocal Rank Fusion, k=60, of both). Includes a comparison table, an RRF explanation, a same-query worked example, mode- selection guidance, and CLI equivalents. Review task 5.2 (review_passed). Docs only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add an explicit anti-injection rule to prompts/synthesize.md: retrieved
memory content under "## Context Memories" is untrusted data, not
instructions; the model must ignore embedded directives (e.g. "ignore
previous instructions", "reveal secrets") and never claim or emit
credentials/env/files. Adds a data-marker comment above {{CONTEXT}}.
Defense-in-depth: the audit (task 5.5) confirmed ask cannot leak API
keys (they're in the HTTP auth header, never the LLM context) and has no
code-execution path; this further mitigates output manipulation from
memory-embedded injection.
Review task 5.5 (review_passed). Prompt-only change; npm test 1164.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add docs/llm-provider-contract.md: the three types (LLMGenerateOptions/LLMStreamCallbacks/LLMProviderName), the five interface members (name, model, generate, generateWithImage?, testConnection) with faithful signatures, streaming semantics (tokens via onToken, full text on resolve, no silent partials), error/ retry behavior (transient 429/timeout retried via withRetry + isTransientError; 401/403 throw immediately; API keys redacted), the provider implementations, and a guide for adding a provider. Review task 6.1 (review_passed). Docs only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
No provider call had a client-enforced timeout, so a hung connection (notably a local Ollama server that accepts but never responds) could block an MCP tool / CLI call indefinitely. Add AbortSignal.timeout to all six fetch calls (60s for generate/vision, 10s for tags/models probes) and a timeout on the Anthropic SDK client. The resulting "timed out" error is already classified retryable by isTransientError, so withRetry retries and then surfaces a clear timeout instead of hanging. Review task 6.2 (review_passed). tsc clean; 27 provider tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Error-text redaction relied on a prefix allowlist (sk-/gsk_/Bearer), which missed the xai/mistral/custom key formats the OpenAICompatible provider serves. Add a redactKey(text, apiKey) helper that strips the literal key value (length >= 8, format-agnostic) and applies an extended prefix regex (adds sk-ant-, xai-) as a secondary net. Route all three provider error paths (Anthropic, OpenAICompatible request + vision) through it. New llm-redact.test.ts covers the literal-key, prefix, and short-string-guard cases. Review task 6.3 (review_passed). tsc clean; 30 provider+redact tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The local embeddings path imported @huggingface/transformers without a guard, so a missing optional dep threw a raw ERR_MODULE_NOT_FOUND stack. Wrap the dynamic import and rethrow a clear one-liner: "Local embeddings require @huggingface/transformers. Install it with: npm install @huggingface/transformers" (matching the Whisper hint in audioExtract). New embeddings-optional-dep.test.ts covers missing (asserts the hint, not the raw error) and installed (mocked pipeline → 384-dim Float32Array). Review task 6.5 (review_passed). tsc clean; 2 tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There is a per-call output cap (maxTokens, default 4096) but no per-call input cap and no daily/cumulative cost cap or spend tracking. Per task 6.6's "document why there isn't" branch, add docs/cost-and-limits.md covering: caps that exist, caps that don't (by design), that spend bills to the user's own provider account (set limits in the provider billing console), and how to bound cost — local providers ($0), Dream Mode defaulting to local Ollama, budget-tier models, and lower maxTokens. Review task 6.6 (review_passed). Docs only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New ingest-fixtures.test.ts drives 8 hostile/malformed inputs through ingestFile and asserts each is handled gracefully (success or a clear Error — never a crash, OOM, or hang): normal PDF, 0-byte, UTF-8 BOM, 100MB-over-cap text (generated at runtime, hits the maxFileSizeMb cap), corrupt DOCX, non-existent path, and a PDF with embedded JS (handled without executing JS). Encrypted PDF is skipped with a TODO. Only two sub-1KB PDFs are committed; large fixtures are generated and cleaned up. Review task 7.1 (review_passed). tsc clean; 7 passed + 1 skipped. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
extractDocxText handed the buffer straight to mammoth, which decompresses all zip entries into memory — so a <=100MB DOCX whose word/document.xml is highly compressible could expand to tens of GB and OOM the process (the 100MB input cap bounds the file, not the decompressed size). Add a central-directory size check: JSZip.loadAsync parses entry metadata (without decompressing payloads), sum uncompressedSize, and reject totals over 200MB with a clear "possible zip bomb" error before mammoth runs. New docx-bomb.test.ts: a 210MB-decompressed bomb is rejected without OOM, and billion-laughs entities don't expand (xmldom is non-validating). Review task 7.2 (review_passed). tsc clean; 2 tests green (~3s). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
chunkSplitter.ts is a pure function (no clock/random) but had no test coverage. Add chunk-splitter.test.ts asserting splitIntoChunks returns deeply-equal output across repeated calls for varied inputs (empty, short, many-paragraph, oversized), stability across repetitions, and a stable fnv1a content hash. Guards identical-input -> identical-chunks against future regressions. No production change. Review task 7.3 (review_passed). tsc clean; 6 tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New ingest-special-paths.test.ts ingests temp files whose names contain spaces, unicode (café), emoji (🎉), and trailing whitespace, asserting each ingests cleanly (>=1 memory). Node fs/path handle UTF-8 natively and the ffmpeg path uses execFileSync argv (7.5), so these already worked; this locks it against regressions. No production change. Review task 7.6 (review_passed). tsc clean; 4 tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ings) webIngest's URL guard only validated the initial URL and explicitly allowed loopback, so SSRF was possible via (a) a public URL that 302-redirects to 127.0.0.1 / 169.254.169.254 (fetch followed redirects unchecked) and (b) direct loopback/encoded-IP hosts. - isSafeUrl: block loopback by default (opt-in allowLoopback), 0.0.0.0, hex hosts (0x7f000001 / 0x7f.0.0.1), all-numeric decimal IPs (2130706433), IPv6 ULA fc00::/7 + link-local fe80::/10; keep metadata + private IPv4 + non-http(s) scheme blocks. Uses node:net isIP. - safeFetch: redirect:"manual" with per-hop re-validation (max 5), replacing all raw fetch() calls in sitemap + page fetching. - New webingest-ssrf.test.ts: 14 blocked vectors, public URL allowed, loopback opt-in, and a mocked redirect-to-metadata that is rejected. Review task 7.7 (review_passed). tsc clean; 17 SSRF tests green; npm test 1205. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New remote-two-machine.test.ts runs the full cross-machine sequence against two GnosysDB instances sharing one temp NAS dir: A push → B pull → B edit/push → A pull → both edit offline → B push → A sync(skip-and-flag). Asserts correct propagation at each step, exactly one flagged unresolved conflict, and no data loss (A retains its local v3-from-A while the conflict is recorded, not silently overwritten). Review task 9.1 (review_passed). tsc clean; test green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…otent) New remote-resume.test.ts simulates a mid-push kill (12 memories seeded, 5 copied to the NAS DB, lastSync unchanged), then re-runs push() and asserts: remote integrity_check ok, all 12 present exactly once (no duplicates from re-pushing the 5), per-memory content matches, and a second push is idempotent (pushed=0). Locks the crash-safe, idempotent resume behavior (atomic INSERT OR REPLACE per memory + lastSync gating). Review task 9.4 (review_passed). tsc clean; test green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dream.ts was in the coverage exclude list despite having 5 test files, hiding a C.1 target. Remove it from exclude (maintenance/recall/llm excludes kept) so its real coverage (28.5%) is measured. Thresholds still pass (overall 57.7% stmts). Baseline vs 80% goal: mcpHttp 89% (pass); db 77%, remote 74%, dream 29%, ingest 17% (tracked debt — dedicated coverage effort recommended). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cover four pure-logic lib modules that had no tests: retry (96%), heartbeat (82%), progress (73%), modelValidation buildRequest (50%; I/O paths justified). Un-exclude retry.ts from coverage (pure logic, was wrongly bundled with llm). Typed fetch mocks keep tsc --noEmit clean. 10 external/IO/template/type-only files justified (no unit test). Full suite: 1275 passed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ys (C.4) resolver-routing.test.ts wrote to the developer's real ~/.config/gnosys/projects.json (backup/restore was crash-fragile). Isolate it via a per-test GNOSYS_CONFIG_DIR tmpdir; drop the real-registry mutation. Verified: the real projects.json is byte-identical before/after the test. - resolver.ts: getRegistryPath() delegates to getProjectRegistryPath() so the GNOSYS_CONFIG_DIR hook applies (removes duplicated home-path logic) - setup-ui-screen10.test.ts: fake home /home/gnosys-test (was /Users/edward) Supervised edits to 2 existing test files + 1 production change (resolver). tsc clean; full suite 1275 passed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add os: [ubuntu-latest, macos-latest] to the build-and-test matrix (runs-on: matrix.os) so Node 18/20/22/24 are exercised on both Linux and macOS per C.7. Re-gate the 4 coverage steps to ubuntu-latest + Node 24 only, so coverage runs once and the coverage-report artifact isn't uploaded by two jobs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
acceptance.test.ts covered Central brain, Federated search, Dream Mode, and Obsidian export, but had no happy-path smoke for MCP server, Web KB, or multi-machine sync (each only depth-tested in dedicated suites). Add acceptance-features.test.ts with three subprocess/API-level smokes: - MCP server: spawn real dist/index.js (stdio, isolated HOME) and round-trip listTools + gnosys_init/gnosys_add_structured/gnosys_search - Web KB: buildIndexSync + search returns hits - Multi-machine sync: RemoteSync push A -> pull B propagates a memory Every documented "What you get" feature now has an acceptance happy path. tsc clean; full suite 1278 passed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add src/lib/log.ts with logError/Warn/Info/Debug; env-driven sinks:
- default: plain text to stderr (UX unchanged)
- GNOSYS_LOG_FORMAT=json -> JSON lines to stderr
- GNOSYS_LOG_FILE=<path> -> append JSON lines to file
- GNOSYS_LOG_LEVEL gates emission
JSON records carry timestamp/level/message/error.{name,message,stack}
plus any context. Logger is best-effort (never throws on file errors).
Migrate 2 representative error sites (db open fallback; dream scheduler);
remaining stderr.write/console.error sites adopt gradually. NEW log.test.ts.
tsc clean; full suite 1283 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
51 published 5.x versions vs 37 CHANGELOG entries: - Backfill 5.4.1 (remote-first, ULID, 10-bug sweep) and 5.4.3 (postinstall visibility, upgrade nudge, CODE_OF_CONDUCT) — both real npm releases with no prior entry. - Remove duplicate 5.4.3 bullets misplaced under 5.5.0. - Add a Historical versions note disclosing the 15 pre-5.2.16 + 5.2.x patch gaps (5.0.0-5.2.15 plus 5.2.17/18/21) as tracked via git tags; note 5.2.13-15 were CHANGELOG-only and never published to npm. Every 5.x published version is now entry'd or honestly disclosed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add scripts/gen-mcp-tools.mjs (scans regTool(...) registrations in src/index.ts, sorts alphabetically, escapes pipes) and the generated docs/mcp-tools.md (50 tools). npm run docs:mcp-tools regenerates; regenerating produces a byte-identical file (no drift today). README's curated tool table untouched (still useful on the npm landing page); the generated doc is the in-repo source of truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirror of E.4 for CLI commands. Add scripts/gen-cli-docs.mjs (scans Commander .command()/.description() pairs in src/cli.ts, handles multiline descriptions, preserves registration order) and committed docs/cli.md (103 sections). npm run docs:cli regenerates; regenerating produces a byte-identical file (no drift today). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add docs/adr/ with README index and 8 short (150-300 words each) architectural-decision records sourced from the corresponding Gnosys memory entries: 0001 MCP-First Architecture (dec-009) 0002 Layered Multi-Store Architecture (deci-030) 0003 Why Not RAG (dec-001) 0004 TypeScript Implementation (dec-010) 0005 DB-only Architecture (deci-032) 0006 Built-in Server + Obsidian (dec-011) 0007 Open Source from Day One (dec-005) 0008 Automated npm Publish (OIDC) (deci-033) Each ADR: Status/Date/Memory header + Context/Decision/Consequences sections. Gnosys memory remains the rolling source-of-truth; these are stable snapshots for new contributors. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add docs/source-of-truth.md — single page naming the canonical home for each kind of content (quickstart→README, full guide→gnosys.ai, CLI/MCP refs→generated docs, decisions→Gnosys memory + docs/adr/, security→SECURITY.md + docs/threat-model.md, …). Includes rules of thumb so contributors never duplicate-maintain the same info in two places. Final task of the 222-task review. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NEW src/test/ingest-structured.test.ts (21 tests) covering the GnosysIngestion.ingest() LLM-structuring path with a stubbed provider: - provider-missing error paths for every envVarMap arm (anthropic / openai / groq / xai / mistral / custom / ollama / lmstudio / unknown) - JSON parsing variants: bare, json-fenced, plain-fenced, prose + fenced - prototype-pollution sanitization (__proto__, constructor, prototype keys stripped) - tag-registry validation + proposedNewTags surfacing - field defaults when LLM returns minimal JSON - configOverride: fresh provider resolution + missing-provider fallback - isLLMAvailable / providerName getters Coverage: ingest.ts now 100% statements / 91.93% branches / 100% functions / 100% lines (gate: ≥80%). No source changes; no existing test files modified. Full suite 1304 passed, 1 skipped; tsc --noEmit clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NEW src/test/dream-coverage.test.ts (29 tests) covering the four uncovered regions of dream.ts with mocked LLM provider and mocked desktopNotify: Orchestrator (GnosysDreamEngine.dream): - DB-unavailable early exit - Too-few-memories early exit - Provider-init error → dream_provider_unreachable audit + incrementDreamConsecutiveFailures - Layer-4 desktop notify at consecutive-failure threshold - All-phases happy path with stubbed LLM - Abort at shouldStop checkpoint - Max runtime exceeded path - finalize resets consecutive-failures when LLM work succeeded Phase implementations: - decaySweep: skip recent / skip tiny delta / update stale - critiquMemory rule arms (low conf, never-reinforced+old, short content, no tags, no relevance, invalid tags) - llmCritique branches (ok / review / needs-update / malformed) - generateSummaries: create / skip-unchanged / update - summarizeCategory provider-error swallow - discoverRelationships: self-ref filter, low-confidence filter, dedup of existing pairs - findRelationships malformed-JSON branch formatDreamReport: happy / aborted / empty paths. DreamScheduler: prototype-pollution allowlist, disabled / not- designated no-ops, designated idle trigger (fake timers), recordActivity abort, stop+abort, isDesignatedMachine exception swallow, getLocalMachineId hostname fallback + meta cache, isDreaming, checkIdle rejection swallow. Coverage: dream.ts now 91.68% statements / 78.04% branches / 95% functions / 95.42% lines (gate: ≥80% lines). No source changes; existing dream test files (dream-resume, phase7d.dream, phase9b.dream-prefs-sync, v594-dream-provider-inheritance, v594-dream-state) untouched. Full suite 1333 passed, 1 skipped; tsc --noEmit clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NEW src/test/remote-coverage.test.ts (28 tests) covering five
previously-uncovered branch groups in remote.ts:
resolve() edge cases:
- merged content applied to both DBs
- merged without payload falls to Invalid choice
- invalid choice strings
- remote unreachable
- memory not found on either side
- localDb.insertMemory throw caught and reported
migrate() partial failures:
- happy path with projects + memories + META_LAST_SYNC
- remote unreachable returns clean error
- per-project insert failure continues the loop
- per-memory insert failure continues the loop
getMachineId / resolveHostname:
- HOSTNAME env var
- COMPUTERNAME fallback
- os.hostname() fallback
- os.hostname() throw → unknown- prefix
- v5.9.5 self-heal: stale unknown- id overwritten;
dream_machine_id healed when pointed at stale
- self-heal kept when hostname still cannot resolve
- stable cached non-stale id
- host- prefix is NOT treated as stale
getStatus SQLITE_BUSY:
- friendly message on SQLITE_BUSY
- non-busy sqlite errors rethrow
formatStatus + validateLocation + closeRemote:
- formatStatus: not configured / unreachable / conflicts / message
- validateLocation: create-dir warning, high-latency warning,
sqlite setMeta failure
- closeRemote clears the cached remoteDb handle
Coverage: remote.ts now 80.83% statements / 75% branches /
95.65% functions / 80.61% lines (gate: ≥80% lines). No source
changes; existing remote test files (remote, remote-audit-sync,
remote-resume, remote-two-machine) untouched. Full suite 1361
passed, 1 skipped; tsc --noEmit clean.
Note: identified dead code at line ~778 (resolve('merged') with
no base memory) — unreachable because line ~760 early-returns
when both sides are null. Flagged for OPEN-track cleanup; no
source patch in this coverage-only task.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NEW src/test/db-coverage.test.ts (14 tests) covering the plan-named
audit/dream-result query helpers:
getRecentDreamRuns:
- Default DESC ordering with parsed details
- limit truncation
- sinceIso filter
- JSON-parse catch (returns details: {} on bad JSON)
- failuresOnly: errors > 0 OR providerUnreachable arms
- failuresOnly: false returns all
- started fallback when startedAt is missing
- Default-limit happy path
getLastSuccessfulDreamRun:
- Empty audit_log → null
- Only failed runs → null
- Mixed success/fail → most recent successful row
- decay-only counts as successful
- relationships-only counts as successful
- summaries-only counts as successful
Coverage: db.ts now 85.06% statements / 78.07% branches / 92.3%
functions / 88.47% lines (gate: ≥80% lines; baseline 81.26% from
CC.1-CC.3 side effects, now +7.21 pp). No source changes; all 6
existing db test files (db-recovery, db-recovery-extended,
phase8a.central-db, phase9a.sandbox, v511-db-schema,
v593-no-central-db-pollution) untouched. Full suite 1375 passed,
1 skipped; tsc --noEmit clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NEW gnosys-public/docs/coverage-baseline.md — closes the CC follow-up track by documenting the post-CC.1-CC.4 coverage state. All five C.1 target files now meet the ≥80% lines gate: | File | C.1 Baseline | Post-CC.4 | Δ Lines | |---------------|--------------|-----------|---------| | mcpHttp.ts | 89% | 92.42% | +3.42 | | ingest.ts | 17% | 100% | +83 | | dream.ts | 29% | 95.42% | +66.42 | | remote.ts | 74% | 80.61% | +6.61 | | db.ts | 77% | 88.47% | +11.47 | The doc also records per-file statement/branch/function/line columns, overall totals, and attribution of each lift to its CC task (ingest-structured.test.ts / dream-coverage.test.ts / remote-coverage.test.ts / db-coverage.test.ts). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…L.1) Curates the 84 commits between v5.10.0 and HEAD into the four Keep-a-Changelog sub-sections under ## [Unreleased]: Added (15 bullet groups): - Network-hosted MCP transport (v5.12 Phases A-E): serve --transport http, capability replay thunks, gnosys connect, gnosys centralize, Docker support. - Structured logging (D.5) with text/JSON/file sinks. - Audit rows for remote sync. - HTTP CORS guard. - Atomic config writes. - Preference key validation with did-you-mean hints. - --json on 7 read-only CLI commands. - Provenance: source_file in reads. - Export excluded-archived count. - gnosys upgrade PM detection. - npm discoverability keywords. - Documentation and ADRs (E.2-E.8, A.13, CC.5). - Acceptance smokes (C.9). - Test coverage expansion (CC.1-CC.4 + others). Changed: CI matrix Linux+macOS (C.7), Node 18/20 CI, Biome (B.2), dep cleanup (B.3/B.4), DB-only history, CHANGELOG backfill (E.2), README updates, DB index perf. Fixed: path traversal (A.5), shell injection (A.8), file perms 0600 (A.11), clean dist (20.13), legacy schema migration, npm provenance, README/pkg fixes, MCP error envelopes, machine ID, busy timeout, embeddings hint, LLM timeouts, HTTP session cleanup. Security: HTTP auth on non-loopback, DoS body limits, SSRF parity (17.4), DOCX zip-bomb guard, API key redaction, prompt injection hardening, CORS default-deny. The ## [Unreleased] header carries no date; the date is added when the release is cut (REL.3). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The threat model already linked back to SECURITY.md from its header; this adds the missing forward reference so the two docs are properly cross-linked. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…N.2) Adds a new ## Documentation section between Project Structure and Testing that points future contributors to the content map (user- facing site vs in-repo source of truth vs Gnosys memory). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NEW gnosys-public/tsconfig.publish.json extends the dev tsconfig with sourceMap:false and declarationMap:false. Declarations (.d.ts) are preserved so TypeScript consumers still get types. Wires a publish-specific build pipeline in package.json: - prebuild:publish — clean dist mirror of the existing prebuild - build:publish — tsc -p tsconfig.publish.json - prepublishOnly — now invokes build:publish (was: npm run build) The dev workflow is unchanged: 'npm run build' still uses tsconfig.json and emits sourcemaps + declaration maps for local debugging. Impact: npm tarball unpacked size drops dramatically — 1.9 MB / 255 files in the publish build, down from ~7.4 MB with all 516+ .map sidecars. Faster installs, smaller footprint on disk for the ~80 MB-equivalent-size of all .map files removed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…* (OPEN.4 round 1)
Round 1 of the gradual D.5 follow-on migration. Converts exactly 10
catch-block / fatal-startup console.error calls to structured
logError/logWarn from src/lib/log.js. UI prints (per-chunk ingest
progress, user-facing summaries) are deliberately left as
console.error.
Sites converted:
- src/sandbox/server.ts:678 → logError(new Error('Failed to open
GnosysDB'), { module: 'sandbox', op: 'openDb', hint })
- src/sandbox/server.ts:680 → logWarn(`Network path may be
unavailable`, { module: 'sandbox', dbDir })
- src/sandbox/server.ts:709 → logError(err, { module: 'sandbox',
op: 'dreamInit' })
- src/lib/projectIdentity.ts:144 → logWarn(...) — project-move
notification (not a caught exception)
- src/lib/chat/index.ts:101 → logError(new Error('Session not
found: ...'), { module: 'chat', op: 'resume' })
- src/cli.ts:313, 354, 400, 443, 527 → logError(err, { module:
'cli', op: <command name> }) where <command name> is derived
from the surrounding program.command() block (discover, discover,
search, search, list)
Net effect: total src/ console.error count drops from 232 → 222
(−10). Each modified file imports logError/logWarn from
'../lib/log.js' (relative path adjusted per file depth).
Future rounds (OPEN.4.2, .3, ...) can continue the migration at
~10 high-signal sites per round; dream.ts progress prints and
genuine UI output remain out of scope.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds 4 new ADRs to docs/adr/ following the canonical 0001-0008 format (Status / Date / Memory metadata + Context / Decision / Consequences sections, 187-210 words each): - 0009-remote-first-reads.md — Remote-First Reads, Local-as- Offline-Only Cache (deci-037; supersedes deci-034). NAS is the source of truth; local DB is an offline-resilience cache, not a performance layer. - 0010-prompt-injection-threat-model.md — Prompt Injection Threat Model (deci-01KSGSX8SJXAVAY7EV2VS9YJJP, from task A.9). Bounded accepted risk; defend at the Gnosys boundary (no exfiltration primitives, SSRF guards, API-key redaction, provenance fields) without stripping legitimate instruction-like content. - 0011-readme-positioning.md — README Positioning: No Competitor Comparisons (deci-01KSGRQ4GEGPHJQMYDD3V2XCWK, from task 20.27). README stays minimal; gnosys.ai is the canonical positioning surface. - 0012-categorized-tag-registry.md — Categorized Tag Registry (dec-006). Tags live in .gnosys/tags.yml under named categories (domain, type, concern, status-tag); LLM proposes new tags but user approves before they're added; orthogonal to directory categories. docs/adr/README.md index updated with 4 new rows. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Upgrades the legacy 'Is better-sqlite3 installed?' phrasing to parity with the @huggingface/transformers install hint at src/lib/embeddings.ts:66. Sites updated: - src/index.ts:2312 — Archive-not-available MCP tool response - src/cli.ts:3720 — Archive command catch - src/cli.ts:6380, 6429, 6492 — 'Error: GnosysDB not available' catches - src/sandbox/server.ts:679 — logError hint field (already migrated to log.* in OPEN.4; this commit updates only the hint text) New phrasing (consistent across all 6 sites): '... Install it with: npm install better-sqlite3' Acceptance gate: grep -c 'npm install better-sqlite3' in src/ runtime error paths is now 6 (plan target: ≥4). Legacy phrasing count: 0. Pure string replacement; +6/-6 lines across 3 files. No structural changes, no test files touched. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
edtadros
force-pushed
the
feat/network-mcp
branch
2 times, most recently
from
59c144e to
b7dc75b
Compare
84 commits since v5.10.0. Highlights: Security hardening — path-traversal in export blocked (assertWithin), shell injection eliminated (argv arrays), .env/gnosys.db at mode 0600, HTTP MCP auth+CORS+body-limits+idle-reaper, SSRF safeFetch, ask layer prompt-injection-resistant. Coverage closure — every C.1 target file ≥80%: ingest.ts 100%, dream.ts 95%, db.ts 88%, remote.ts 80%, mcpHttp.ts 89%. Tooling — Biome lint, Node 20/22/24 × Linux+macOS CI matrix (Node 18 dropped; past EOL April 2025, toolchain needs node:util.styleText which is Node 20.12+), prebuild dist clean, knip dead-code, structured logger (GNOSYS_LOG_*), sourcemap-trimmed publish (tarball 7.4MB → 1.9MB), updated package metadata (keywords, repo URL canonicalized, optional-deps documented). Docs — generated docs/mcp-tools.md + docs/cli.md, threat-model.md, 12 ADRs (decisions backfilled from Gnosys memory), source-of-truth map, SECURITY.md update-integrity section, CHANGELOG historical-versions note. engines.node raised: >=18.0.0 → >=20.12.0. README prereq updated. See CHANGELOG.md ## [5.11.0] section for the full curated list. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
edtadros
force-pushed
the
feat/network-mcp
branch
from
b7dc75b to
64a40a1
Compare
edtadros
added a commit
that referenced
this pull request
May 30, 2026
Bring in the GitHub merge commit for v5.11.0 before pushing the feat/network-mcp fast-forward line.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bundles 84 commits since v5.10.0. Full curated changelog in
CHANGELOG.md(## [5.11.0] — 2026-05-26).Highlights
Security hardening
gnosys exportclosed (slugify category +assertWithinresolve+prefix check at every write site; A.5)execSync(\cp -a "${path}"`)` → argv arrays (A.8).envandgnosys.db(+ WAL sidecars) created mode0600, parent dirs0700(A.11)safeFetch/isSafeUrlblock loopback, RFC1918, link-local, IPv6, integer-encoded IPs, with per-hop redirect re-check (A.7)asksynthesis (rule 8 insynthesize.md); no MCP exfiltration primitive (A.9)Coverage closure — every C.1 target file ≥80% lines:
ingest.ts17% → 100% ·dream.ts29% → 95% ·db.ts81% → 88% ·remote.ts74% → 80% ·mcpHttp.ts89% (already there)Tooling
useImportTypeenforced; pragmatic ruleset; lint clean)prebuildclean step (orphan-freedist/); knip wired (no dead exports); jszip declaredsrc/lib/log.tswithGNOSYS_LOG_FORMAT=json/GNOSYS_LOG_FILE)tsconfig.publish.json+prepublishOnly): tarball 7.4 MB → 1.9 MB unpackedDocs
docs/mcp-tools.md+docs/cli.md(source-of-truth fromsrc/index.ts/src/cli.ts)docs/threat-model.md,docs/adr/(12 ADRs backfilled),docs/source-of-truth.md,docs/coverage-baseline.mdMetadata
repository.urlcanonicalized for npm provenancemodel-context-protocol,agent-memory)gnosys+gnosys-mcp) and the optional native depsNo breaking changes. SemVer minor bump.