Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
# Changelog

## v1.5.0 — ChatGPT Actions (2026-03-26)

### Added
- **OpenAPI 3.1.0 spec** (`openapi.rs`) — hand-written spec for all 23 endpoints, served at `GET /openapi.json`
- **ChatGPT plugin manifest** — served at `GET /.well-known/ai-plugin.json`
- **`--setup-chatgpt` CLI helper** — interactive setup: enables HTTP, creates API key, configures CORS, prompts for public URL
- **Plugin config** — `[http.plugin]` section for name, description, contact_email, public_url

### Changed
- Module count: 25 → 26
- Test count: 417 → 426
- `/openapi.json` and `/.well-known/ai-plugin.json` routes require no authentication

## v1.4.0 — PARA Migration (2026-03-26)

### Added
Expand Down
9 changes: 5 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Local knowledge graph + intelligence layer for Obsidian vaults. Rust CLI + MCP s

## Architecture

Single binary with 25 modules behind a lib crate:
Single binary with 26 modules behind a lib crate:

- `config.rs` — loads `~/.engraph/config.toml` and `vault.toml`, merges CLI args, provides `data_dir()`. Includes `intelligence: Option<bool>`, `[models]` section for model overrides, `[obsidian]` section (CLI path, enabled flag), and `[agents]` section (registered AI agent names). `Config::save()` writes back to disk.
- `chunker.rs` — smart chunking with break-point scoring algorithm. Finds optimal split points considering headings, code fences, blank lines, and thematic breaks. `split_oversized_chunks()` handles token-aware secondary splitting with overlap
Expand All @@ -23,16 +23,17 @@ Single binary with 25 modules behind a lib crate:
- `placement.rs` — folder placement engine. Uses folder centroids (online mean of embeddings per folder) to suggest the best folder for new notes. Falls back to inbox when confidence is low. Includes placement correction detection (`detect_correction_from_frontmatter`) and frontmatter stripping for moved files
- `writer.rs` — write pipeline orchestrator. 5-step pipeline: resolve tags (fuzzy match + register new), discover links (exact + fuzzy), place in folder, atomic file write (temp + rename), and index update. Supports create, append, update_metadata, move_note, archive, unarchive, edit (section-level replace/prepend/append), rewrite (full content with frontmatter preservation), edit_frontmatter (granular set/remove/add_tag/remove_tag/add_alias/remove_alias ops), and delete (soft archive or hard permanent) operations with mtime-based conflict detection and crash recovery via temp file cleanup
- `watcher.rs` — file watcher for `engraph serve`. OS thread producer (notify-debouncer-full, 2s debounce) sends `Vec<WatchEvent>` over tokio::mpsc to async consumer task. Two-pass batch processing: mutations (index_file/remove_file/rename_file) then edge rebuild. Move detection via content hash matching. Placement correction on file moves. Centroid adjustment on file add/remove. Startup reconciliation via `run_index_shared`. `recent_writes` map coordination with MCP server to prevent double re-indexing of files written through the write pipeline
- `serve.rs` — MCP stdio server via rmcp SDK. Exposes 22 tools: 8 read (search, read, read_section, list, vault_map, who, project, context) + 10 write (create, append, update_metadata, move_note, archive, unarchive, edit, rewrite, edit_frontmatter, delete) + 1 diagnostic (health) + 3 migrate (migrate_preview, migrate_apply, migrate_undo). `edit_frontmatter` replaces `update_metadata` for granular frontmatter mutations. EngraphServer struct with Arc+Mutex wrapping for async handlers. Loads intelligence models (orchestrator + reranker) when enabled, wires into `search_with_intelligence`. Spawns file watcher on startup. CLI events table provides audit log for write operations. `recent_writes` map prevents double re-indexing of MCP-written files
- `serve.rs` — MCP stdio server via rmcp SDK. Exposes 22 tools: 8 read (search, read, read_section, list, vault_map, who, project, context) + 10 write (create, append, update_metadata, move_note, archive, unarchive, edit, rewrite, edit_frontmatter, delete) + 1 diagnostic (health) + 3 migrate (migrate_preview, migrate_apply, migrate_undo). `edit_frontmatter` replaces `update_metadata` for granular frontmatter mutations. EngraphServer struct with Arc+Mutex wrapping for async handlers. Loads intelligence models (orchestrator + reranker) when enabled, wires into `search_with_intelligence`. Spawns file watcher on startup. CLI events table provides audit log for write operations. `recent_writes` map prevents double re-indexing of MCP-written files. HTTP mode also serves `openapi.rs` routes (`/openapi.json`, `/.well-known/ai-plugin.json`) with no auth required
- `http.rs` — axum-based HTTP REST API server, enabled via `engraph serve --http`. 23 REST endpoints mirroring all 22 MCP tools + update-metadata. API key authentication with `eg_` prefixed keys and read/write permission levels. Per-key token bucket rate limiting (configurable requests/minute). CORS with configurable allowed origins for web-based agents. `--no-auth` mode for local development (127.0.0.1 only). Graceful shutdown via `CancellationToken` coordinating MCP + HTTP + watcher exit
- `openapi.rs` — OpenAPI 3.1.0 spec builder and ChatGPT plugin manifest. Hand-written spec for all 23 HTTP endpoints, served at `GET /openapi.json`. Plugin manifest served at `GET /.well-known/ai-plugin.json`. Both routes require no authentication. `[http.plugin]` config section for name, description, contact_email, and public_url. Used by `engraph configure --setup-chatgpt` for interactive ChatGPT Actions setup
- `graph.rs` — vault graph agent. Extracts wikilink targets, expands search results by following graph connections 1-2 hops. Relevance filtering via FTS5 term check and shared tags
- `profile.rs` — vault profile detection. Auto-detects PARA/Folders/Flat structure, vault type (Obsidian/Logseq/Plain), wikilinks, frontmatter, tags. Content-based role detection for people/daily/archive folders by content patterns (not just names). Writes/loads `vault.toml`
- `store.rs` — SQLite persistence. Tables: `meta`, `files` (with docid, created_by), `chunks` (with vector BLOBs), `chunks_fts` (FTS5), `edges` (vault graph), `tombstones`, `tag_registry`, `folder_centroids`, `placement_corrections`, `link_skiplist` (reserved), `llm_cache` (orchestrator result cache), `cli_events` (audit log for CLI operations). `vec_chunks` virtual table (sqlite-vec) for KNN search. Dynamic embedding dimension stored in meta. `has_dimension_mismatch()` and `reset_for_reindex()` for migration. Enhanced `resolve_file()` with fuzzy Levenshtein matching as final fallback
- `indexer.rs` — orchestrates vault walking (via `ignore` crate for `.gitignore` support), diffing, chunking, embedding, writes to store + sqlite-vec + FTS5, vault graph edge building (wikilinks + people detection), and folder centroid computation. Exposes `index_file`, `remove_file`, `rename_file` as public per-file functions. `run_index_shared` accepts external store/embedder for watcher FullRescan. Dimension migration on model change.
- `temporal.rs` — temporal search lane. Extracts note dates from frontmatter `date:` field or `YYYY-MM-DD` filename patterns. Heuristic date parsing for natural language ("today", "yesterday", "last week", "this month", "recent", month names, ISO dates, date ranges). Smooth decay scoring for files near but outside target date range. Provides `extract_note_date()` for indexing and `score_temporal()` + `parse_date_range_heuristic()` for search
- `search.rs` — hybrid search orchestrator. `search_with_intelligence()` runs the full pipeline: orchestrate (intent + expansions) → 5-lane RRF retrieval (semantic + FTS5 + graph + reranker + temporal) per expansion → two-pass RRF fusion. `search_internal()` is a thin wrapper without intelligence models. Adaptive lane weights per query intent including temporal (1.5 weight for time-aware queries). Results display normalized confidence percentages (0-100%) instead of raw RRF scores.

`main.rs` is a thin clap CLI (async via `#[tokio::main]`). Subcommands: `index` (with progress bar), `search` (with `--explain`, loads intelligence models when enabled), `status` (shows intelligence state + date coverage stats), `clear`, `init` (intelligence onboarding prompt, detects Obsidian CLI + AI agents), `configure` (`--enable-intelligence`, `--disable-intelligence`, `--model`, `--obsidian-cli`, `--no-obsidian-cli`, `--agent`, `--add-api-key`, `--list-api-keys`, `--revoke-api-key`), `models`, `graph` (show/stats), `context` (read/list/vault-map/who/project/topic), `write` (create/append/update-metadata/move/edit/rewrite/edit-frontmatter/delete), `migrate` (para with `--preview`/`--apply`/`--undo` for PARA vault restructuring), `serve` (MCP stdio server with file watcher + intelligence + optional `--http`/`--port`/`--host`/`--no-auth` for HTTP REST API).
`main.rs` is a thin clap CLI (async via `#[tokio::main]`). Subcommands: `index` (with progress bar), `search` (with `--explain`, loads intelligence models when enabled), `status` (shows intelligence state + date coverage stats), `clear`, `init` (intelligence onboarding prompt, detects Obsidian CLI + AI agents), `configure` (`--enable-intelligence`, `--disable-intelligence`, `--model`, `--obsidian-cli`, `--no-obsidian-cli`, `--agent`, `--add-api-key`, `--list-api-keys`, `--revoke-api-key`, `--setup-chatgpt`), `models`, `graph` (show/stats), `context` (read/list/vault-map/who/project/topic), `write` (create/append/update-metadata/move/edit/rewrite/edit-frontmatter/delete), `migrate` (para with `--preview`/`--apply`/`--undo` for PARA vault restructuring), `serve` (MCP stdio server with file watcher + intelligence + optional `--http`/`--port`/`--host`/`--no-auth` for HTTP REST API).

## Key patterns

Expand Down Expand Up @@ -81,7 +82,7 @@ Single vault only. Re-indexing a different vault path triggers a confirmation pr

## Testing

- Unit tests in each module (`cargo test --lib`) — 417 tests, no network required
- Unit tests in each module (`cargo test --lib`) — 426 tests, no network required
- Integration tests (`cargo test --test integration -- --ignored`) — require GGUF model download
- Build requires CMake (for llama.cpp C++ compilation)

Expand Down
58 changes: 54 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -374,6 +374,55 @@ Notes that don't match any signal with sufficient confidence stay in place. Dail

**HTTP endpoints:** `POST /api/migrate/preview`, `/api/migrate/apply`, `/api/migrate/undo` — available via `engraph serve --http`.

## ChatGPT Actions

`engraph serve --http` can expose your vault to ChatGPT as a custom Action. engraph generates a standards-compliant OpenAPI 3.1.0 spec and a ChatGPT plugin manifest automatically.

**Set up:**

```bash
engraph configure --setup-chatgpt
```

Interactive helper that:
1. Enables HTTP mode (if not already on)
2. Creates a write-permission API key
3. Configures CORS for `https://chat.openai.com`
4. Prompts for your public URL (ngrok or similar)

**Endpoints served automatically (no auth required):**

| Endpoint | Description |
|----------|-------------|
| `GET /openapi.json` | OpenAPI 3.1.0 spec for all 23 endpoints |
| `GET /.well-known/ai-plugin.json` | ChatGPT plugin manifest |

**Quick setup steps:**

```bash
# 1. Run the setup helper
engraph configure --setup-chatgpt

# 2. Start engraph with HTTP
engraph serve --http

# 3. Expose via tunnel (example with ngrok)
ngrok http 3030

# 4. In ChatGPT → Explore GPTs → Create → Configure → Add Action
# Import from URL: https://<your-ngrok-url>/openapi.json
```

**Plugin config in `~/.engraph/config.toml`:**

```toml
[http.plugin]
name = "My Vault"
description = "Search and read my personal knowledge base"
contact_email = "you@example.com"
public_url = "https://abc123.ngrok.io"
```

## Use cases

**AI-assisted knowledge work** — Give Claude or Cursor deep access to your personal knowledge base. Instead of copy-pasting context, the agent searches, reads, and cross-references your notes directly.
Expand Down Expand Up @@ -425,7 +474,7 @@ engraph is not a replacement for Obsidian — it's the intelligence layer that s
- Content-based folder role detection (people, daily, archive) by content patterns
- PARA migration: AI-assisted vault restructuring into Projects/Areas/Resources/Archive with preview, apply, and undo workflow
- Configurable model overrides for multilingual support
- 417 unit tests, CI on macOS + Ubuntu
- 426 unit tests, CI on macOS + Ubuntu

## Roadmap

Expand All @@ -437,7 +486,8 @@ engraph is not a replacement for Obsidian — it's the intelligence layer that s
- [x] ~~Temporal search — find notes by time period, date-aware queries~~ (v1.2)
- [x] ~~HTTP/REST API — complement MCP with a standard web API~~ (v1.3)
- [x] ~~PARA migration — AI-assisted vault restructuring with preview/apply/undo~~ (v1.4)
- [ ] Multi-vault — search across multiple vaults (v1.5)
- [x] ~~ChatGPT Actions — OpenAPI 3.1.0 spec + plugin manifest + `--setup-chatgpt` helper~~ (v1.5)
- [ ] Multi-vault — search across multiple vaults (v1.6)

## Configuration

Expand Down Expand Up @@ -471,7 +521,7 @@ All data stored in `~/.engraph/` — single SQLite database (~10MB typical), GGU
## Development

```bash
cargo test --lib # 417 unit tests, no network (requires CMake for llama.cpp)
cargo test --lib # 426 unit tests, no network (requires CMake for llama.cpp)
cargo clippy -- -D warnings
cargo fmt --check

Expand All @@ -483,7 +533,7 @@ cargo test --test integration -- --ignored

Contributions welcome. Please open an issue first to discuss what you'd like to change.

The codebase is 25 Rust modules behind a lib crate. `CLAUDE.md` in the repo root has detailed architecture documentation for AI-assisted development.
The codebase is 26 Rust modules behind a lib crate. `CLAUDE.md` in the repo root has detailed architecture documentation for AI-assisted development.

## License

Expand Down
23 changes: 23 additions & 0 deletions src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,15 @@ pub struct AgentsConfig {
pub windsurf: bool,
}

/// ChatGPT Actions plugin metadata.
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct PluginConfig {
pub name: Option<String>,
pub description: Option<String>,
pub contact_email: Option<String>,
pub public_url: Option<String>,
}

/// HTTP REST API configuration.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(default)]
Expand All @@ -44,6 +53,8 @@ pub struct HttpConfig {
pub rate_limit: u32, // requests per minute per key, 0 = unlimited
pub cors_origins: Vec<String>,
pub api_keys: Vec<ApiKeyConfig>,
#[serde(default)]
pub plugin: PluginConfig,
}

impl Default for HttpConfig {
Expand All @@ -55,6 +66,7 @@ impl Default for HttpConfig {
rate_limit: 60,
cors_origins: vec![],
api_keys: vec![],
plugin: PluginConfig::default(),
}
}
}
Expand Down Expand Up @@ -356,4 +368,15 @@ permissions = "read"
Some("hf:custom/model/embed.gguf".into())
);
}

#[test]
fn test_config_with_plugin() {
let toml = r#"
[http.plugin]
name = "my-vault"
public_url = "https://vault.example.com"
"#;
let config: Config = toml::from_str(toml).unwrap();
assert_eq!(config.http.plugin.name.as_deref(), Some("my-vault"));
}
}
70 changes: 70 additions & 0 deletions src/http.rs
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,9 @@ pub fn build_router(state: ApiState) -> Router {
.route("/api/migrate/preview", post(handle_migrate_preview))
.route("/api/migrate/apply", post(handle_migrate_apply))
.route("/api/migrate/undo", post(handle_migrate_undo))
// OpenAPI / ChatGPT plugin discovery (no auth required)
.route("/openapi.json", get(handle_openapi))
.route("/.well-known/ai-plugin.json", get(handle_plugin_manifest))
.layer(cors)
.with_state(state)
}
Expand All @@ -388,6 +391,36 @@ async fn health_check() -> &'static str {
"ok"
}

async fn handle_openapi(State(state): State<ApiState>) -> impl IntoResponse {
let default_url = format!(
"http://{}:{}",
state.http_config.host, state.http_config.port
);
let server_url = state
.http_config
.plugin
.public_url
.as_deref()
.unwrap_or(&default_url);
let spec = crate::openapi::build_openapi_spec(server_url);
Json(spec)
}

async fn handle_plugin_manifest(State(state): State<ApiState>) -> impl IntoResponse {
let default_url = format!(
"http://{}:{}",
state.http_config.host, state.http_config.port
);
let server_url = state
.http_config
.plugin
.public_url
.as_deref()
.unwrap_or(&default_url);
let manifest = crate::openapi::build_plugin_manifest(&state.http_config, server_url);
Json(manifest)
}

// ---------------------------------------------------------------------------
// Read endpoint handlers
// ---------------------------------------------------------------------------
Expand Down Expand Up @@ -941,6 +974,7 @@ mod tests {
permissions: "write".into(),
},
],
plugin: crate::config::PluginConfig::default(),
}
}

Expand Down Expand Up @@ -1272,4 +1306,40 @@ mod tests {
assert_eq!(response.status(), StatusCode::TOO_MANY_REQUESTS);
assert!(response.headers().get("retry-after").is_some());
}

// -----------------------------------------------------------------------
// OpenAPI / Plugin manifest tests (no auth required)
// -----------------------------------------------------------------------

#[tokio::test]
async fn test_openapi_no_auth_required() {
let state = test_api_state();
let app = build_router(state);
let response = app
.oneshot(
axum::http::Request::builder()
.uri("/openapi.json")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::OK);
}

#[tokio::test]
async fn test_plugin_manifest_no_auth_required() {
let state = test_api_state();
let app = build_router(state);
let response = app
.oneshot(
axum::http::Request::builder()
.uri("/.well-known/ai-plugin.json")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::OK);
}
}
1 change: 1 addition & 0 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ pub mod llm;
pub mod markdown;
pub mod migrate;
pub mod obsidian;
pub mod openapi;
pub mod placement;
pub mod profile;
pub mod search;
Expand Down
Loading
Loading