feat(core,mcp): graph algorithms — PageRank, components, shortest path

.changeset/graph-algorithms.md

@@ -0,0 +1,11 @@
+---
+'@prosdevlab/dev-agent': patch
+---
+
+Graph algorithms for dev_map and dev_refs
+
+- `dev_map` hot paths now use PageRank over the weighted dependency graph — files depended on by other important files rank higher
+- `dev_map` shows connected subsystems ("Subsystems: packages/core (45 files), packages/cli (12 files)")
+- `dev_refs` new `traceTo` parameter traces the dependency chain between files through the call graph
+- All algorithms are hand-rolled pure functions (~230 lines), no new dependencies
+- Inspired by aider's repo map (PageRank over dependency graphs)

.claude/da-plans/README.md

@@ -9,9 +9,9 @@ Implementation deviations are logged at the bottom of each plan file.
 
 | Track | Description | Status |
 |-------|-------------|--------|
-| [Core](core/) | Scanner, vector storage, services, indexer | Phase 1: Merged, Phase 2: Merged (indexing rethink) |
+| [Core](core/) | Scanner, vector storage, services, indexer | Phase 1: Merged, Phase 2: Merged, Phase 3: Draft (graph cache) |
 | [CLI](cli/) | Command-line interface | Not started |
-| [MCP Server](mcp/) | Model Context Protocol server + adapters | Phase 1: Draft (tools improvement) |
+| [MCP Server](mcp/) | Model Context Protocol server + adapters | Phase 1: Merged (tools improvement) |
 | [Subagents](subagents/) | Coordinator, explorer, planner, GitHub agents | Not started |
 | [Integrations](integrations/) | Claude Code, VS Code, Cursor | Not started |
 | [Logger](logger/) | @prosdevlab/kero centralized logging | Not started |

.claude/da-plans/core/phase-3-graph-cache/3.1-index-time-graph.md

@@ -0,0 +1,106 @@
+# Part 3.1: Build and Save Dependency Graph at Index Time
+
+See [overview.md](overview.md) for architecture context.
+
+## Goal
+
+After `linearMerge` (full index) or `batchUpsertAndDelete` (incremental), build the
+dependency graph from the scan results and save it as JSON.
+
+## What changes
+
+### `packages/core/src/storage/path.ts`
+
+Add `dependencyGraph` to `getStorageFilePaths`:
+
+```typescript
+export function getStorageFilePaths(storagePath: string): {
+  vectors: string;
+  metadata: string;
+  watcherSnapshot: string;
+  dependencyGraph: string;  // NEW
+  // ... deprecated paths
+} {
+  return {
+    // ... existing
+    dependencyGraph: path.join(storagePath, 'dependency-graph.json'),
+  };
+}
+```
+
+### `packages/core/src/map/graph.ts`
+
+Add serialization/deserialization:
+
+```typescript
+export interface CachedGraph {
+  version: 1;
+  generatedAt: string;
+  nodeCount: number;
+  edgeCount: number;
+  graph: Record<string, WeightedEdge[]>;
+}
+
+export function serializeGraph(graph: Map<string, WeightedEdge[]>): string {
+  let edgeCount = 0;
+  const obj: Record<string, WeightedEdge[]> = {};
+  for (const [key, edges] of graph) {
+    obj[key] = edges;
+    edgeCount += edges.length;
+  }
+  return JSON.stringify({
+    version: 1,
+    generatedAt: new Date().toISOString(),
+    nodeCount: graph.size,
+    edgeCount,
+    graph: obj,
+  });
+}
+
+export function deserializeGraph(json: string): Map<string, WeightedEdge[]> | null {
+  try {
+    const data = JSON.parse(json);
+    if (data.version !== 1) return null;
+    const graph = new Map<string, WeightedEdge[]>();
+    for (const [key, edges] of Object.entries(data.graph)) {
+      graph.set(key, edges as WeightedEdge[]);
+    }
+    return graph;
+  } catch {
+    return null;
+  }
+}
+```
+
+### `packages/core/src/indexer/index.ts`
+
+After `linearMerge` completes in `index()`, build and save the graph:
+
+```typescript
+// After linearMerge (line ~180)
+const documents = prepareDocumentsForEmbedding(scanResult.documents);
+// ... linearMerge call ...
+
+// Build and cache dependency graph
+const graph = buildDependencyGraph(
+  documents.map(d => ({ id: d.id, score: 0, metadata: d.metadata }))
+);
+const graphJson = serializeGraph(graph);
+await fs.writeFile(filePaths.dependencyGraph, graphJson, 'utf-8');
+```
+
+## Tests
+
+| Test | What it verifies |
+|------|-----------------|
+| `serializeGraph` round-trips correctly | Serialize → deserialize → same graph |
+| `deserializeGraph` returns null for invalid JSON | Error handling |
+| `deserializeGraph` returns null for wrong version | Schema evolution |
+| `getStorageFilePaths` includes `dependencyGraph` | Path registration |
+| After `index()`, graph file exists | Integration |
+
+## Commit
+
+```
+feat(core): build and save dependency graph at index time
+```

.claude/da-plans/core/phase-3-graph-cache/3.2-load-on-demand.md

@@ -0,0 +1,109 @@
+# Part 3.2: Load Cached Graph in dev_map and dev_refs
+
+See [overview.md](overview.md) for architecture context.
+
+## Goal
+
+Replace `getAll(limit: 10000)` → `buildDependencyGraph()` in `dev_map` and `dev_refs`
+with loading the cached graph from disk. Falls back to current approach if graph file
+is missing or corrupted.
+
+## What changes
+
+### `packages/core/src/map/graph.ts`
+
+Add a loader that reads from disk with fallback:
+
+```typescript
+import * as fs from 'node:fs/promises';
+
+/**
+ * Load dependency graph from cache, or build from docs as fallback.
+ */
+export async function loadOrBuildGraph(
+  graphPath: string | undefined,
+  fallbackDocs: () => Promise<SearchResult[]>
+): Promise<Map<string, WeightedEdge[]>> {
+  // Try cached graph first
+  if (graphPath) {
+    try {
+      const json = await fs.readFile(graphPath, 'utf-8');
+      const graph = deserializeGraph(json);
+      if (graph) return graph;
+    } catch {
+      // File missing or unreadable — fall through to build
+    }
+  }
+
+  // Fallback: build from docs (current approach)
+  const docs = await fallbackDocs();
+  return buildDependencyGraph(docs);
+}
+```
+
+### `packages/core/src/map/index.ts`
+
+Replace the graph build in `generateCodebaseMap`:
+
+```typescript
+// Before (current):
+const graph = buildDependencyGraph(allDocs);
+
+// After:
+const graph = await loadOrBuildGraph(
+  context.graphPath,  // new optional field on MapGenerationContext
+  async () => allDocs  // fallback uses already-fetched docs
+);
+```
+
+Add `graphPath` to `MapGenerationContext`:
+
+```typescript
+export interface MapGenerationContext {
+  indexer: RepositoryIndexer;
+  gitExtractor?: LocalGitExtractor;
+  logger?: Logger;
+  graphPath?: string;  // NEW — path to cached dependency-graph.json
+}
+```
+
+### `packages/mcp-server/src/adapters/built-in/refs-adapter.ts`
+
+Replace the `getDependencyGraph` method:
+
+```typescript
+private async getDependencyGraph() {
+  const CACHE_TTL_MS = 60_000;
+  if (this.cachedGraph && Date.now() - this.cachedGraphTime < CACHE_TTL_MS) {
+    return this.cachedGraph;
+  }
+
+  // Try loading from disk first (no getAll needed)
+  this.cachedGraph = await loadOrBuildGraph(
+    this.graphPath,
+    async () => this.indexer!.getAll({ limit: 50000 })  // raised limit as fallback
+  );
+  this.cachedGraphTime = Date.now();
+  return this.cachedGraph;
+}
+```
+
+### `packages/mcp-server/bin/dev-agent-mcp.ts`
+
+Pass `graphPath` to both MapAdapter and RefsAdapter from `getStorageFilePaths`.
+
+## Tests
+
+| Test | What it verifies |
+|------|-----------------|
+| `loadOrBuildGraph` with valid cached file | Loads from disk, doesn't call fallback |
+| `loadOrBuildGraph` with missing file | Calls fallback, builds from docs |
+| `loadOrBuildGraph` with corrupted file | Calls fallback, doesn't crash |
+| `generateCodebaseMap` uses cached graph when available | Integration |
+| `dev_refs dependsOn` uses cached graph | Integration |
+
+## Commit
+
+```
+feat(core,mcp): load cached dependency graph in dev_map and dev_refs
+```

.claude/da-plans/core/phase-3-graph-cache/3.3-incremental-graph.md

@@ -0,0 +1,89 @@
+# Part 3.3: Incremental Graph Updates via File Watcher
+
+See [overview.md](overview.md) for architecture context.
+
+## Goal
+
+When the file watcher detects changes and calls `applyIncremental`, update the
+cached dependency graph without a full rebuild. This keeps the graph fresh as
+files are edited.
+
+## What changes
+
+### `packages/core/src/map/graph.ts`
+
+Add an incremental update function:
+
+```typescript
+/**
+ * Update a dependency graph incrementally.
+ *
+ * For changed/new files: remove old edges from those files, add new edges.
+ * For deleted files: remove all edges from those files.
+ * Pure function — returns a new graph.
+ */
+export function updateGraphIncremental(
+  existing: Map<string, WeightedEdge[]>,
+  changedDocs: SearchResult[],
+  deletedFiles: string[]
+): Map<string, WeightedEdge[]> {
+  const updated = new Map(existing);
+
+  // Remove edges for deleted files
+  for (const file of deletedFiles) {
+    updated.delete(file);
+  }
+
+  // Remove old edges for changed files, then add new ones
+  const changedGraph = buildDependencyGraph(changedDocs);
+  for (const file of changedGraph.keys()) {
+    // Remove old edges (the file was re-scanned)
+    updated.delete(file);
+  }
+  for (const [file, edges] of changedGraph) {
+    updated.set(file, edges);
+  }
+
+  return updated;
+}
+```
+
+### `packages/core/src/indexer/index.ts`
+
+In `applyIncremental`, update the cached graph:
+
+```typescript
+async applyIncremental(upserts: EmbeddingDocument[], deleteIds: string[]): Promise<void> {
+  await this.vectorStorage.batchUpsertAndDelete(upserts, deleteIds);
+
+  // Update cached dependency graph
+  const graphPath = getStorageFilePaths(this.config.vectorStorePath).dependencyGraph;
+  try {
+    const existing = await loadGraphFromDisk(graphPath);
+    if (existing) {
+      const deletedFiles = extractFilesFromDeleteIds(deleteIds);
+      const changedDocs = upserts.map(d => ({ id: d.id, score: 0, metadata: d.metadata }));
+      const updated = updateGraphIncremental(existing, changedDocs, deletedFiles);
+      await fs.writeFile(graphPath, serializeGraph(updated), 'utf-8');
+    }
+  } catch {
+    // Graph update failed — next full index will fix it
+  }
+}
+```
+
+## Tests
+
+| Test | What it verifies |
+|------|-----------------|
+| `updateGraphIncremental` adds edges for new files | New file → new edges appear |
+| `updateGraphIncremental` removes edges for deleted files | Deleted file → edges gone |
+| `updateGraphIncremental` replaces edges for changed files | Changed file → old edges removed, new edges added |
+| `updateGraphIncremental` with empty existing graph | Handles first incremental gracefully |
+| Incremental update failure doesn't crash indexer | Error resilience |
+
+## Commit
+
+```
+feat(core): incremental dependency graph updates via file watcher
+```