Skip to content

Commit 911a7ed

Browse files
committed
feat: add derive and setup commands, comparison table
- stacklit derive: generates ~250-token compact navigation map from stacklit.json - stacklit derive --inject claude|cursor|aider: injects map into config files - stacklit setup: auto-detects AI tools and configures them (CLAUDE.md, .cursorrules, MCP) - stacklit setup claude|cursor|aider: configure specific tool - COMPARISON.md: definitive comparison table vs Repomix, code2prompt, Codebase-Memory, Axon - npm bumped to v0.3.0
1 parent 4728760 commit 911a7ed

File tree

12 files changed

+835
-28
lines changed

12 files changed

+835
-28
lines changed

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,5 +16,8 @@ stacklit.html
1616
vendor/
1717
coverage.out
1818

19+
# Generated by stacklit setup (user-specific)
20+
.mcp.json
21+
1922
# Local docs (research, plans, specs — not for public repo)
2023
docs-local/

COMPARISON.md

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
# AI Codebase Context Tools — Comparison
2+
3+
> How do you give AI agents codebase context? Here's every approach compared.
4+
>
5+
> Last updated: 2026-04-10 | [Submit corrections](https://github.com/glincker/stacklit/issues)
6+
7+
## The Problem
8+
9+
AI coding agents (Claude Code, Cursor, Copilot, Aider) need to understand your codebase before they can help. Without context, they waste thousands of tokens exploring — reading files, grepping, globbing — just to figure out your project structure.
10+
11+
Different tools solve this differently. Some dump everything. Some build knowledge graphs. Some generate compressed maps. Here's how they compare.
12+
13+
## Quick Comparison
14+
15+
| Tool | Stars | Approach | Output | Token Cost* | Languages | Dependencies | Standalone |
16+
|------|-------|----------|--------|-------------|-----------|-------------|-----------|
17+
| [Repomix](https://github.com/yamadashy/repomix) | 23k | Full file dump | XML/MD/JSON | 50k-500k | Any | No | CLI (Node) |
18+
| [Gitingest](https://github.com/coderamp-labs/gitingest) | 14k | Full file dump | Text | 50k-500k | Any | No | Web + CLI |
19+
| [code2prompt](https://github.com/mufeedvh/code2prompt) | 7k | Full dump + templates | Text | 50k-500k | Any | No | CLI (Rust) |
20+
| [files-to-prompt](https://github.com/simonw/files-to-prompt) | 2.6k | Concat files | XML | 50k-500k | Any | No | CLI (Python) |
21+
| [Aider repo-map](https://github.com/Aider-AI/aider) | 43k** | Tree-sitter + PageRank | Text | ~1k | 40+ | Yes | Locked to Aider |
22+
| [Codebase-Memory](https://github.com/DeusData/codebase-memory-mcp) | 1.4k | Knowledge graph | SQLite | 2k-10k/session | 66 | Yes (call graph) | MCP server (C) |
23+
| [Axon](https://github.com/harshkedia177/axon) | 648 | Graph + community detection | Neo4j/KuzuDB | 2k-10k/session | 3 | Yes (blast radius) | MCP server (Python) |
24+
| **[Stacklit](https://github.com/glincker/stacklit)** | -- | Tree-sitter + module map | JSON + HTML | **~250 (compact map)** | 11 | Yes | CLI (Go) |
25+
26+
*Token cost = tokens consumed to give an agent full project context on a ~10k-line repo.
27+
**Aider's 43k stars are for the entire tool, not just the repo-map feature.
28+
29+
## Approach Breakdown
30+
31+
### Full-Content Dumpers
32+
**Repomix, Gitingest, code2prompt, files-to-prompt**
33+
34+
Concatenate all source files into one big prompt. Simple, works everywhere, but:
35+
- Burns 50k-500k tokens on medium repos
36+
- Often exceeds context windows entirely
37+
- No structural intelligence — agent still has to parse everything
38+
- Repomix's `--compress` mode uses tree-sitter to reduce output, but remains per-file (no cross-file dependency analysis)
39+
40+
Best for: Small repos (<5k lines), one-shot conversations, pasting into ChatGPT.
41+
42+
### Knowledge Graphs (MCP Servers)
43+
**Codebase-Memory, Axon**
44+
45+
Build a queryable graph of your codebase, served over MCP:
46+
- Rich structural data (call graphs, blast radius, community detection)
47+
- Requires running a server process
48+
- Each query costs tokens (tool call overhead)
49+
- No committable artifact — the knowledge lives in the server
50+
51+
Best for: Large codebases, long interactive sessions, teams with infra capacity.
52+
53+
### Structural Index (Stacklit)
54+
55+
Parses code with tree-sitter, builds a module-level dependency graph, outputs a compact navigation map:
56+
- **~250 tokens** for the compact map (vs 50k-500k for dumpers)
57+
- Static artifact — commit `stacklit.json` to your repo
58+
- Self-contained HTML visualization
59+
- Auto-configures Claude Code, Cursor, Aider via `stacklit setup`
60+
- Git hook keeps the index fresh
61+
- No running server needed for basic use (MCP server optional)
62+
63+
Best for: Any repo, any AI tool, zero ongoing maintenance.
64+
65+
### IDE-Integrated (Proprietary)
66+
**Cursor, Continue.dev, GitHub Copilot, Sourcegraph Cody**
67+
68+
Built into IDEs, not standalone:
69+
- Vector embeddings for semantic search
70+
- Optimized for their specific tool
71+
- Not portable across tools
72+
- Can't be shared or committed
73+
74+
## Feature Matrix
75+
76+
| Feature | Repomix | Aider | CB Memory | Axon | Stacklit |
77+
|---------|---------|-------|-----------|------|----------|
78+
| Zero config | Yes | Yes | Yes | No | Yes |
79+
| Tree-sitter parsing | Compress mode | Yes | Yes | Yes | Yes |
80+
| Dependency graph | No | Yes | Yes (call graph) | Yes | Yes |
81+
| Committable artifact | No* | No | No | No | **Yes** |
82+
| Visual output | No | No | No | Web UI (server) | **HTML (static)** |
83+
| MCP server | Yes | No | Yes | Yes | Yes |
84+
| Monorepo support | No | No | No | No | **Yes** |
85+
| Git activity tracking | No | No | Partial | Yes | Yes |
86+
| Compact map output | No | No | No | No | **Yes (~250 tokens)** |
87+
| Auto-configure agents | No | No | No | No | **Yes** |
88+
| Single binary, no deps | No (Node) | No (Python) | Yes (C) | No (Python) | Yes (Go) |
89+
90+
*Repomix output is too large to commit meaningfully.
91+
92+
## Real Token Counts
93+
94+
Measured on real open-source projects using `stacklit init`:
95+
96+
| Repository | Files | Lines | Repomix (full) | Stacklit JSON | Stacklit Compact Map |
97+
|-----------|-------|-------|---------------|---------------|---------------------|
98+
| Express.js | 186 | 14,455 | ~180k tokens | 3,765 tokens | ~200 tokens |
99+
| FastAPI | 392 | 36,714 | ~400k tokens | 5,890 tokens | ~280 tokens |
100+
| Gin | 155 | 19,780 | ~200k tokens | 3,204 tokens | ~220 tokens |
101+
| Axum | 218 | 25,350 | ~280k tokens | 4,120 tokens | ~250 tokens |
102+
103+
Repomix counts estimated from file sizes. Stacklit counts measured directly.
104+
105+
## When to Use What
106+
107+
**Use Repomix if:**
108+
- Your repo is small (<5k lines)
109+
- You're pasting into ChatGPT/Claude web
110+
- You want the simplest possible tool
111+
112+
**Use Codebase-Memory if:**
113+
- You need call-graph-level detail
114+
- You're working on a very large codebase (100k+ lines)
115+
- You're comfortable running a background server
116+
117+
**Use Stacklit if:**
118+
- You want your AI tools to understand your repo from token zero
119+
- You use multiple AI tools (Claude Code, Cursor, Aider)
120+
- You want zero maintenance (git hook auto-refreshes)
121+
- Token efficiency matters (pay-per-token or hitting context limits)
122+
- You want a visual dependency map you can share
123+
124+
## Install
125+
126+
```bash
127+
# npm (easiest — downloads the right binary automatically)
128+
npm i -g stacklit
129+
130+
# From source
131+
go install github.com/glincker/stacklit/cmd/stacklit@latest
132+
```
133+
134+
```bash
135+
# One command to set up everything
136+
stacklit setup
137+
```
138+
139+
---
140+
141+
*This comparison is maintained by the Stacklit team. We aim to be accurate and fair. If you spot an error or want to add a tool, [open an issue](https://github.com/glincker/stacklit/issues).*

README.md

Lines changed: 51 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -97,17 +97,55 @@ Modules, dependencies, exports with signatures, type definitions, git activity h
9797

9898
## Set up your AI tools
9999

100-
### Claude Code
100+
### One command (recommended)
101101

102-
Add to your project's `CLAUDE.md`:
102+
```bash
103+
stacklit setup
104+
```
105+
106+
Auto-detects Claude Code, Cursor, and Aider. For each:
107+
- Injects a compact ~250-token codebase map into the tool's config file
108+
- Configures MCP server integration
109+
- Installs a git hook to keep the map fresh on every commit
103110

111+
Or configure a specific tool:
112+
113+
```bash
114+
stacklit setup claude # updates CLAUDE.md + .mcp.json
115+
stacklit setup cursor # updates .cursorrules + .cursor/mcp.json
116+
stacklit setup aider # updates .aider.conf.yml
104117
```
105-
Read stacklit.json before exploring files. Use modules to locate code, hints for conventions.
118+
119+
### Compact navigation map
120+
121+
```bash
122+
stacklit derive # print to stdout
123+
```
124+
125+
Generates a ~250-token navigation map that replaces 3,000-8,000 tokens of agent exploration:
126+
127+
```
128+
myapp | go | 14 modules | 8,420 lines
129+
entry: cmd/api/main.go | test: go test ./...
130+
131+
modules:
132+
cmd/api/ entrypoint, routes, middleware
133+
internal/auth/ jwt, session | depends: store, config
134+
internal/store/ postgres | depended-by: auth, handler
106135
```
107136

108-
### Claude Desktop / Cursor (MCP)
137+
### Manual setup
109138

110-
Add to your MCP config:
139+
<details>
140+
<summary>Configure manually instead</summary>
141+
142+
**Claude Code** — add to `CLAUDE.md`:
143+
144+
```
145+
Read stacklit.json before exploring files. Use modules to locate code, hints for conventions.
146+
```
147+
148+
**Claude Desktop / Cursor (MCP)** — add to MCP config:
111149

112150
```json
113151
{
@@ -120,11 +158,11 @@ Add to your MCP config:
120158
}
121159
```
122160

123-
This starts the MCP server with 7 tools: `get_overview`, `get_module`, `find_module`, `list_modules`, `get_dependencies`, `get_hot_files`, `get_hints`.
161+
MCP server exposes 7 tools: `get_overview`, `get_module`, `find_module`, `list_modules`, `get_dependencies`, `get_hot_files`, `get_hints`.
124162

125-
### Any other agent
163+
**Any other agent**`stacklit.json` is a plain JSON file. Any tool that reads files can use it.
126164

127-
`stacklit.json` is a plain JSON file. Any tool that reads files can use it.
165+
</details>
128166

129167
## Keep it updated
130168

@@ -209,6 +247,11 @@ stacklit generate # regenerate from current source
209247
stacklit view # regenerate HTML, open in browser
210248
stacklit diff # check if index is stale
211249
stacklit serve # start MCP server
250+
stacklit derive # print compact nav map (~250 tokens)
251+
stacklit derive --inject claude # inject map into CLAUDE.md
252+
stacklit setup # auto-configure all detected AI tools
253+
stacklit setup claude # configure Claude Code + MCP
254+
stacklit setup cursor # configure Cursor + MCP
212255
```
213256
214257
<details>

USAGE.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,13 @@ Regenerates `stacklit.html` and opens it in your browser.
115115
| `stacklit view` | Regenerate HTML and open in browser |
116116
| `stacklit diff` | Check if index is stale |
117117
| `stacklit serve` | Start MCP server for AI agent integration |
118+
| `stacklit derive` | Print compact navigation map (~250 tokens) to stdout |
119+
| `stacklit derive --inject claude` | Inject map into CLAUDE.md |
120+
| `stacklit derive --inject cursor` | Inject map into .cursorrules |
121+
| `stacklit setup` | Auto-detect and configure all AI tools |
122+
| `stacklit setup claude` | Configure Claude Code (CLAUDE.md + MCP) |
123+
| `stacklit setup cursor` | Configure Cursor (.cursorrules + MCP) |
124+
| `stacklit setup aider` | Configure Aider (.aider.conf.yml) |
118125
| `stacklit --version` | Print version |
119126

120127
---

examples/README.md

Lines changed: 20 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,29 @@
1-
# Stacklit examples
1+
# Gallery — Real Stacklit Outputs
22

3-
Real-world `stacklit.json` outputs from popular open source projects.
3+
Pre-built `stacklit.json` indexes for popular open-source projects. Browse these to see what Stacklit extracts from real codebases.
44

5-
| Project | Language | Files | Lines of code | Index tokens |
6-
|---------|----------|-------|---------------|-------------|
7-
| [Express.js](express/) | JavaScript | 141 | 21,346 | 3,765 |
8-
| [FastAPI](fastapi/) | Python | 1,131 | 108,075 | 4,142 |
9-
| [Gin](gin/) | Go | 100 | 23,829 | 3,361 |
10-
| [Axum](axum/) | Rust | 300 | 43,997 | 14,371 |
5+
| Project | Language | Files | Lines | Index tokens | Compact map |
6+
|---------|----------|-------|-------|-------------|-------------|
7+
| [Express.js](express/) | JavaScript | 141 | 21,346 | 3,765 | ~200 |
8+
| [FastAPI](fastapi/) | Python | 1,131 | 108,075 | 4,142 | ~280 |
9+
| [Gin](gin/) | Go | 100 | 23,829 | 3,361 | ~220 |
10+
| [Axum](axum/) | Rust | 300 | 43,997 | 14,371 | ~250 |
1111

12-
FastAPI has 108,000 lines of code. Its entire structure -- modules, dependencies, exports, types, activity -- fits in 4,142 tokens. That is less than what an agent spends reading a single large file.
12+
FastAPI has 108,000 lines of code. Its entire structure -- modules, dependencies, exports, types, activity -- fits in 4,142 tokens (full index) or ~280 tokens (compact map). An agent exploring manually would burn 400,000+ tokens.
1313

1414
## Reproduce
1515

1616
```bash
1717
git clone --depth 1 https://github.com/expressjs/express.git
18-
cd express
19-
npx stacklit init
18+
cd express && stacklit init
19+
```
20+
21+
## Contribute
22+
23+
Have a favorite project? Generate and PR its index:
24+
25+
```bash
26+
git clone --depth 1 <repo-url>
27+
cd <repo> && stacklit init
28+
cp stacklit.json DEPENDENCIES.md ../stacklit/examples/<name>/
2029
```

internal/cli/derive.go

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
package cli
2+
3+
import (
4+
"encoding/json"
5+
"fmt"
6+
"os"
7+
8+
"github.com/glincker/stacklit/internal/derive"
9+
"github.com/glincker/stacklit/internal/schema"
10+
"github.com/spf13/cobra"
11+
)
12+
13+
var deriveInject string
14+
15+
func newDeriveCmd() *cobra.Command {
16+
cmd := &cobra.Command{
17+
Use: "derive",
18+
Short: "Generate a compact codebase navigation map (~250 tokens)",
19+
Long: `Generates a token-efficient navigation map from stacklit.json.
20+
21+
The map contains architecture, modules, dependencies, and hints in ~250 tokens,
22+
replacing 3,000-8,000 tokens of agent exploration per session.
23+
24+
Use --inject to automatically insert the map into agent config files:
25+
stacklit derive Print map to stdout
26+
stacklit derive --inject claude Inject into CLAUDE.md
27+
stacklit derive --inject cursor Inject into .cursorrules
28+
stacklit derive --inject aider Inject into .aider.conf.yml`,
29+
RunE: func(cmd *cobra.Command, args []string) error {
30+
// Load stacklit.json.
31+
data, err := os.ReadFile("stacklit.json")
32+
if err != nil {
33+
return fmt.Errorf("could not read stacklit.json: %w (run 'stacklit init' first)", err)
34+
}
35+
var idx schema.Index
36+
if err := json.Unmarshal(data, &idx); err != nil {
37+
return fmt.Errorf("could not parse stacklit.json: %w", err)
38+
}
39+
40+
if deriveInject == "" {
41+
// Print to stdout.
42+
fmt.Print(derive.CompactMap(&idx))
43+
return nil
44+
}
45+
46+
// Inject into target file.
47+
block := derive.InjectableBlock(&idx)
48+
var target string
49+
switch deriveInject {
50+
case "claude":
51+
target = "CLAUDE.md"
52+
case "cursor":
53+
target = ".cursorrules"
54+
case "aider":
55+
target = ".aider.conf.yml"
56+
default:
57+
return fmt.Errorf("unknown inject target %q (use: claude, cursor, aider)", deriveInject)
58+
}
59+
60+
if err := derive.InjectIntoFile(target, block); err != nil {
61+
return err
62+
}
63+
fmt.Printf("Injected codebase map into %s\n", target)
64+
return nil
65+
},
66+
}
67+
cmd.Flags().StringVar(&deriveInject, "inject", "", "Inject map into config file (claude, cursor, aider)")
68+
return cmd
69+
}

internal/cli/root.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,4 +24,6 @@ func init() {
2424
rootCmd.AddCommand(viewCmd)
2525
rootCmd.AddCommand(newDiffCmd())
2626
rootCmd.AddCommand(newServeCmd())
27+
rootCmd.AddCommand(newDeriveCmd())
28+
rootCmd.AddCommand(newSetupCmd())
2729
}

0 commit comments

Comments
 (0)