diff --git a/README.md b/README.md index 18bd73d..e5a0f80 100644 --- a/README.md +++ b/README.md @@ -11,128 +11,12 @@ Give your OpenClaw agents access to a 4-agent swarm with ~2M token context for c ## Features -<<<<<<< HEAD - **4-Agent Swarm** — Grok 4.20 coordinates multiple agents for deeper analysis - **Massive Context** — ~2M token window, handles entire codebases - **5 Modes** — Analyze, Refactor, Code, Reason, Orchestrate - **Tool Passthrough** — Pass OpenAI-format tool schemas for function calling - **File Writing** — Write annotated code blocks directly to disk - **Timeout Safety** — Process-level timeout enforcement prevents hangs -======= -You've been building with AI coding agents for a while now. They're great — they can write features, refactor modules, analyze codebases. But there's always been this ceiling. The models they run on are designed for single-turn conversations. They can collaborate with themselves behind the scenes to think through complex problems. - -**Enter Grok 4.20 Multi-Agent Beta.** - -It's different. Instead of one model responding, it's *four agents coordinating* in real-time. An orchestrator, specialists, critics — all working together to break down your request and reason through it from multiple angles. It can hold ~2M tokens of context — that's entire codebases in a single request. - -**The Problem:** - -Grok 4.20 is groundbreaking, but it doesn't play nicely with current coding tools. Claude Code doesn't have a Grok integration. OpenClaw's tooling system doesn't support multi-agent swarms. If you wanted to use Grok, you'd have to hack together custom scripts or modify your platform's core components. Not ideal. - -**The Solution:** - -This plugin bridges that gap. It makes Grok 4.20 available as a tool that any agent in Claude Code or OpenClaw can call. No core modifications, no hacking — just install and go. - -Now when your agent needs deep codebase analysis, large-scale refactoring, or complex reasoning, it can delegate to Grok's swarm and get back the kind of coordinated, multi-perspective thinking that single models can't deliver. - ---- - -## File Writing Capabilities - -Grok Swarm can now **write files directly** from code blocks in its responses. This solves the context flooding problem — Grok writes files to disk, and only a brief summary returns to your agent. - -### How It Works - -Grok responses with code blocks like: -```` -```python src/auth.py -import jwt -... -``` -```` - -Are automatically parsed and written to the output directory. - -### Usage - -| Command | Behavior | -|---------|----------| -| `--output-dir ./src` | Preview files (dry-run) | -| `--apply --output-dir ./src` | Write files to `./src` | -| `--apply --execute "make test"` | Write files, then run tests | - -### Example Workflow - -```bash -# Ask Grok to generate a module and write it -python -m src.bridge.cli code \ - --prompt "Write a FastAPI auth module with JWT" \ - --output-dir ./src \ - --apply - -# Then run tests -python -m src.bridge.cli code \ - --prompt "Refactor auth to use async" \ - --apply --execute "pytest tests/" \ - --output-dir ./src -``` - -### Morph LLM Integration - -For **partial file edits** (not full replacement), use the `--use-morph` flag: - -```bash -python -m src.bridge.cli refactor \ - --prompt "Convert this function to async" \ - --use-morph --apply -``` - -This requires Morph LLM MCP installed: - -```bash -claude mcp add morphllm -``` - ---- - -## Known Limitations - -> **Note:** When using `--apply`, Grok Swarm parses code blocks and writes files. For targeted edits within existing files, use `--use-morph` (requires Morph LLM MCP). - -### Why This Matters - -Grok can hold ~1.5M tokens of context and generate ~350K token responses. If that entire response floods back through your orchestrator's context window, you've just wasted precious tokens and slowed down your agent. - -```text -Current Flow: -Files (1.5M tokens) → Grok → Full response (376K) → Orchestrator (flooded!) - -Ideal Flow: -Files (1.5M tokens) → Grok → Writes files + brief summary → Orchestrator (clean) -``` - -### Current Use Cases - -Grok Swarm now supports **direct file writing**: -- ✅ **Code generation with file output** — Grok writes files directly -- ✅ **Codebase analysis** — Security audits, architecture reviews -- ✅ **Refactoring with partial edits** — Use `--use-morph` for targeted changes -- ✅ **Complex reasoning** — Research synthesis, decision making - -The `--apply` flag makes Grok write files to disk. Combined with `--execute`, you can build generate → test workflows. - ---- - -## What It Does - -| Feature | Why It Matters | -|----------|------------------| -| **4-Agent Coordination** | Multiple perspectives on every request | -| **2M Token Context** | Holds entire codebases without truncation | -| **5 Task Modes** | Analyze, Refactor, Generate, Reason, Orchestrate | -| **Dual Platform** | Works in both Claude Code and OpenClaw | -| **Zero Core Changes** | Drop-in tool, no platform modifications | ->>>>>>> 574902f (Add file writing and Morph LLM integration) --- @@ -143,19 +27,17 @@ When `write_files=true`, Grok parses code blocks for filename annotations and wr ### Supported Patterns **Fenced code blocks with path in the language tag:** -```markdown -```typescript:src/auth/login.ts -export function login() { ... } -``` -``` + + ```typescript:src/auth/login.ts + export function login() { ... } + ``` **Fenced code blocks with `// FILE:` marker:** -```markdown -```typescript -// FILE: src/auth/login.ts -export function login() { ... } -``` -``` + + ```typescript + // FILE: src/auth/login.ts + export function login() { ... } + ``` ### Example @@ -309,15 +191,19 @@ ls ~/.openclaw/skills/grok-refactor/grok_bridge.py Grok Swarm resolves your API key in this order (highest to lowest priority): -1. **Environment variables** — `OPENROUTER_API_KEY` or `XAI_API_KEY` +1. **Environment variables** — `OPENROUTER_API_KEY` or `OPENCLAW_OPENROUTER_DEFAULT_KEY` 2. **Local config file** — `~/.config/grok-swarm/config.json` with `{"api_key": "..."}` -3. **OpenClaw auth profiles** — `~/.openclaw/agents/coder/agent/auth-profiles.json` +3. **OpenClaw auth profiles** — searched in order: + - `~/.openclaw/agents/coder/agent/auth-profiles.json` + - `~/.openclaw/agents/main/agent/auth-profiles.json` + - `~/.openclaw/auth-profiles.json` + - `~/.config/openclaw/auth-profiles.json` ```bash -# If you set an env var, it takes precedence over config files: -export OPENROUTER_API_KEY="sk-or-v1-xxx" # This overrides ~/.config/grok-swarm/config.json! +# Set via environment variable (highest priority): +export OPENROUTER_API_KEY="sk-or-v1-xxx" -# To use the local config file instead, unset the env var: +# To use a lower-priority source instead, unset the env var: unset OPENROUTER_API_KEY ``` @@ -366,4 +252,4 @@ MIT — see [LICENSE](LICENSE) ## Support - [Issues](https://github.com/KHAEntertainment/grok-multiagent-plugin/issues) -- [Discord](https://discord.com/invite/clawd) +- [Discord](https://discord.com/invite/clawd) \ No newline at end of file diff --git a/src/bridge/grok_bridge.py b/src/bridge/grok_bridge.py old mode 100755 new mode 100644 index f7eab2b..6569d21 --- a/src/bridge/grok_bridge.py +++ b/src/bridge/grok_bridge.py @@ -14,6 +14,7 @@ import argparse import json import os +import re import sys import time from pathlib import Path @@ -146,6 +147,78 @@ def load_tools(tools_path): return tools +def _safe_dest(output_path, file_path): + """ + Resolve ``file_path`` relative to ``output_path`` and verify the result + stays inside ``output_path``. Returns the resolved Path or raises + ValueError for unsafe paths (absolute, containing ``..``, etc.). + """ + raw = Path(file_path) + if raw.is_absolute(): + raise ValueError(f"Absolute paths are not allowed: {file_path!r}") + if ".." in raw.parts: + raise ValueError(f"Path traversal not allowed: {file_path!r}") + dest = (output_path / raw).resolve() + resolved_root = output_path.resolve() + try: + dest.relative_to(resolved_root) + except ValueError: + raise ValueError(f"Path escapes output directory: {file_path!r}") + return dest + + +def parse_and_write_files(response_text, output_dir): + """ + Scan response for fenced code blocks with filename annotations and write to disk. + + Supports patterns: + ```lang:path/to/file ... ``` + ```lang + // FILE: path/to/file + ... + ``` + + Returns list of (relative_path, byte_count) tuples written, where + byte_count is the number of UTF-8 bytes written. + """ + written = [] + output_path = Path(output_dir) + + # Pattern for lang:path at start of block (language tag contains path) + lang_path_pattern = re.compile(r'^(\w+):([^\s\n]+)\n', re.MULTILINE) + # Pattern for // FILE: or # FILE: markers + file_marker_pattern = re.compile(r'^\s*(?://|#)\s*FILE:\s*(.+?)\s*$', re.MULTILINE) + + def _write_file(file_path, content): + """Validate path, write content, and record result. Returns True on success.""" + try: + dest = _safe_dest(output_path, file_path) + except ValueError as exc: + print(f"WARNING: Skipping unsafe path — {exc}", file=sys.stderr) + return False + dest.parent.mkdir(parents=True, exist_ok=True) + encoded = content.strip().encode("utf-8", errors="replace") + dest.write_bytes(encoded) + written.append((file_path, len(encoded))) + return True + + # Split into code blocks by ``` fences + parts = re.split(r'```', response_text) + + for part in parts: + # Check for lang:path at start (language tag contains the path) + lang_match = lang_path_pattern.match(part) + if lang_match: + _write_file(lang_match.group(2), part[lang_match.end():]) + continue + + # Check for // FILE: or # FILE: marker within the block + marker_match = file_marker_pattern.search(part) + if marker_match: + _write_file(marker_match.group(1).strip(), part[marker_match.end():]) + + return written + def call_grok(prompt, mode="reason", context="", system_override=None, tools=None, timeout=120): """Make the API call to Grok 4.20 Multi-Agent Beta.""" api_key = get_api_key() @@ -254,6 +327,10 @@ def main(): parser.add_argument("--tools", help="Path to JSON file with OpenAI-format tool definitions") parser.add_argument("--timeout", type=int, default=120, help="Timeout in seconds (default: 120)") parser.add_argument("--output", help="Output file path (default: stdout)") + parser.add_argument("--write-files", action="store_true", + help="Parse response for annotated code blocks and write to --output-dir") + parser.add_argument("--output-dir", default="./grok-output/", + help="Directory for file writes (default: ./grok-output/)") args = parser.parse_args() @@ -287,9 +364,24 @@ def main(): if args.output: Path(args.output).write_text(result) print(f"Written to: {args.output}", file=sys.stderr) - else: + + if args.write_files: + written = parse_and_write_files(result, args.output_dir) + if written: + total_bytes = sum(b for _, b in written) + print(f"Wrote {len(written)} files to {args.output_dir}") + for rel_path, byte_count in written: + print(f" {rel_path} ({byte_count:,} bytes)") + print(f"Total: {total_bytes:,} bytes") + else: + print( + "No annotated files found in model response to write to disk.\n" + "Re-run without --write-files to see the full response.", + file=sys.stderr, + ) + elif not args.output: print(result) if __name__ == "__main__": - main() + main() \ No newline at end of file diff --git a/src/bridge/index.js b/src/bridge/index.js old mode 100755 new mode 100644 index f3637aa..0d7dc80 --- a/src/bridge/index.js +++ b/src/bridge/index.js @@ -28,6 +28,8 @@ function parseArgs() { tools: null, timeout: 120, output: null, + writeFiles: false, + outputDir: './grok-output/', }; for (let i = 0; i < args.length; i++) { @@ -55,6 +57,12 @@ function parseArgs() { case '--output': parsed.output = args[++i]; break; + case '--write-files': + parsed.writeFiles = true; + break; + case '--output-dir': + parsed.outputDir = args[++i]; + break; case '--help': console.log(` grok_swarm — Bridge to xAI Grok 4.20 Multi-Agent Beta (4-agent swarm) @@ -70,6 +78,8 @@ Options: --tools JSON file with OpenAI-format tool definitions --timeout Timeout in seconds (default: 120) --output Output file (default: stdout) + --write-files Parse response for annotated code blocks and write files + --output-dir Directory for file writes (default: ./grok-output/) --help Show this help Modes: @@ -128,6 +138,14 @@ function run() { pyArgs.push('--output', opts.output); } + if (opts.writeFiles) { + pyArgs.push('--write-files'); + } + + if (opts.outputDir && opts.outputDir !== './grok-output/') { + pyArgs.push('--output-dir', opts.outputDir); + } + // Spawn Python process const child = spawn(PYTHON, pyArgs, { stdio: ['inherit', 'pipe', 'pipe'], @@ -181,4 +199,4 @@ function run() { }); } -run(); +run(); \ No newline at end of file diff --git a/src/plugin/index.ts b/src/plugin/index.ts index 83bacd6..40d77fa 100644 --- a/src/plugin/index.ts +++ b/src/plugin/index.ts @@ -51,6 +51,16 @@ const GrokSwarmSchema = Type.Object({ timeout: Type.Optional( Type.Number({ description: "Timeout in seconds (default: 120)" }), ), + write_files: Type.Optional( + Type.Boolean({ + description: "Write generated files directly to disk; orchestrator receives a brief summary only", + }), + ), + output_dir: Type.Optional( + Type.String({ + description: "Directory to write files into (default: ./grok-output/)", + }), + ), }); export default function (api: any) { @@ -62,7 +72,8 @@ export default function (api: any) { "Delegate tasks to xAI Grok 4.20 Multi-Agent Beta (4-agent swarm with 2M context). " + "Use for codebase analysis, refactoring, code generation, or complex reasoning. " + "Modes: refactor, analyze, code, reason, orchestrate. " + - "Orchestrate mode requires a custom system prompt.", + "With write_files=true, annotated code blocks are written directly to disk and a " + + "compact summary is returned instead of the full response.", parameters: GrokSwarmSchema, async execute(_toolCallId: string, params: any) { const json = (payload: unknown) => ({ @@ -108,6 +119,16 @@ export default function (api: any) { args.push("--system", params.system); } + if (params.write_files) { + args.push("--write-files"); + } + + if (params.output_dir) { + args.push("--output-dir", params.output_dir); + } else if (api.config?.defaultOutputDir) { + args.push("--output-dir", api.config.defaultOutputDir); + } + // Spawn Python bridge with timeout enforcement return new Promise((resolve) => { const child = spawn(pythonPath, args, { @@ -174,4 +195,4 @@ export default function (api: any) { }, { optional: true }, ); -} +} \ No newline at end of file diff --git a/src/plugin/openclaw.plugin.json b/src/plugin/openclaw.plugin.json index d1e4aa4..99f41db 100644 --- a/src/plugin/openclaw.plugin.json +++ b/src/plugin/openclaw.plugin.json @@ -18,7 +18,11 @@ "bridgeScript": { "type": "string", "description": "Path to grok_bridge.py script" + }, + "defaultOutputDir": { + "type": "string", + "description": "Default output directory for file writes (default: ./grok-output/)" } } } -} +} \ No newline at end of file