Open
Conversation
new `firecrawl agent` interactive mode that auto-detects locally installed ACP providers (Claude Code, Codex, OpenCode) and launches them to gather and structure web data into CSV, JSON, or markdown. - src/utils/acp.ts: provider detection + session persistence in ~/.firecrawl/sessions/ - src/commands/agent-interactive.ts: interactive flow with suggestions, format selection, schema confirmation - src/index.ts: wire interactive mode into agent command (no prompt = interactive, or -i flag) existing `firecrawl agent "prompt"` API mode is unchanged.
replace subprocess-spawn model with proper Agent Client Protocol integration using @agentclientprotocol/sdk. firecrawl now acts as an ACP client that can connect to any ACP-compatible agent (Claude Code, Codex, Gemini CLI, Copilot, etc.) via JSON-RPC over stdio. - src/acp/client.ts: ACP client using ClientSideConnection from SDK handles session updates, permission requests, file system ops - src/acp/registry.ts: detect installed ACP agents on PATH - src/commands/agent-interactive.ts: rewired to use ACP client instead of launchAgent() subprocess spawn - package.json: add @agentclientprotocol/sdk dependency no API keys needed — each agent handles its own auth.
claude-agent-acp and codex-acp are the actual ACP-compatible binaries (from @zed-industries packages), not the raw CLI tools.
after each agent turn, prompt the user for follow-up input so they can confirm schemas, refine results, or continue the conversation. type "done", "exit", or press enter to end the session.
parse rawInput from ACP tool calls to show real commands: 🔥 searching "AI startups" 🔥 scraping https://example.com/pricing 🔥 writing output.csv ✅ done instead of opaque "Terminal..." / "Read File..." labels.
proper casing, aligned columns, professional formatting:
🔥 Search "AI startups"
Done
🔥 Scrape https://example.com/pricing
Done
🔥 Write output.csv
Done
only show firecrawl operations (search, scrape, map, crawl) and session output writes. hide internal operations (reads, grep, python scripts, help lookups, temp files). animated spinner for in-progress work, checkmark on completion. ⠹ Searching "AI startups" ✓ Searching "AI startups" ⠼ Scraping forbes.com/lists/ai50 ✓ Scraping forbes.com/lists/ai50 ✓ Writing output.csv
each concurrent operation gets its own spinner line instead of collapsing into "(+N more)". lines clear and re-render as operations complete. ANSI escape sequences for smooth updates.
- replace broken ANSI multi-line spinner with simple start/done line pairs that don't fight with streaming text - fix search query parsing to strip pipes and redirects (was showing "query" 2>&1 | head -2000") - group unavailable agents at bottom of selection with separator
ink-based terminal UI that shows: - animated spinners per active firecrawl operation - checkmarks on completion - persistent status bar: session | agent | credits | time | format - unmounts between turns for clean user input prompts also: suppress agent stderr noise (hook warnings), enable JSX in tsconfig for .tsx support, add ink + react deps.
ink/yoga requires top-level await which breaks tsx/commonjs. replaced with zero-dependency ANSI status bar: - pinned to bottom of terminal via scroll region - animated spinner when operations are active - shows: session | agent | credits | elapsed | format - pauses for user input, resumes for next turn - no npm dependencies added
no scroll regions or ANSI cursor tricks. status prints inline at natural breakpoints (between turns, on completion). works in any terminal, pipeable, agent-friendly. · Scraping forbes.com/lists/ai50 ✓ Scraping forbes.com/lists/ai50 ── xy90n9zy · Claude Code · 12 credits · 34s · CSV
categorize every tool call into prominent (firecrawl ops, output writes) or background (processing, reading, spawning agents). prominent calls show start + done. background calls show once per label (deduplicated) in dim text so the user always knows the agent is working. · Searching "AI startups" ← prominent · Processing ← background (dim) · Spawning agents ← background (dim) · Scraping forbes.com/lists/ai50 ← prominent ✓ Scraping forbes.com/lists/ai50 · Reading sources ← background (dim) · Processing data ← background (dim) ✓ Writing output.csv
- deduplicate by URL: scraping the same URL 5x shows one start line and one done line, not ten - hide background operations entirely (agent text provides context) - visual spacing between agent text and tool call blocks - reset dedup between conversation turns - status line at each turn boundary with credits + elapsed
two modes for running the agent: - ACP: structured protocol with progress tracking and session mgmt - Pipe: inject directly into Claude Code/Codex (full terminal, interactive) user picks mode after selecting the agent. pipe mode falls back to the original launchAgent() subprocess approach. also rewrites system prompt to: - work in clear phases (plan → discover → extract → write) - stop after plan phase and wait for user confirmation - report progress with numbers (found 4 sources, extracted 50 records) - use bullet points not tables (renders better in terminals) - keep user informed at every step
after the conversation ends, show a menu with all session output files (with sizes) plus the session folder. selecting one opens it with the system default handler (open/xdg-open/start).
the main agent thread should only orchestrate — all heavy scraping and parsing is delegated to subagents that return just structured JSON records. keeps the main context window clean for planning and consolidation.
removes the doubled-up · start / ✓ done lines. now each operation prints once when it finishes — clean single line. the agent's own text provides context for what's in progress.
complete TUI rewrite: - auto-detected phase sections (Planning → Discovering → Extracting → Output) with ━━━ header lines based on tool call patterns - wire ACP usage_update events for token count + cost tracking - status line shows tokens used + cost + elapsed time - summary footer at session end - remove pipe mode (commit to ACP) - remove old buildCallbacks/describeToolCall (moved to tui.ts) - clean header: "🔥 Firecrawl Agent / Claude Code · CSV · Session xyz"
ACP agents handle their own billing — don't show dollar amounts. status line now shows: "25k tokens · 12 credits · 47s" also stop printing status before every → prompt — only at session end.
when the agent is doing non-firecrawl work (reading files, running scripts, spawning subagents), show a dim "..." on the current line. clears when agent text arrives or a firecrawl operation completes. prevents the dead cursor stare.
replace FIRECRAWL_TOOLS_BLOCK with explicit command list that excludes browser/interact tools. agent should use scrape with --wait-for for JS-rendered pages instead of browser sessions.
- display output file path (dimmed) above each → prompt - add "follow-up suggestions" instruction to system prompt: agent proposes 2-3 specific next steps after completing output, like perplexity's suggested questions
system prompt now tells the agent its session directory path and output file path. agent can write intermediate files, build data incrementally, and preserve progress if interrupted.
the module:"preserve" and moduleResolution:"bundler" changes were for ink which was removed. reverted to commonjs + node.
track whether last stdout text ended with \n. if not, write a newline before any tool call results, phase headers, or status lines to prevent them from overwriting partial agent text.
first run shows agent picker, saves selection. subsequent runs skip the prompt and use the saved default. --provider flag always overrides.
firecrawl agent "prompt" now uses ACP locally instead of the
firecrawl API. prints session path as JSON so other agents can
poll for results. use --api flag to opt into the old API path.
output: { sessionId, sessionDir, output, agent, status }
fork a detached child process for the ACP session so the CLI exits immediately. caller gets session path to poll for results.
logs agent text, tool calls, and completions to the session directory so callers can tail -f for live progress.
tell the agent to run firecrawl --help first, show practical command examples with useful flags (--only-main-content, --wait-for, --limit, -o), and save to temp files for parsing.
tool calls now show · label... when starting and ✓ label when done. background work still shows animated spinner. deduped so each operation only shows start/done once.
agent now saves raw scraped markdown alongside the output file, organized by hostname/path like firecrawl download does. session folder becomes a self-contained package of output + sources.
categorize tool calls by title in addition to rawInput.command. catches mcp__firecrawl__firecrawl_scrape, WebSearch, WebFetch, and all other tool invocation patterns the agent might use.
- always show agent selection (no auto-skip), default to last-used - detect firecrawl MCP tools and WebSearch/WebFetch in TUI display - stronger system prompt: must use firecrawl CLI via Bash only, no MCP tools, no WebSearch, no WebFetch, no curl
spinner provides feedback while running, ✓ line on completion. no more doubled · start / ✓ done lines.
shows all sessions sorted by date with prompt, agent, format, and age. select one to resume, open the output, or open the session folder.
fork() doesn't survive tsx exit. use spawn() with process.execPath and process.execArgv to preserve the tsx loader. pass args via temp file (system prompts are too large for argv).
no more dead cursor after typing a follow-up — the working spinner with rotating messages starts instantly on every turn, including the first one and after resume.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.