Skip to content

Commit 639a6b0

Browse files
committed
Bump plugin version and update CLI docs
Bump firecrawl plugin version to 1.0.5 and expand the firecrawl-cli documentation. Clarifies that `search --scrape` already retrieves full page content (so avoid redundant scraping), adds a research-task example, warns against re-scraping URLs or using `--html` to re-extract metadata, and emphasizes reading existing scraped files before fetching more data. Changes in .claude-plugin/plugin.json and skills/firecrawl-cli/SKILL.md.
1 parent 07493d6 commit 639a6b0

2 files changed

Lines changed: 21 additions & 1 deletion

File tree

.claude-plugin/plugin.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"name": "firecrawl",
33
"description": "Scrape, search, crawl, and map the web with a single command.",
4-
"version": "1.0.4",
4+
"version": "1.0.5",
55
"author": {
66
"name": "Firecrawl"
77
},

skills/firecrawl-cli/SKILL.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ Follow this escalation pattern when fetching web data:
3636
4. **Crawl** — You need bulk content from an entire site section (e.g., all docs pages).
3737
5. **Browser** — Scrape didn't return the needed data because it's behind interaction (pagination, modals, form submissions, multi-step navigation). Open a browser session to click through and extract it.
3838

39+
**Note:** `search --scrape` already fetches full page content for every result. Don't scrape those URLs again individually — only scrape URLs that weren't part of the search results.
40+
3941
**Example: fetching API docs from a large documentation site**
4042

4143
```
@@ -54,6 +56,20 @@ browser "click @e12" → click "Next Page"
5456
browser "scrape" -o .firecrawl/products-p2.md → extract page 2 content
5557
```
5658

59+
**Example: research task**
60+
61+
```
62+
search "firecrawl vs competitors 2024" --scrape -o .firecrawl/search-comparison-scraped.json
63+
→ full content already fetched for each result
64+
grep -n "pricing\|features" .firecrawl/search-comparison-scraped.json
65+
head -200 .firecrawl/search-comparison-scraped.json → read and process what you have
66+
→ notice a relevant URL mentioned in the content
67+
that wasn't in the search results
68+
scrape https://newsite.com/comparison -o .firecrawl/newsite-comparison.md
69+
→ only scrape this new URL
70+
→ synthesize all collected data into answer
71+
```
72+
5773
### Browser restrictions
5874

5975
Never use browser on sites with bot detection — it will be blocked. This includes Google, Bing, DuckDuckGo, and sites behind Cloudflare challenges or CAPTCHAs. Use `firecrawl search` for web searches instead.
@@ -209,6 +225,8 @@ firecrawl scrape https://example.com --include-tags article,main -o .firecrawl/a
209225
firecrawl scrape https://example.com --exclude-tags nav,aside,.ad -o .firecrawl/clean.md
210226
```
211227

228+
Don't re-scrape a URL with `--html` just to extract metadata (dates, authors, etc.) — that information is already present in the markdown output.
229+
212230
**Scrape Options:**
213231

214232
- `-f, --format <formats>` - Output format(s): markdown, html, rawHtml, links, screenshot, json
@@ -458,6 +476,8 @@ firecrawl browser close --session <id>
458476

459477
## Reading Scraped Files
460478

479+
Always read and process the files you already have before fetching more data. Don't re-scrape a URL you already have content for.
480+
461481
NEVER read entire firecrawl output files at once unless explicitly asked or required - they're often 1000+ lines. Instead, use grep, head, or incremental reads. Determine values dynamically based on file size and what you're looking for.
462482

463483
Examples:

0 commit comments

Comments
 (0)