Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sdks/python.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ print(parsed.markdown)

### Crawl a Website

To crawl a website, use the `crawl` method. It takes the starting URL and optional options as arguments. The options allow you to specify additional settings for the crawl job, such as the maximum number of pages to crawl, allowed domains, and the output format. See [Pagination](#pagination) for auto/manual pagination and limiting.
Crawl an entire website with the `crawl` method. Pass a starting URL plus options to control the maximum number of pages, allowed domains, and output formats. The method blocks until the crawl finishes and returns the aggregated results. See [Pagination](#pagination) for auto and manual paging.

<CrawlPythonShort />

Expand Down