Skip to content

docs: add Obscura as a CDP backend example#2005

Open
SGavrl wants to merge 1 commit into
unclecode:developfrom
SGavrl:feature/obscura-cdp-backend
Open

docs: add Obscura as a CDP backend example#2005
SGavrl wants to merge 1 commit into
unclecode:developfrom
SGavrl:feature/obscura-cdp-backend

Conversation

@SGavrl

@SGavrl SGavrl commented Jun 8, 2026

Copy link
Copy Markdown

Summary

Adds a runnable example that connects Crawl4AI to Obscura, an open-source Rust
headless browser, over CDP. Obscura speaks the DevTools Protocol, so Crawl4AI
drives it through the existing cdp_url path with no library changes (same
pattern as the Scrapeless example). No issue to link, this is a new example only.

List of files changed and why

  • docs/examples/obscura_cdp_backend.py - new example. Start obscura serve,
    connect with BrowserConfig(browser_mode="cdp", cdp_url=...), and crawl a
    static page plus a JS-rendered page to Markdown.

How Has This Been Tested?

Built Obscura from source, ran obscura serve --port 9222, and ran Crawl4AI
against it through connect_over_cdp. Verified:

  • static page to Markdown (example.com)
  • JS rendering: quotes.toscrape.com/js builds its content client-side, the quotes
    were extracted after Obscura ran the page JS (they are not in the raw HTML)
  • wait_for="css:div.quote"
  • js_code execution before extraction
  • JsonCssExtractionStrategy (10 quotes with authors)
  • link extraction (49 links)

Also ran the project's unit tests: pytest tests/unit/ -> 48 passed. The
browser and integration suites need a chromium binary plus network/API access
not available in my environment; this PR adds an example file and touches no
library code, so it does not affect them.

Screenshots and PDFs are not supported by Obscura (no paint engine) and are out
of scope.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added/updated unit tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant