Skip to content

Feature: Browser automation via MCP (Playwright) #38

@Rustam-Z

Description

@Rustam-Z

Goal

Enable the bot to drive a real browser — fill forms, click buttons, extract data from JS-heavy pages, take screenshots, automate web flows.

Why

Currently the bot can only do read-only HTTP fetches via WebFetch / WebSearch. That misses:

  • JS-rendered content (SPAs, dashboards, infinite scroll)
  • Authenticated flows (logging in, navigating gated content)
  • Form submissions (filling and submitting on user's behalf)
  • Multi-step UI flows (booking, checkout, account management)
  • Visual extraction (screenshots, layout inspection)

OpenClaw has this natively. Adding it to pyclaudir closes a major capability gap for business automation use cases.

Approach

Use a Playwright-based MCP server. Two options:

A) Existing MCP — there are community Playwright MCP servers (e.g. @modelcontextprotocol/server-playwright). Pull in, configure in plugins.json, exposed as tools. Lowest effort, fastest path.

B) Custom MCP wrapper — write a thin pyclaudir-specific MCP server that wraps Playwright. More control over what gets exposed, easier to lock down dangerous actions.

Recommend A for v1, B if security or scope becomes a concern.

What the bot would gain

New tools:

  • browser_navigate(url)
  • browser_click(selector)
  • browser_fill(selector, value)
  • browser_screenshot() — returns image
  • browser_extract(selector) — returns text
  • browser_wait_for(selector, timeout)

Risks / safeguards

  • Token cost: browser automation runs longer than HTTP — each session can burn many tool calls. Keep in plugins.json opt-in like the bash tool group.
  • Privacy: sites should not run with the operator's logged-in browser profile by default — use isolated browser contexts.
  • Abuse: non-allowlisted users could try to weaponize for scraping; access control already handles this.

Acceptance

  • Browser MCP enabled in plugins.json
  • Bot can navigate, click, fill, extract, screenshot
  • Headless mode by default
  • Off by default (in line with bash/code tool groups)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions