diff --git a/py/ASYNC_DESIGN.md b/py/ASYNC_DESIGN.md new file mode 100644 index 0000000000000..f87d9e976b562 --- /dev/null +++ b/py/ASYNC_DESIGN.md @@ -0,0 +1,802 @@ +# Async Python Bindings — Architecture Design + +## Overview + +This document describes the design for adding native async/await support to the Selenium +Python bindings. The goal is a `selenium.webdriver.async_` namespace whose high-level +driver API (`await driver.get(url)`, `await driver.find_element(...)`, etc.) calls down +to WebDriver BiDi wherever the spec provides a command, falling back to HTTP only for +session lifecycle and operations not yet covered by BiDi. + +This approach serves two goals simultaneously: +1. Give users a clean async API with good DX — same surface as the sync driver +2. Migrate the implementation toward BiDi progressively, so users get BiDi semantics + (event-driven, no polling) without needing to know the protocol details + +Existing sync code is untouched. + +--- + +## Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ User code │ +│ async with Chrome() as driver: │ +│ await driver.get("https://example.com") │ +│ el = await driver.find_element(By.ID, "q") │ +└────────────────────┬────────────────────────────────────────┘ + │ +┌────────────────────▼────────────────────────────────────────┐ +│ High-level async driver (HAND-WRITTEN) │ +│ AsyncWebDriver / AsyncWebElement / AsyncWebDriverWait etc. │ +│ selenium.webdriver.async_ │ +└──────────┬────────────────────────────┬─────────────────────┘ + │ BiDi where possible │ HTTP for session + │ │ lifecycle + BiDi gaps +┌──────────▼──────────────┐ ┌─────────▼─────────────────────┐ +│ Async BiDi modules │ │ AsyncRemoteConnection (httpx) │ +│ (GENERATED) │ │ (HAND-WRITTEN) │ +│ create-async-bidi-src │ │ POST /session │ +│ selenium.webdriver │ │ DELETE /session/{id} │ +│ .async_.bidi │ │ HTTP fallback for gaps │ +└──────────┬──────────────┘ └────────────────────────────────┘ + │ +┌──────────▼──────────────┐ +│ AsyncWebSocketConnection│ +│ (HAND-WRITTEN) │ +│ websockets + anyio │ +└─────────────────────────┘ +``` + +--- + +## Decisions + +| Decision | Choice | Rationale | +|---|---|---| +| Namespace | `selenium.webdriver.async_` | `async` is a Python keyword; PEP 8 convention | +| Async framework | anyio | Supports asyncio and trio backends with one implementation | +| HTTP transport | httpx (`AsyncClient`) | anyio-compatible; needed for session init/quit and BiDi gaps | +| WebSocket | websockets library with anyio backend | Mature, native async | +| BiDi low-level layer | Generated via `create-async-bidi-src` | Same CDDL source as sync BiDi; async command methods | +| High-level driver | Hand-written, calls BiDi low-level | Bespoke per-method BiDi mapping; cannot be mechanically generated | +| Min Python version | 3.10 | Reliable anyio support; `match`, `TypeAlias` available | +| I/O-bound properties | Become `async def` methods of same name | Python has no `async` property | +| BiDi callbacks | Native `async def`, dispatched via anyio task group | Thread-bridging is fragile | +| Dependencies | Optional extra: `pip install selenium[async]` | Does not affect sync-only users | +| Naming | `AsyncWebDriver`, `AsyncChrome`, etc. | Explicit; visible at the call site | + +--- + +## Target User API + +```python +import pytest +from selenium.webdriver.async_ import Chrome +from selenium.webdriver.async_.support.wait import AsyncWebDriverWait +from selenium.webdriver.async_.support import expected_conditions as EC +from selenium.webdriver.common.by import By + +@pytest.mark.anyio +async def test_search(): + async with Chrome() as driver: + await driver.get("https://example.com") + el = await driver.find_element(By.ID, "q") + await el.send_keys("selenium") + + wait = AsyncWebDriverWait(driver, 10) + await wait.until(EC.title_contains("Search")) + + assert "Search" in await driver.title() +``` + +The sync API is unchanged: + +```python +from selenium.webdriver import Chrome # unaffected + +def test_search(): + with Chrome() as driver: + driver.get("https://example.com") + assert "Example" in driver.title +``` + +--- + +## File Structure + +``` +py/ +├── ASYNC_DESIGN.md # this document +└── selenium/ + └── webdriver/ + └── async_/ + ├── __init__.py # exports AsyncChrome, AsyncFirefox, etc. + ├── bidi/ # GENERATED by create-async-bidi-src + │ ├── __init__.py + │ ├── common.py # async command_builder + │ ├── browsing_context.py # AsyncBrowsingContext + │ ├── script.py # AsyncScript + │ ├── network.py # AsyncNetwork + │ ├── input.py # AsyncInput + │ ├── browser.py # AsyncBrowser + │ ├── session.py # AsyncSession + │ ├── storage.py # AsyncStorage + │ └── ... (all current bidi modules) + ├── remote/ + │ ├── __init__.py + │ ├── remote_connection.py # HAND-WRITTEN: AsyncRemoteConnection (httpx) + │ ├── websocket_connection.py # HAND-WRITTEN: AsyncWebSocketConnection + │ ├── webdriver.py # HAND-WRITTEN: AsyncWebDriver + │ ├── webelement.py # HAND-WRITTEN: AsyncWebElement + │ ├── shadowroot.py # HAND-WRITTEN: AsyncShadowRoot + │ ├── switch_to.py # HAND-WRITTEN: AsyncSwitchTo + │ ├── alert.py # HAND-WRITTEN: AsyncAlert + │ └── mobile.py # HAND-WRITTEN: AsyncMobile + ├── chrome/ + │ ├── __init__.py + │ └── webdriver.py # HAND-WRITTEN: AsyncChrome + ├── firefox/ + │ ├── __init__.py + │ └── webdriver.py # HAND-WRITTEN: AsyncFirefox + ├── edge/ + │ ├── __init__.py + │ └── webdriver.py # HAND-WRITTEN: AsyncEdge + ├── safari/ + │ ├── __init__.py + │ └── webdriver.py # HAND-WRITTEN: AsyncSafari + ├── common/ + │ ├── __init__.py + │ └── action_chains.py # HAND-WRITTEN: AsyncActionChains (BiDi input) + └── support/ + ├── __init__.py + ├── wait.py # HAND-WRITTEN: AsyncWebDriverWait + ├── expected_conditions.py # HAND-WRITTEN: async EC callables + └── select.py # HAND-WRITTEN: AsyncSelect +``` + +--- + +## Layer 1 — Generated Async BiDi (`create-async-bidi-src`) + +### What changes from `create-bidi-src` + +The existing `create-bidi-src` target generates sync BiDi modules in +`selenium/webdriver/common/bidi/`. The new `create-async-bidi-src` target runs the +same `generate_bidi.py` generator with an `--async` flag, outputting to +`selenium/webdriver/async_/bidi/`. The generated code differs in three ways: + +| Sync generated | Async generated | +|---|---| +| `import threading` | `import anyio` | +| `threading.Lock()` | `anyio.Lock()` | +| `def navigate(self, ...)` | `async def navigate(self, ...)` | +| `result = self._conn.execute(cmd)` | `result = await self._conn.execute(cmd)` | +| `Session(self.conn).subscribe(...)` | `await AsyncSession(self.conn).subscribe(...)` | +| `callback: Callable` | `callback: Callable[..., Coroutine]` | + +The dataclasses (`NavigateParameters`, `LocateNodesResult`, `CssLocator`, etc.) are +identical — they are protocol-agnostic data structures and require no changes. + +The `command_builder` generator function is also identical — it `yield`s a dict and +returns a result, regardless of whether the executor is sync or async. + +### `generate_bidi.py` changes + +Add an `--async` flag. When set: +- Command method bodies emit `await self._conn.execute(cmd)` instead of + `self._conn.execute(cmd)` +- Method signatures gain `async def` +- `_EventManager.__init__` uses `anyio.Lock()` instead of `threading.Lock()` +- `_EventManager.subscribe_to_event` and `unsubscribe_from_event` become `async def` +- `add_event_handler` becomes `async def` +- The module-level import block swaps `threading` for `anyio` +- Output path is `selenium/webdriver/async_/bidi/` instead of + `selenium/webdriver/common/bidi/` + +### Bazel wiring + +```python +# py/BUILD.bazel + +generate_bidi( + name = "create-async-bidi-src", + cddl_file = "@webdriver_bidi_all_cddl//file:spec.cddl", + enhancements_manifest = "//py/private:bidi_enhancements_manifest.py", + extra_cddl_files = [ + "@permissions_all_cddl//file:spec.cddl", + "@prefetch_all_cddl//file:spec.cddl", + "@ua_client_hints_all_cddl//file:spec.cddl", + "@web_bluetooth_all_cddl//file:spec.cddl", + ], + extra_srcs = [ + "//py/private:_event_manager.py", # async version of event manager utilities + "//py/private:cdp.py", + ], + generator = ":generate_bidi", + merge_tool = "//py/private:merge_cddl", + module_name = "selenium/webdriver/async_/bidi", + spec_version = "1.0", + async_mode = True, # new attribute, passes --async to the generator +) + +py_library( + name = "async_bidi", + srcs = [":create-async-bidi-src"], + deps = [ + requirement("anyio"), + ], +) +``` + +The `generate_bidi` Bazel rule gains an `async_mode` boolean attribute that, when true, +passes `--async` to the generator script. + +--- + +## Layer 2 — Hand-Written Async Driver + +### Why hand-written (not generated from sync HTTP code) + +The sync driver's `get()`, `find_element()`, `execute_script()` etc. all call +`self.execute(Command.X, params)` — a single HTTP dispatch. The async equivalents call +*different things*: `browsing_context.navigate()`, `browsing_context.locate_nodes()`, +`script.evaluate()`. These are bespoke mappings, not mechanical transformations. There is +no AST transformation that can produce them automatically. + +### `AsyncWebDriver` — method to BiDi mapping + +The table below shows the primary mapping. Methods marked **HTTP fallback** use +`AsyncRemoteConnection` because no BiDi command exists yet in the spec. + +| Public API method | BiDi command | +|---|---| +| `await driver.get(url)` | `browsing_context.navigate(context, url, wait=COMPLETE)` | +| `await driver.find_element(by, value)` | `browsing_context.locate_nodes(context, locator, max=1)` | +| `await driver.find_elements(by, value)` | `browsing_context.locate_nodes(context, locator)` | +| `await driver.execute_script(script, *args)` | `script.evaluate(expression, target, await_promise=False)` | +| `await driver.execute_async_script(script, *args)` | `script.evaluate(expression, target, await_promise=True)` | +| `await driver.title()` | `script.evaluate("document.title", target)` | +| `await driver.current_url()` | `browsing_context.get_tree(root=context)` → `url` | +| `await driver.page_source()` | `script.evaluate("document.documentElement.outerHTML", target)` | +| `await driver.back()` | `browsing_context.traverse_history(context, delta=-1)` | +| `await driver.forward()` | `browsing_context.traverse_history(context, delta=1)` | +| `await driver.refresh()` | `browsing_context.reload(context)` | +| `await driver.close()` | `browsing_context.close(context)` | +| `await driver.current_window_handle()` | tracked locally on the driver object | +| `await driver.window_handles()` | `browsing_context.get_tree()` → all context IDs | +| `await driver.switch_to.window(handle)` | updates tracked context; subscribes to BiDi events for new context | +| `await driver.screenshot_as_base64()` | `browsing_context.capture_screenshot(context)` | +| `await driver.print_page(opts)` | `browsing_context.print(context, ...)` | +| `await driver.get_cookies()` | `storage.get_cookies(partition)` | +| `await driver.add_cookie(cookie)` | `storage.set_cookie(cookie, partition)` | +| `await driver.delete_cookie(name)` | `storage.delete_cookies(filter, partition)` | +| `await driver.delete_all_cookies()` | `storage.delete_cookies(partition)` | +| `await driver.new_window(type)` | `browsing_context.create(type)` | +| `await driver.maximize_window()` | **HTTP fallback** (no BiDi equivalent yet) | +| `await driver.set_window_rect(...)` | **HTTP fallback** | +| `await driver.get_window_rect()` | **HTTP fallback** | +| `await driver.set_timeouts(...)` | **HTTP fallback** | +| `await driver.get_log(type)` | **HTTP fallback** | +| Virtual authenticator methods | **HTTP fallback** | +| FedCM methods | **HTTP fallback** | + +### `AsyncWebElement` — method to BiDi mapping + +`AsyncWebElement` holds a BiDi **shared reference** (`sharedId`) returned by +`locate_nodes`. Actions use `script.call_function` (for JS-level ops) or +`input.perform_actions` (for pointer/keyboard). + +| Public API method | BiDi command | +|---|---| +| `await element.click()` | `input.perform_actions` (pointer: move to center, down, up) | +| `await element.send_keys(*value)` | `input.perform_actions` (key sequence) | +| `await element.clear()` | `script.call_function` (clear value via JS) | +| `await element.submit()` | `script.call_function` (form.submit()) | +| `await element.text()` | `script.call_function` → `.textContent` | +| `await element.tag_name()` | `script.call_function` → `.tagName` | +| `await element.get_attribute(name)` | `script.call_function` → `getAttribute` atom | +| `await element.get_property(name)` | `script.call_function` → property access | +| `await element.get_dom_attribute(name)` | `script.call_function` → `getAttribute` | +| `await element.is_displayed()` | `script.call_function` → `isDisplayed` atom | +| `await element.is_enabled()` | `script.call_function` → `.disabled` check | +| `await element.is_selected()` | `script.call_function` → `.checked` / `selected` | +| `await element.rect()` | `script.call_function` → `getBoundingClientRect()` | +| `await element.location()` | derived from `rect()` | +| `await element.size()` | derived from `rect()` | +| `await element.find_element(by, value)` | `browsing_context.locate_nodes(start_nodes=[self])` | +| `await element.find_elements(by, value)` | `browsing_context.locate_nodes(start_nodes=[self])` | +| `await element.screenshot_as_base64()` | `browsing_context.capture_screenshot(clip=element)` | +| `await element.value_of_css_property(prop)` | **HTTP fallback** | +| `await element.shadow_root()` | `script.call_function` → `shadowRoot` | + +### Session lifecycle (HTTP) + +Session creation (`POST /session`) and teardown (`DELETE /session/{id}`) remain HTTP. +This is unavoidable: the BiDi WebSocket URL is returned *in the session response*, so +HTTP must come first. `AsyncRemoteConnection` (httpx) handles these two operations. +All subsequent WebDriver commands go through `AsyncWebSocketConnection`. + +```python +class AsyncWebDriver: + async def __aenter__(self) -> Self: + self._task_group_ctx = anyio.create_task_group() + self._task_group = await self._task_group_ctx.__aenter__() + + # HTTP: create session, get WebSocket URL + await self.command_executor.open() + response = await self.command_executor.execute(Command.NEW_SESSION, caps) + self.session_id = response["sessionId"] + self.caps = response["capabilities"] + + # BiDi: open WebSocket, start receive loop + ws_url = self.caps["webSocketUrl"] + self._ws = AsyncWebSocketConnection(ws_url, ...) + await self._ws.connect(self._task_group) + + # Instantiate async BiDi module objects + self._browsing_context = AsyncBrowsingContext(self._ws) + self._script = AsyncScript(self._ws) + self._storage = AsyncStorage(self._ws) + self._input = AsyncInput(self._ws) + + return self + + async def __aexit__(self, *exc_info): + await self.command_executor.execute(Command.QUIT, {}) # HTTP DELETE /session + await self._ws.close() + await self.command_executor.close() + await self._task_group_ctx.__aexit__(*exc_info) +``` + +### Example method implementations + +```python +# driver.get() +async def get(self, url: str) -> None: + from selenium.webdriver.async_.bidi.browsing_context import ReadinessState + await self._browsing_context.navigate( + context=self._current_context, + url=url, + wait=ReadinessState.COMPLETE, + ) + +# driver.find_element() +async def find_element(self, by: str, value: str) -> AsyncWebElement: + from selenium.webdriver.async_.bidi.browsing_context import CssLocator, XPathLocator + locator = _build_locator(by, value) # maps By.* to BiDi locator dataclass + result = await self._browsing_context.locate_nodes( + context=self._current_context, + locator=locator, + max_node_count=1, + ) + if not result.nodes: + raise NoSuchElementException(f"Unable to locate element: {by}={value}") + return AsyncWebElement(self, result.nodes[0].shared_id) + +# driver.execute_script() +async def execute_script(self, script: str, *args) -> Any: + from selenium.webdriver.async_.bidi.script import ContextTarget + result = await self._script.evaluate( + expression=_wrap_script(script, args), + target=ContextTarget(context=self._current_context), + await_promise=False, + ) + return _unwrap_bidi_result(result) + +# element.click() +async def click(self) -> None: + center = await self._get_center_point() # getBoundingClientRect via script + await self._driver._input.perform_actions( + actions=[_pointer_click_sequence(center)], + context=self._driver._current_context, + ) +``` + +--- + +## Hand-Written Pieces + +### `AsyncRemoteConnection` (httpx) + +Used only for session creation and teardown, plus HTTP fallback for commands not yet +in BiDi. A single `httpx.AsyncClient` lives for the session duration. + +```python +class AsyncRemoteConnection: + async def open(self): + self._client = httpx.AsyncClient( + verify=self._client_config.ca_certs, + timeout=self._client_config.timeout, + ) + + async def close(self): + await self._client.aclose() + + async def execute(self, command, params): + method, path = self._commands[command] + url = self._url + _substitute_params(path, params) + response = await self._client.request(method, url, json=params) + return self._process_response(response) +``` + +### `AsyncWebSocketConnection` (websockets + anyio) + +The receive loop runs as an anyio task in the driver's task group. Async callbacks are +dispatched as new tasks in that group, so they run concurrently with user code. + +```python +class AsyncWebSocketConnection: + async def connect(self, task_group): + self._ws = await websockets.connect(self.url) + self._task_group = task_group + task_group.start_soon(self._receive_loop) + + async def execute(self, command): + async with self._send_lock: + self._id += 1 + current_id = self._id + payload = self._serialize_command(command) + payload["id"] = current_id + event = anyio.Event() + self._pending[current_id] = event + await self._ws.send(json.dumps(payload, cls=_BiDiEncoder)) + with anyio.fail_after(self._timeout): + await event.wait() + return self._results.pop(current_id) + + async def _receive_loop(self): + async for raw in self._ws: + message = json.loads(raw) + if "id" in message: + self._results[message["id"]] = message + if event := self._pending.pop(message["id"], None): + event.set() + if "method" in message: + for cb in self.callbacks.get(message["method"], []): + self._task_group.start_soon(cb, message["params"]) +``` + +--- + +## I/O-Bound Properties That Become Methods + +Because Python has no `async` property, these are the only places the async API diverges +structurally from sync. Same name, but called as methods. + +### `AsyncWebDriver` + +| Sync (property) | Async (method) | Implemented via | +|---|---|---| +| `driver.title` | `await driver.title()` | `script.evaluate("document.title")` | +| `driver.current_url` | `await driver.current_url()` | `browsing_context.get_tree()` | +| `driver.page_source` | `await driver.page_source()` | `script.evaluate("document.documentElement.outerHTML")` | +| `driver.current_window_handle` | `await driver.current_window_handle()` | tracked locally | +| `driver.window_handles` | `await driver.window_handles()` | `browsing_context.get_tree()` | +| `driver.timeouts` | `await driver.timeouts()` | HTTP fallback | + +### `AsyncWebElement` + +| Sync (property) | Async (method) | Implemented via | +|---|---|---| +| `element.tag_name` | `await element.tag_name()` | `script.call_function` | +| `element.text` | `await element.text()` | `script.call_function` | +| `element.rect` | `await element.rect()` | `script.call_function` | +| `element.location` | `await element.location()` | derived from `rect()` | +| `element.size` | `await element.size()` | derived from `rect()` | +| `element.screenshot_as_base64` | `await element.screenshot_as_base64()` | `browsing_context.capture_screenshot` | + +### Properties that stay as properties (no network call) + +`driver.session_id`, `driver.name`, `driver.capabilities`, +`element.id`, `element.parent` + +--- + +## `AsyncWebDriverWait` + +```python +class AsyncWebDriverWait(Generic[D]): + async def until(self, method, message=""): + end_time = anyio.current_time() + self._timeout + while True: + try: + value = await method(self._driver) + if value: + return value + except self._ignored_exceptions: + pass + if anyio.current_time() > end_time: + break + await anyio.sleep(self._poll) + raise TimeoutException(message) +``` + +`expected_conditions` callables become `async def __call__(self, driver)`. +User-supplied condition functions must be `async def`. + +--- + +## Bazel Wiring + +### `generate_bidi.py` change + +Add `--async` CLI flag. When set: +- Output directory becomes `selenium/webdriver/async_/bidi/` +- Command method signatures gain `async def` +- `self._conn.execute(cmd)` → `await self._conn.execute(cmd)` +- `threading.Lock()` → `anyio.Lock()` +- Event subscription methods become `async def` +- Imports swap `threading` for `anyio` + +### New Bazel targets in `py/BUILD.bazel` + +```python +# Async BiDi low-level modules (generated) +generate_bidi( + name = "create-async-bidi-src", + cddl_file = "@webdriver_bidi_all_cddl//file:spec.cddl", + enhancements_manifest = "//py/private:bidi_enhancements_manifest.py", + extra_cddl_files = [ + "@permissions_all_cddl//file:spec.cddl", + "@prefetch_all_cddl//file:spec.cddl", + "@ua_client_hints_all_cddl//file:spec.cddl", + "@web_bluetooth_all_cddl//file:spec.cddl", + ], + generator = ":generate_bidi", + merge_tool = "//py/private:merge_cddl", + module_name = "selenium/webdriver/async_/bidi", + spec_version = "1.0", + async_mode = True, # new attribute → passes --async to generate_bidi.py +) + +# Async driver library (hand-written sources + generated BiDi dep) +py_library( + name = "async", + srcs = glob(["selenium/webdriver/async_/**/*.py"], + exclude=["selenium/webdriver/async_/bidi/**"]), + deps = [ + ":async_bidi", + ":common", + requirement("anyio"), + requirement("httpx"), + requirement("websockets"), + ], +) + +py_library( + name = "async_bidi", + srcs = [":create-async-bidi-src"], + deps = [requirement("anyio")], +) +``` + +--- + +## Test Structure + +Tests live in `py/test/selenium/webdriver/async_/`, mirroring the sync structure. +Every public API method has a test. Tests are adapted from the sync equivalents. + +``` +py/test/selenium/webdriver/async_/ + __init__.py + conftest.py # async fixtures: driver, pages + common/ + __init__.py + navigation_tests.py + element_finding_tests.py + children_finding_tests.py + element_property_tests.py + typing_tests.py + click_tests.py + visibility_tests.py + window_switching_tests.py + takes_screenshots_tests.py + timeout_tests.py + quit_tests.py + executing_javascript_tests.py + executing_async_javascript_tests.py + rendered_webelement_tests.py + form_handling_tests.py + select_element_handling_tests.py + ... (one file per sync test file in scope) + support/ + __init__.py + webdriverwait_tests.py + expected_conditions_tests.py + chrome/ + __init__.py + chrome_tests.py + firefox/ + __init__.py + firefox_tests.py +``` + +### `conftest.py` + +No global singleton driver — each test gets a clean `async with` scope. The `pages` +fixture exposes `async def load()` since `driver.get()` is now awaitable. + +```python +import pytest +from selenium.webdriver.async_ import Chrome, Firefox, Edge + +@pytest.fixture +def anyio_backend(): + return "asyncio" + +@pytest.fixture +async def driver(request): + driver_name = getattr(request, "param", "chrome").lower() + cls = {"chrome": Chrome, "firefox": Firefox, "edge": Edge}[driver_name] + async with cls(options=_build_options(driver_name, request)) as d: + yield d + +@pytest.fixture +def pages(driver, webserver): + class Pages: + def url(self, name): + return webserver.where_is(name) + + async def load(self, name): + await driver.get(self.url(name)) + + return Pages() +``` + +### Test adaptation rules + +Given sync test: +```python +def test_should_return_page_title(driver, pages): + pages.load("simpleTest.html") + assert driver.title == "Hello WebDriver World" +``` + +Async equivalent: +```python +import pytest + +@pytest.mark.anyio +async def test_should_return_page_title(driver, pages): + await pages.load("simpleTest.html") + assert await driver.title() == "Hello WebDriver World" +``` + +Mechanical rules: +1. Add `@pytest.mark.anyio` before every test function +2. `def test_` → `async def test_` +3. `pages.load(x)` → `await pages.load(x)` +4. `driver.title` → `await driver.title()` (and all I/O properties; see tables above) +5. All driver/element network calls gain `await` +6. `with driver:` → `async with driver:` + +### Bazel test targets + +```python +ASYNC_TEST_DEPS = TEST_DEPS + [ + requirement("anyio"), + requirement("pytest-anyio"), + requirement("httpx"), + requirement("websockets"), +] + +[ + py_test_suite( + name = "test-%s-async" % browser, + size = "large", + srcs = glob(["test/selenium/webdriver/async_/**/*.py"]), + args = ["--instafail", "--anyio-backends=asyncio"] + BROWSERS[browser]["args"], + data = BROWSERS[browser]["data"], + env_inherit = ["DISPLAY"], + tags = ["no-sandbox"] + BROWSERS[browser]["tags"], + target_compatible_with = BROWSERS[browser]["target_compatible_with"], + test_suffix = "%s-async" % browser, + deps = [":init-tree", ":async", ":webserver"] + ASYNC_TEST_DEPS, + ) + for browser in ["chrome", "firefox", "edge"] +] + +# Optional trio backend validation +[ + py_test_suite( + name = "test-%s-async-trio" % browser, + size = "large", + srcs = glob(["test/selenium/webdriver/async_/**/*.py"]), + args = ["--instafail", "--anyio-backends=trio"] + BROWSERS[browser]["args"], + data = BROWSERS[browser]["data"], + env_inherit = ["DISPLAY"], + tags = ["no-sandbox"] + BROWSERS[browser]["tags"], + target_compatible_with = BROWSERS[browser]["target_compatible_with"], + test_suffix = "%s-async-trio" % browser, + deps = [":init-tree", ":async", ":webserver", requirement("trio")] + ASYNC_TEST_DEPS, + ) + for browser in ["chrome", "firefox", "edge"] +] +``` + +--- + +## Delivery Phases + +Code and tests are delivered together — each phase ends with working Bazel targets. + +### Phase 1 — Foundations + +- Add `--async` flag to `generate_bidi.py`; validate output against sync version +- `create-async-bidi-src` Bazel target producing `selenium/webdriver/async_/bidi/` +- `AsyncWebSocketConnection` (hand-written) +- `AsyncRemoteConnection` (hand-written, httpx) +- Package scaffolding and optional dependency declaration +- `async_bidi` py_library Bazel target +- **Tests:** none yet + +### Phase 2 — Core driver + +- `AsyncWebDriver` (hand-written, BiDi-backed): `get`, `find_element`, `find_elements`, + `execute_script`, `execute_async_script`, `close`, `quit`, `title`, `current_url`, + `page_source`, `current_window_handle`, `window_handles`, `back`, `forward`, `refresh` +- `AsyncWebElement` (hand-written): `click`, `send_keys`, `clear`, `text`, `tag_name`, + `get_attribute`, `is_displayed`, `is_enabled`, `is_selected`, `rect` +- `async` py_library Bazel target +- **Tests:** `navigation_tests`, `element_finding_tests`, `element_property_tests`, + `quit_tests`, `click_tests`, `typing_tests`, `visibility_tests` + +### Phase 3 — Browser-specific drivers + cookies + windows + +- `AsyncChrome`, `AsyncFirefox`, `AsyncEdge` (hand-written thin subclasses) +- Cookie management: `get_cookies`, `add_cookie`, `delete_cookie`, `delete_all_cookies` +- Window management: `new_window`, `switch_to.window`, `window_handles` +- `AsyncSwitchTo`, `AsyncAlert` +- **Tests:** `window_switching_tests`, `takes_screenshots_tests`, browser-specific smoke + +### Phase 4 — Support utilities + screenshots + +- `AsyncWebDriverWait` + `AsyncExpectedConditions` +- `AsyncSelect` +- `AsyncActionChains` (wraps `input.perform_actions`) +- Screenshot methods (driver and element) +- **Tests:** `webdriverwait_tests`, `expected_conditions_tests`, + `select_element_handling_tests`, `takes_screenshots_tests` + +### Phase 5 — HTTP fallback for BiDi gaps + +- HTTP fallback path in `AsyncWebDriver` for: window rect/maximize/minimize, + timeouts, log retrieval, virtual authenticators, FedCM +- Document which methods are HTTP fallback and which BiDi commands they will migrate + to as the spec matures +- **Tests:** `timeout_tests`, `rendered_webelement_tests` + +### Phase 6 — Hardening and documentation + +- Migration guide: "sync → async in 3 steps" (change import, add `async with`, + add `await`, rename I/O properties to method calls) +- Type stub (`.pyi`) generation for the async namespace +- pytest-anyio fixture examples in docs +- Full parity sweep — every sync test file in scope has an async counterpart + +--- + +## Explicitly Out of Scope for V1 + +- **`EventFiringWebDriver`** — decorator pattern, defer to V2 +- **`RelativeLocator`** — `locate_nodes` with context locator can support this but adds + complexity; defer to V2 +- **The legacy `bidi_connection()` CDP method** — left as-is; CDP is separate from BiDi +- **Safari** — Safari does not support WebDriver BiDi; async namespace targets + Chrome, Firefox, Edge. Safari can be added once it has BiDi support. + +--- + +## Risks and Mitigations + +| Risk | Mitigation | +|---|---| +| BiDi `locate_nodes` behaviour differs subtly from HTTP `find_element` (timing, shadow DOM) | Comprehensive test suite run against the same HTML fixtures as sync tests catches divergence | +| `input.perform_actions` for `element.click()` requires coordinate calculation | Use `getBoundingClientRect` via `script.evaluate` to find center point; document known edge cases (off-screen elements) | +| `generate_bidi.py` `--async` flag makes the generator more complex | Keep async/sync output paths in the generator as parallel branches; test both outputs in CI via both Bazel targets | +| HTTP fallback methods silently bypass BiDi | Mark HTTP fallback methods with a `_HTTP_FALLBACK = True` class attribute; emit a deprecation-style log at `DEBUG` level so developers know which methods are not yet BiDi | +| Safari excluded limits V1 reach | Document clearly; Safari BiDi adoption is a browser vendor dependency, not a Selenium one | +| anyio task group lifetime — user forgets `async with` | Raise `RuntimeError("AsyncWebDriver must be used as an async context manager")` on first BiDi call if task group not started |