Background: An integration test failed today because the driver installed on the system didn't match the version selenium manager expected. Integration tests shouldn't depend on system state, so I removed the system driver. But that exposed the underlying problem: the driver-resolution / version-matching logic those tests were implicitly exercising has no unit coverage of its own. Once the test stops leaning on a happens-to-be-present system driver, nothing is left verifying that logic.
Much of rust/ driver-resolution and cache logic is exercised only by the networked integration suite, which asserts "a path exists" rather than which source produced it or what the logic decided. Pure, deterministic logic has almost no unit coverage, so regressions and edge-case bugs are invisible.
Why it matters — what a quick audit of just the resolution/cache path turned up:
- Confirmed: an empty discovered driver version gets cached by Edge (
rust/src/edge.rs:255) and Firefox (rust/src/firefox.rs:287) but not Chrome (rust/src/chrome.rs:342, which has && !driver_version.is_empty()). On a version-less HTTP 200 (a misconfigured driver-mirror-url or an intercepting proxy), the empty string is written to metadata and poisons it for the TTL window (default 1h); every subsequent run reads back Some("") and fails "version cannot be discovered", while Chrome self-heals. Underlying smell: request_driver_version returns Ok("") instead of erroring on a version-less response.
- Latent:
find_best_driver_from_cache sorts cached version directories lexically rather than by semver, and matches with starts_with(major_browser_version). Neither bites current version numbers, but both are silent wrong-driver risks (lexical sort misorders across digit-count boundaries like 0.9→0.10; starts_with collides for short majors, and starts_with("") matches every cached driver when the major version is empty).
Coverage to add (pure, fast, deterministic, currently zero unit tests):
parse_version — per-browser --version string parsing (Chrome/Edge/Firefox/ESR/snap variants, the error short-circuit, empty-on-garbage).- the cache-selection fallback (
find_best_driver_from_cache / find_latest_from_cache / is_driver_and_matches_browser_version).
- version predicates and helpers (
get_major_version, get_index_version, is_stable/is_beta/is_dev/is_nightly/is_esr, is_version_specific).
- config parsing (
str_to_os, OS::is, ARCH::is, get_env_name).
A lightweight in-process mock-HTTP server + temp-cache harness can additionally pin driver-version discovery and metadata/TTL behavior without a network connection or a real browser.
Scope note: only the driver-resolution / cache / metadata path was audited. Other untested areas (archive uncompress, Grid jar resolution, browser detection, config precedence) were mapped as uncovered but not bug-hunted — they are unaudited, not verified clean.
Background: An integration test failed today because the driver installed on the system didn't match the version selenium manager expected. Integration tests shouldn't depend on system state, so I removed the system driver. But that exposed the underlying problem: the driver-resolution / version-matching logic those tests were implicitly exercising has no unit coverage of its own. Once the test stops leaning on a happens-to-be-present system driver, nothing is left verifying that logic.
Much of
rust/driver-resolution and cache logic is exercised only by the networked integration suite, which asserts "a path exists" rather than which source produced it or what the logic decided. Pure, deterministic logic has almost no unit coverage, so regressions and edge-case bugs are invisible.Why it matters — what a quick audit of just the resolution/cache path turned up:
rust/src/edge.rs:255) and Firefox (rust/src/firefox.rs:287) but not Chrome (rust/src/chrome.rs:342, which has&& !driver_version.is_empty()). On a version-less HTTP 200 (a misconfigureddriver-mirror-urlor an intercepting proxy), the empty string is written to metadata and poisons it for the TTL window (default 1h); every subsequent run reads backSome("")and fails "version cannot be discovered", while Chrome self-heals. Underlying smell:request_driver_versionreturnsOk("")instead of erroring on a version-less response.find_best_driver_from_cachesorts cached version directories lexically rather than by semver, and matches withstarts_with(major_browser_version). Neither bites current version numbers, but both are silent wrong-driver risks (lexical sort misorders across digit-count boundaries like0.9→0.10;starts_withcollides for short majors, andstarts_with("")matches every cached driver when the major version is empty).Coverage to add (pure, fast, deterministic, currently zero unit tests):
parse_version— per-browser--versionstring parsing (Chrome/Edge/Firefox/ESR/snap variants, theerrorshort-circuit, empty-on-garbage).find_best_driver_from_cache/find_latest_from_cache/is_driver_and_matches_browser_version).get_major_version,get_index_version,is_stable/is_beta/is_dev/is_nightly/is_esr,is_version_specific).str_to_os,OS::is,ARCH::is,get_env_name).A lightweight in-process mock-HTTP server + temp-cache harness can additionally pin driver-version discovery and metadata/TTL behavior without a network connection or a real browser.
Scope note: only the driver-resolution / cache / metadata path was audited. Other untested areas (archive
uncompress, Grid jar resolution, browser detection, config precedence) were mapped as uncovered but not bug-hunted — they are unaudited, not verified clean.