Skip to content

Commit 8a1e893

Browse files
authored
test: add comprehensive e2e scenario integration tests (#49)
* test: add comprehensive e2e scenario integration tests Add 24 end-to-end tests covering the full FirefoxClient API against a realistic multi-page web app fixture: todo CRUD, search, form submission, click/dblclick/hover, browser history, viewport resize, console & network monitoring, screenshots, tab management, stale UID detection, and error handling. All tests are self-contained with no ordering dependencies. Also update README with testing docs and known Firefox 148 startup crash on macOS ARM64 (Bug 2027228). * docs: move testing details to docs/testing.md, keep README concise * fix: address PR review — simplify element predicates, remove stale docs table Remove unnecessary data-testid/data-uid fallback conditions from element lookup predicates since the implementation always stores id with priority. Remove the "Covered scenarios" table from docs/testing.md to avoid staleness.
1 parent 90f274d commit 8a1e893

File tree

4 files changed

+1104
-1
lines changed

4 files changed

+1104
-1
lines changed

README.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,15 @@ npx @modelcontextprotocol/inspector node dist/index.js --headless --viewport 128
162162
npm run inspector:dev
163163
```
164164

165+
## Testing
166+
167+
```bash
168+
npm run test:run # all tests once (unit + integration)
169+
npm test # watch mode
170+
```
171+
172+
See [docs/testing.md](docs/testing.md) for full details on running specific test suites, the e2e scenario coverage, and known issues.
173+
165174
## Troubleshooting
166175

167176
- Firefox not found: pass `--firefox-path "/Applications/Firefox.app/Contents/MacOS/firefox"` (macOS) or the correct path on your OS.
@@ -180,7 +189,7 @@ npm run inspector:dev
180189
```
181190

182191
> **The Key Change:** On Windows, running a Node.js package via `npx` often requires the `cmd /c` prefix to be executed correctly from within another process like VSCode's extension host. Therefore, `"command": "npx"` was replaced with `"command": "cmd"`, and the actual `npx` command was moved into the `"args"` array, preceded by `"/c"`. This fix allows Windows to interpret the command correctly and launch the server.
183-
192+
184193
- **Solution 2** Instead of another layer of shell you can write the absolute path to `npx`:
185194

186195
```json

docs/testing.md

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Testing
2+
3+
## Running tests
4+
5+
```bash
6+
# Run all tests once (unit + integration)
7+
npm run test:run
8+
9+
# Run only unit tests (fast, no Firefox needed)
10+
npx vitest run tests/tools tests/firefox tests/utils tests/snapshot tests/cli tests/config tests/smoke.test.ts
11+
12+
# Run only integration tests (launches real Firefox in headless mode)
13+
npx vitest run tests/integration
14+
15+
# Run the e2e scenario suite
16+
npx vitest run tests/integration/e2e-scenario.integration.test.ts
17+
18+
# Watch mode (re-runs on file changes)
19+
npm test
20+
```
21+
22+
## E2E scenario tests
23+
24+
The file `tests/integration/e2e-scenario.integration.test.ts` contains end-to-end
25+
tests that exercise the full `FirefoxClient` API against a realistic multi-page
26+
web application (`tests/fixtures/e2e-app.html`).
27+
28+
The fixture app has three pages (Todo List, Search, Registration Form) plus
29+
always-visible hover/double-click targets. Each `describe` block launches its own
30+
headless Firefox instance and tears it down after the tests.
31+
32+
All tests are self-contained (no ordering dependencies) and use active polling
33+
(`waitFor`) instead of fixed sleeps for async BiDi events.
34+
35+
### Design principles
36+
37+
- **Self-contained**: each test navigates to its own page, no inter-test dependencies
38+
- **Active polling**: async events (console, network) use `waitFor` instead of fixed sleeps
39+
- **Relative assertions**: viewport tests assert relative change, not exact pixel values (platform-dependent)
40+
- **Isolated Firefox instances**: each `describe` block gets its own headless Firefox
41+
42+
## Known issues
43+
44+
- **Firefox 148 startup crash on macOS ARM64** ([Bug 2027228](https://bugzilla.mozilla.org/show_bug.cgi?id=2027228)): Intermittent SIGSEGV in `RegisterFonts` thread (`RWLockImpl::writeLock()` null pointer) when launching Firefox in headless mode via Selenium. The crash is a race condition in Firefox font initialization and does not affect test results — Selenium recovers automatically. More likely to occur under fast sequential startup/shutdown cycles.

0 commit comments

Comments
 (0)