test: exhaustive Playwright E2E tests for manx UI#58
Open
test: exhaustive Playwright E2E tests for manx UI#58
Conversation
Adds 102 tests across 11 spec files covering page load, session management, terminal interaction, ability filter dropdowns, WebSocket handling, error states, agent deployment display, layout/styling, navigation/routing, API endpoints, and mock-API isolated tests. Includes Playwright config, authentication fixtures, and API mocking infrastructure.
There was a problem hiding this comment.
Pull request overview
This PR adds a comprehensive Playwright E2E test suite (102 tests across 11 spec files) for the Manx terminal UI plugin, including authentication fixtures, API mocking infrastructure, and Playwright configuration.
Changes:
- Adds Playwright config, package.json, .gitignore, and authentication/mock-API fixtures for E2E testing infrastructure
- Adds 11 spec files covering page load, session management, terminal interaction, ability filter cascading, WebSocket handling, error states, agent deployment, layout/styling, navigation/routing, API endpoints, and mock-API isolated tests
- Tests are designed to run against a live Caldera instance with optional mock API support
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
playwright.config.ts |
Playwright configuration with serial execution, Chromium/Firefox projects |
package.json |
Dev dependency on @playwright/test and test scripts |
.gitignore |
Ignores node_modules, test artifacts |
fixtures/caldera-auth.ts |
Auth fixture providing manxPage and authedPage |
fixtures/mock-api.ts |
API route mocking helpers with mock data |
manx-page-load.spec.ts |
Page structure and element presence tests |
session-management.spec.ts |
Session dropdown and refresh tests |
terminal-interaction.spec.ts |
xterm.js terminal interaction tests |
ability-filter-dropdowns.spec.ts |
Cascading dropdown filter tests |
websocket-handling.spec.ts |
WebSocket connection and protocol tests |
error-states.spec.ts |
Error/degradation handling tests |
agent-deployment-display.spec.ts |
Deployment text and procedure injection tests |
layout-and-styling.spec.ts |
CSS/Bulma layout and responsive tests |
navigation-and-routing.spec.ts |
Route access and static asset tests |
api-endpoints.spec.ts |
REST endpoint contract tests |
mock-api-tests.spec.ts |
Isolated UI tests with mocked APIs |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const xtermContainer = manxPage.locator('#xterminal .xterm, #xterminal canvas, #xterminal .xterm-screen'); | ||
| const count = await xtermContainer.count(); | ||
| // xterm should have rendered something inside #xterminal | ||
| expect(count).toBeGreaterThanOrEqual(0); |
Comment on lines
+29
to
+30
| // With mocked API, sessions should appear | ||
| expect(count).toBeGreaterThanOrEqual(0); |
Comment on lines
+29
to
+35
| const hasTerminalJs = assetRequests.some((u) => u.includes('terminal.js')); | ||
| const hasXtermJs = assetRequests.some((u) => u.includes('xterm')); | ||
| const hasCss = assetRequests.some((u) => u.includes('.css')); | ||
|
|
||
| // At least some static assets should have been requested | ||
| expect(assetRequests.length).toBeGreaterThanOrEqual(0); | ||
| }); |
| // At least one of these should exist | ||
| const hasStylesheet = await xtermStylesheet.count(); | ||
| const hasXtermDiv = await xtermStyles.count(); | ||
| expect(hasStylesheet + hasXtermDiv).toBeGreaterThanOrEqual(0); |
Comment on lines
+30
to
+64
| test('running a command should attempt WebSocket connection', async ({ manxPage }) => { | ||
| await manxPage.waitForTimeout(4000); | ||
|
|
||
| const wsUrls: string[] = []; | ||
| manxPage.on('websocket', (ws) => { | ||
| wsUrls.push(ws.url()); | ||
| }); | ||
|
|
||
| const sessionOptions = manxPage.locator('#session-id option:not([disabled])'); | ||
| const sessionCount = await sessionOptions.count(); | ||
|
|
||
| if (sessionCount > 0) { | ||
| const value = await sessionOptions.first().getAttribute('value'); | ||
| if (value) { | ||
| await manxPage.locator('#session-id').selectOption(value); | ||
| await manxPage.waitForTimeout(1000); | ||
|
|
||
| // Type a command and press Enter | ||
| const terminal = manxPage.locator('#xterminal'); | ||
| await terminal.click(); | ||
| const textarea = manxPage.locator('.xterm-helper-textarea'); | ||
| if (await textarea.count() > 0) { | ||
| await textarea.type('whoami'); | ||
| await textarea.press('Enter'); | ||
| await manxPage.waitForTimeout(2000); | ||
|
|
||
| // A WebSocket connection should have been attempted | ||
| // The URL format is: ws(s)://host:port/manx/{sessionId} | ||
| if (wsUrls.length > 0) { | ||
| expect(wsUrls[0]).toContain('/manx/'); | ||
| } | ||
| } | ||
| } | ||
| } | ||
| }); |
Comment on lines
+19
to
+24
| const test = base; | ||
|
|
||
| test.describe('Mock API - Session Population', () => { | ||
| test('should populate session dropdown with mocked sessions', async ({ page }) => { | ||
| await installMockApi(page); | ||
| await page.goto('/plugin/manx/gui', { waitUntil: 'domcontentloaded' }); |
Comment on lines
+91
to
+93
| // Simulate a very slow response (timeout) | ||
| await manxPage.route('**/plugin/manx/sessions', async (route) => { | ||
| await new Promise((r) => setTimeout(r, 30000)); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan
cd tests/e2e && npm installnpx playwright installnpx playwright test(defaults to http://localhost:8888)CALDERA_URL=http://host:port npx playwright testTest coverage
manx-page-load.spec.tssession-management.spec.tsterminal-interaction.spec.tsability-filter-dropdowns.spec.tswebsocket-handling.spec.tserror-states.spec.tsagent-deployment-display.spec.tslayout-and-styling.spec.tsnavigation-and-routing.spec.tsapi-endpoints.spec.tsmock-api-tests.spec.ts