Add exhaustive Playwright E2E tests for Training plugin#186
Merged
Add exhaustive Playwright E2E tests for Training plugin#186
Conversation
39 tests across 6 spec files covering page load, certificate/badge selection, flag exercises, red vs blue training paths, progress tracking, and error states. Tests run against a live Caldera instance via CALDERA_URL env var (default http://localhost:8888).
There was a problem hiding this comment.
Pull request overview
Adds a standalone Playwright-based E2E test suite for the CALDERA Training plugin UI to validate core user flows (navigation, certificate selection, badge/flag interactions, and basic error handling) against a running Caldera instance.
Changes:
- Introduces Playwright configuration plus a dedicated
tests/e2enpm package (with lockfile) for running E2E tests. - Adds shared helpers for UI login and navigation to the Training plugin tab.
- Adds multiple spec files covering page load, certificate/badge selection, flags UI behavior, progress/completion indicators, training path switching, and error-state scenarios.
Reviewed changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/e2e/specs/training-page-load.spec.js | Verifies basic login state, nav item visibility, Training page heading, and cert dropdown presence. |
| tests/e2e/specs/training-certificate-selection.spec.js | Exercises cert dropdown population and badge/flag filtering behaviors. |
| tests/e2e/specs/training-flags.spec.js | Validates flag card rendering and expand/collapse interactions after cert selection. |
| tests/e2e/specs/training-progress.spec.js | Checks progress-related UI elements (badge text, flag icons, status bar, completion banner assumptions). |
| tests/e2e/specs/training-paths.spec.js | Tests Red/Blue selection (when available) and switching behavior between certificates. |
| tests/e2e/specs/training-error-states.spec.js | Covers “no selection” initial state and basic API-failure resilience via request interception. |
| tests/e2e/helpers/auth.js | Provides shared login helper with env-var overrides. |
| tests/e2e/helpers/navigation.js | Provides shared helper to navigate to the Training plugin view. |
| tests/e2e/playwright.config.js | Playwright runner configuration (baseURL, timeouts, retries, single worker). |
| tests/e2e/package.json | Defines scripts and Playwright test dependency for the E2E suite. |
| tests/e2e/package-lock.json | Locks Playwright dependencies for reproducible installs. |
| tests/e2e/.gitignore | Ignores Playwright artifacts and node_modules for the E2E project. |
Files not reviewed (1)
- tests/e2e/package-lock.json: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| // Both should have badges loaded | ||
| expect(names1.length).toBeGreaterThanOrEqual(1); | ||
| expect(names2.length).toBeGreaterThanOrEqual(1); |
Comment on lines
+35
to
+39
| const allOptions = select.locator("option:not([disabled])"); | ||
| const count = await allOptions.count(); | ||
| expect(count).toBeGreaterThanOrEqual(1); | ||
| }); | ||
|
|
Comment on lines
+22
to
+26
| test("should show Red Certificate option", async ({ page }) => { | ||
| const select = page.locator("#select-certificate select").first(); | ||
| await expect(select).toBeVisible(); | ||
| const redOption = select.locator('option:has-text("Red")'); | ||
| // Red cert may or may not exist depending on server data - check at least one cert |
Comment on lines
+37
to
+38
| const defaultOption = select.locator("option[disabled], option[value='']").first(); | ||
| await expect(defaultOption).toBeDefined(); |
| // On a fresh server the certificate should not be complete | ||
| // The completion banner contains "Certificate complete" | ||
| const banner = page.locator("h3:has-text('Certificate complete')"); | ||
| // Should NOT be visible (assuming fresh training state) |
|
|
||
| async function selectFirstCertificate(page) { | ||
| const select = page.locator("#select-certificate select").first(); | ||
| await expect(select).toBeVisible(); |
| */ | ||
| async function selectFirstCertificate(page) { | ||
| const select = page.locator("#select-certificate select").first(); | ||
| await expect(select).toBeVisible(); |
| await expect(select).toBeVisible(); | ||
| // There should be at least 1 real option beyond the placeholder | ||
| const options = select.locator("option:not([disabled])"); | ||
| await expect(options.first()).toBeVisible({ timeout: 15_000 }); |
- training-paths.spec.js: add assertion that names1 and names2 differ to verify different certificates show different badge sets - training-certificate-selection.spec.js: wait for certs API before checking options; assert Red and Blue options exist with proper counts; replace unreliable toBeVisible() on option elements - training-page-load.spec.js: replace always-true toBeDefined() with toHaveCount(1) for the placeholder option - training-progress.spec.js: wait for certs API in selectFirstCertificate; make completion banner test resilient to pre-completed state - training-flags.spec.js: wait for certs API in selectFirstCertificate before selecting to avoid flaky cert selection
Contributor
Author
|
@github-copilot review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CALDERA_URLenv var (defaulthttp://localhost:8888)Test plan
cd tests/e2e && npm install && npx playwright install chromium && npx playwright testCALDERA_URLenv var pointing to a non-default port