Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion docs/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,10 @@ docs/
import { test, expect } from "@red-hat-developer-hub/e2e-test-utils/test";

test.describe("Feature", () => {
test.beforeAll(async ({ rhdh }) => { /* deploy */ });
test.beforeAll(async ({ rhdh }) => {
await rhdh.configure({ auth: "keycloak" });
await rhdh.deploy();
});
test.beforeEach(async ({ loginHelper }) => { /* login */ });
test("should...", async ({ uiHelper }) => { /* test */ });
});
Expand Down Expand Up @@ -121,6 +124,8 @@ When documenting, reference these source files:
| Fixtures | `src/playwright/fixtures/test.ts` |
| Base Config | `src/playwright/base-config.ts` |
| Global Setup | `src/playwright/global-setup.ts` |
| Teardown Reporter | `src/playwright/teardown-reporter.ts` |
| Teardown Namespaces | `src/playwright/teardown-namespaces.ts` |

## Common Tasks

Expand Down Expand Up @@ -209,5 +214,6 @@ Base URL is configured as `/rhdh-e2e-test-utils/` in `config.ts`.
| Helpers | `@red-hat-developer-hub/e2e-test-utils/helpers` | UIhelper, LoginHelper, etc. |
| Page objects | `@red-hat-developer-hub/e2e-test-utils/pages` | CatalogPage, HomePage, etc. |
| Utilities | `@red-hat-developer-hub/e2e-test-utils/utils` | KubernetesClientHelper, etc. |
| Teardown | `@red-hat-developer-hub/e2e-test-utils/teardown` | Custom namespace teardown registration |
| ESLint | `@red-hat-developer-hub/e2e-test-utils/eslint` | ESLint config |
| TypeScript | `@red-hat-developer-hub/e2e-test-utils/tsconfig` | TSConfig base |
2 changes: 1 addition & 1 deletion docs/api/playwright/base-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Raw base configuration object. Use for advanced customization.
retries: Number(process.env.PLAYWRIGHT_RETRIES ?? 0),
workers: process.env.PLAYWRIGHT_WORKERS || "50%",
outputDir: "node_modules/.cache/e2e-test-results",
reporter: [["list"], ["html"], ["json"]],
reporter: [["list"], ["html"], ["json"], ["teardown-reporter"]],
use: {
viewport: { width: 1920, height: 1080 },
video: { mode: "retain-on-failure", size: { width: 1280, height: 720 } },
Expand Down
32 changes: 31 additions & 1 deletion docs/api/playwright/test-fixtures.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ import { test, expect } from "@red-hat-developer-hub/e2e-test-utils/test";

**Type:** `RHDHDeployment`

Shared RHDH deployment across all tests in a worker.
Shared RHDH deployment across all tests in a worker. `deploy()` automatically skips if the deployment already succeeded, even after worker restarts.

```typescript
test.beforeAll(async ({ rhdh }) => {
Expand Down Expand Up @@ -80,6 +80,36 @@ test("using baseURL", async ({ page, baseURL }) => {
});
```

## `test.runOnce`

```typescript
test.runOnce(key: string, fn: () => Promise<void> | void): Promise<boolean>
```

Executes `fn` exactly once per test run, even across worker restarts. Returns `true` if executed, `false` if skipped.

::: tip
`rhdh.deploy()` already uses `runOnce` internally, so you don't need to wrap simple deployments. Use `test.runOnce` when you have **additional expensive operations** (external services, scripts, data seeding) alongside `deploy()`.
:::

| Parameter | Type | Description |
|-----------|------|-------------|
| `key` | `string` | Unique identifier for this operation |
| `fn` | `() => Promise<void> \| void` | Function to execute once |

```typescript
// Wrap pre-deploy setup that shouldn't repeat
test.beforeAll(async ({ rhdh }) => {
await test.runOnce("full-setup", async () => {
await $`bash deploy-external-service.sh`;
await rhdh.configure({ auth: "keycloak" });
await rhdh.deploy(); // safe to nest, has its own internal protection
});
});
```

See [Deployment Protection](/guide/core-concepts/playwright-fixtures#deployment-protection-built-in) and [`test.runOnce`](/guide/core-concepts/playwright-fixtures#test-runonce-—-run-any-expensive-operation-once) for details.

## Exported Types

```typescript
Expand Down
13 changes: 12 additions & 1 deletion docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,18 @@

All notable changes to this project will be documented in this file.

## [1.1.13] - Current
## [1.1.14] - Current

### Added
- **`deploy()` built-in protection**: `rhdh.deploy()` now automatically skips if the deployment already succeeded in the current test run. No code changes needed — existing `beforeAll` patterns work as before, but deployments are no longer repeated when Playwright restarts workers after test failures.
- **`test.runOnce(key, fn)`**: Execute any function exactly once per test run, even across worker restarts. Use for expensive pre-deploy operations (external services, setup scripts, data seeding) that `deploy()` alone doesn't cover. Safe to nest with `deploy()`'s built-in protection.
- **Teardown reporter**: Built-in Playwright reporter that automatically deletes Kubernetes namespaces after all tests complete. Active only in CI (`process.env.CI`).
- **`registerTeardownNamespace(projectName, namespace)`**: Register custom namespaces for automatic cleanup. Import from `@red-hat-developer-hub/e2e-test-utils/teardown`.

### Changed
- Namespace cleanup moved from worker fixture to teardown reporter to prevent premature deletion on test failures.

## [1.1.13]

### Added
- Support for GitHub authentication provider
Expand Down
159 changes: 152 additions & 7 deletions docs/guide/core-concepts/playwright-fixtures.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,16 +154,161 @@ export default defineConfig({
});
```

## Auto-Cleanup
## Deployment Protection (Built-in)

In CI environments (when `CI` environment variable is set):
`rhdh.deploy()` is automatically protected against redundant re-execution. When a test fails and Playwright restarts the worker, `deploy()` detects that the deployment already succeeded and skips — no re-deployment, no wasted time.

- Namespaces are automatically deleted after tests complete
- Prevents resource accumulation on shared clusters
This works out of the box. A simple `beforeAll` is all you need:

For local development:
- Namespaces are preserved for debugging
- Manual cleanup may be required
```typescript
test.beforeAll(async ({ rhdh }) => {
await rhdh.configure({ auth: "keycloak" });
await rhdh.deploy(); // runs once, skips on worker restart
});
```

::: tip Why is this needed?
Playwright's `beforeAll` runs once **per worker**, not once per test run. When a test fails, Playwright kills the worker and creates a new one for remaining tests — causing `beforeAll` to run again. Without protection, this would re-deploy RHDH from scratch every time a test fails.
:::

## `test.runOnce` — Run Any Expensive Operation Once

While `rhdh.deploy()` has built-in protection, you may have **other expensive operations** in your `beforeAll` that also shouldn't repeat on worker restart — deploying external services, seeding databases, running setup scripts, etc.

`test.runOnce` ensures any function executes **exactly once per test run**, even across worker restarts:

```typescript
test.beforeAll(async ({ rhdh }) => {
await test.runOnce("tech-radar-setup", async () => {
await rhdh.configure({ auth: "keycloak" });
await $`bash ${setupScript} ${namespace}`; // expensive external service
process.env.DATA_URL = await rhdh.k8sClient.getRouteLocation(namespace, "my-service");
await rhdh.deploy(); // also protected internally, nesting is safe
});
});
```

### How It Works

- Uses file-based flags scoped to the Playwright runner process
- When a worker restarts after a test failure, `runOnce` detects the flag and skips
- Any state created by the function (deployments, services, data) stays alive
- Flags reset automatically between test runs

### When to Use

| Scenario | What to use |
|----------|------------|
| Just `configure()` + `deploy()` | Nothing extra — `deploy()` is already protected |
| Pre-deploy setup (external services, scripts, env vars) + `deploy()` | Wrap the entire block in `test.runOnce` |
| Multiple independent expensive operations | Use separate `test.runOnce` calls with different keys |

### Examples

**Simple deployment — no `test.runOnce` needed:**

```typescript
test.beforeAll(async ({ rhdh }) => {
await rhdh.configure({ auth: "keycloak" });
await rhdh.deploy();
});
```

**Pre-deploy setup — wrap in `test.runOnce`:**

```typescript
test.beforeAll(async ({ rhdh }) => {
await test.runOnce("tech-radar-full-setup", async () => {
await rhdh.configure({ auth: "keycloak" });
await $`bash deploy-external-service.sh ${rhdh.deploymentConfig.namespace}`;
process.env.DATA_URL = await rhdh.k8sClient.getRouteLocation(
rhdh.deploymentConfig.namespace, "data-provider"
);
await rhdh.deploy();
});
});
```

**Multiple independent operations with separate keys:**

```typescript
test.describe("Feature A", () => {
test.beforeAll(async ({ rhdh }) => {
await test.runOnce("seed-catalog-data", async () => {
await apiHelper.importEntity("https://example.com/catalog-info.yaml");
});
});
});

test.describe("Feature B", () => {
test.beforeAll(async () => {
await test.runOnce("deploy-mock-api", async () => {
await $`bash deploy-mock.sh`;
});
});
});
```

### Key: Unique Identifier

The `key` parameter must be unique across all `runOnce` calls in your test run. Use a descriptive name that reflects the operation:

```typescript
await test.runOnce("tech-radar-deploy", async () => { ... });
await test.runOnce("tech-radar-data-provider", async () => { ... });
await test.runOnce("catalog-seed-data", async () => { ... });
```

### Nesting

`test.runOnce` can be safely nested. Since `rhdh.deploy()` uses `runOnce` internally, wrapping it in an outer `test.runOnce` is harmless — the outer call skips everything on worker restart, and the inner one never runs:

```typescript
await test.runOnce("full-setup", async () => {
await $`bash setup.sh`; // protected by outer runOnce
await rhdh.deploy(); // has its own internal runOnce (harmless)
});
```

## Namespace Cleanup (Teardown)

In CI environments (`CI` environment variable is set), namespaces are automatically deleted after all tests complete. This is handled by a built-in **teardown reporter** that:

1. Runs in the main Playwright process (survives worker restarts)
2. Waits for **all tests** to finish
3. Deletes the namespace matching the project name

### Default Behavior

No configuration needed. The namespace is derived from your project name:

```typescript
// playwright.config.ts
projects: [
{ name: "tech-radar" }, // Namespace "tech-radar" deleted after all tests
{ name: "catalog" }, // Namespace "catalog" deleted after all tests
]
```

### Custom Namespaces

If you deploy to a namespace that differs from the project name, register it for cleanup:

```typescript
import { registerTeardownNamespace } from "@red-hat-developer-hub/e2e-test-utils/teardown";

test.beforeAll(async ({ rhdh }) => {
await rhdh.configure({ namespace: "my-custom-ns", auth: "keycloak" });
await rhdh.deploy();
registerTeardownNamespace("my-project", "my-custom-ns");
});
```

Multiple namespaces per project are supported — all registered namespaces are deleted after that project's tests complete.

### Local Development

Namespaces are **not** deleted locally (only in CI). This preserves deployments for debugging.

## Best Practices for Projects and Spec Files

Expand Down
33 changes: 19 additions & 14 deletions docs/guide/deployment/rhdh-deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ import { test } from "@red-hat-developer-hub/e2e-test-utils/test";
test.beforeAll(async ({ rhdh }) => {
// rhdh is already instantiated with namespace from project name
await rhdh.configure({ auth: "keycloak" });
await rhdh.deploy();
await rhdh.deploy(); // automatically skips if already deployed
});

test("example", async ({ rhdh }) => {
Expand Down Expand Up @@ -110,6 +110,8 @@ test.setTimeout(900_000);
await rhdh.deploy({ timeout: null });
```

`deploy()` automatically skips if the deployment already succeeded in the current test run (e.g., after a worker restart due to test failure). This prevents expensive re-deployments.

This method:
1. Merges configuration files (common → auth → project)
2. [Injects plugin metadata](/guide/configuration/config-files#plugin-metadata-injection) into dynamic plugins config
Expand Down Expand Up @@ -176,7 +178,7 @@ await deployment.teardown();
```

::: warning
This permanently deletes all resources in the namespace. In CI, this happens automatically.
You typically don't need to call this manually. In CI, the built-in teardown reporter automatically deletes namespaces after all tests complete. See [Namespace Cleanup](/guide/core-concepts/playwright-fixtures#namespace-cleanup-teardown).
:::

## Properties
Expand Down Expand Up @@ -259,21 +261,24 @@ import { test } from "@red-hat-developer-hub/e2e-test-utils/test";
import { $ } from "@red-hat-developer-hub/e2e-test-utils/utils";

test.beforeAll(async ({ rhdh }) => {
const namespace = rhdh.deploymentConfig.namespace;
// Wrap in test.runOnce because the setup script is also expensive
await test.runOnce("my-plugin-setup", async () => {
const namespace = rhdh.deploymentConfig.namespace;

// Configure RHDH
await rhdh.configure({ auth: "keycloak" });
// Configure RHDH
await rhdh.configure({ auth: "keycloak" });

// Run custom setup before deployment
await $`bash scripts/setup.sh ${namespace}`;
// Run custom setup before deployment
await $`bash scripts/setup.sh ${namespace}`;

// Set runtime environment variables
process.env.MY_CUSTOM_URL = await rhdh.k8sClient.getRouteLocation(
namespace,
"my-service"
);
// Set runtime environment variables
process.env.MY_CUSTOM_URL = await rhdh.k8sClient.getRouteLocation(
namespace,
"my-service"
);

// Deploy RHDH (uses env vars set above)
await rhdh.deploy();
// Deploy RHDH (has built-in protection, safe to nest inside runOnce)
await rhdh.deploy();
});
});
```
Loading