feat(testing): E2E runtime, enhanced smoke, and Playwright tests#103
Open
feat(testing): E2E runtime, enhanced smoke, and Playwright tests#103
Conversation
…and Playwright Adds three new testing layers for runtime validation of generated projects: **Layer 1: E2E Runtime Tests** (apps/cli/test/e2e/) - Expanded from 3 to 9 test configs covering Next.js, Nuxt, SvelteKit, TanStack Start, React Router, Astro with both fullstack and standalone backends - New startDevServer() runs turbo dev and validates frontend pages load - checkFrontendPage() validates HTML, detects error patterns - checkStaticAssets() verifies CSS/JS bundles load - validateFrameworkPage() checks framework-specific markers **Layer 2: Enhanced Smoke Tests** (testing/) - Added --strict flag: dev-check failures become blocking (not advisory) - 5 new presets: next-payload, astro-sanity, nuxt-fullstack, react-router-hono, tanstack-start-fullstack - Improved HTML validation with framework-specific marker detection - Empty body detection catches render failures **Layer 3: Playwright Web Builder Tests** (apps/web/test/e2e/) - Playwright configured with chromium, auto-starts dev server - data-testid attributes added to stack builder and preview panel - 5 test specs: builder load, CLI command display, URL state, preview file tree, URL sharing restoration **CI Workflow** (.github/workflows/e2e-test.yaml) - Nightly at 4am UTC + manual dispatch - 3 parallel jobs: e2e-runtime, smoke-strict, playwright - Artifact upload for debugging failures Root scripts: test:e2e, test:smoke:strict, test:playwright
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
…ification, and turbo wiring Addresses gaps in testing coverage: - CLI binary tests: spawn actual `node dist/cli.mjs` with --yes, --yolo, --ecosystem flags, verify project files are created correctly - CLI interaction tests: test flag validation (incompatible combos rejected), ecosystem selection, directory conflict handling via spawned process - TypeScript verification: typecheckProject() runs tsc --noEmit on generated projects to verify zero type errors - Turbo pipeline: test:e2e and test:integration tasks added to turbo.json, `turbo test` now runs all test suites across all packages - Root scripts: test:e2e (CLI), test:e2e:web (Playwright), test:integration Test results: 8/8 interaction tests pass, 7/7 binary tests pass
Adds the 2 missing Playwright test files: - compatibility.spec.ts: Convex disables options, Payload requires Next.js, Sanity works everywhere - mobile.spec.ts: mobile viewport, no overflow, CLI command visible, landing page Total: 13 Playwright tests across 5 spec files
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds three new testing layers for runtime validation of generated projects, closing critical gaps in test coverage. All run nightly + manually — not on every PR.
Layer 1: E2E Runtime Tests
startDevServer()— runsturbo dev, waits for correct frontend portcheckFrontendPage()— validates HTML, detects SSR errors, hydration mismatchescheckStaticAssets()— verifies CSS/JS bundles load (200 status)validateFrameworkPage()— checks_next,_nuxt,type="module"markersbun run test:e2eLayer 2: Enhanced Smoke Tests
--strictflag makes dev-check failures blocking (not advisory)next-payload,astro-sanity,nuxt-fullstack,react-router-hono,tanstack-start-fullstackbun run test:smoke:strictLayer 3: Playwright Web Builder Tests
data-testidattributes on key stack builder elementsbun run test:playwrightCI Workflow
.github/workflows/e2e-test.yaml— nightly at 4am UTC + manual dispatchTest plan
bun run lint— 6/6 passE2E=1 bun test apps/cli/test/e2e/e2e.e2e.tswith one configbun run test:smoke -- --preset tanstack-fullstack --dev-check --strictcd apps/web && npx playwright test