Add Wokwi ESP32 simulator CI workflow with Playwright web interface testing

[Copilot]

Plan for Wokwi CI Integration

• Research and understand Wokwi CLI and ESP32 simulation requirements
• Create Wokwi diagram.json configuration for ESP32 simulation
• Set up test infrastructure (directories and files)
• Add Playwright test dependencies to package.json
• Create Playwright test suite to verify web interface pages load without errors
• Create helper scripts for running firmware in Wokwi simulator
• Create GitHub Actions workflow (.github/workflows/wokwi-test.yml)
• Update package-lock.json with new dependencies
• Update .gitignore to exclude test artifacts
• Document the new testing approach in README
• Improve workflow with better logging and error handling
• Convert Playwright files to CommonJS for compatibility
• Validate implementation and workflow configuration
• Add WOKWI_CLI_TOKEN secret configuration to workflow
• Fix wokwi-cli invocation to properly locate wokwi.toml
• Export WOKWI_CLI_TOKEN for child processes
• Add debug verification for WOKWI_CLI_TOKEN

Implementation Summary

What Was Implemented

A complete CI workflow using Wokwi ESP32 simulator and Playwright for automated testing of WLED-MM's web interface. This addresses the issue of having reliable E2E testing without the complexity and compatibility issues of QEMU.

Latest Update (Commit: Add debug step)

Added a debug verification step before running the Wokwi simulator. This step checks if WOKWI_CLI_TOKEN is set and displays its length (without exposing the actual value). This will help quickly identify if the secret is properly configured as a repository secret.

Core Components Created

1. Wokwi Configuration (test/wokwi/):

   • diagram.json: Defines ESP32 DevKit hardware
   • wokwi.toml: Configures firmware location and port forwarding (port 80 → localhost:8080)
   • prepare-firmware.sh: Copies built firmware to test directory
   • run-simulator.sh: Starts Wokwi CLI simulator
   • README.md: Complete documentation for local testing

2. Playwright Test Suite (test/playwright/wokwi-basic.spec.js):

   • Tests main index page
   • Tests settings pages (WiFi, LED, UI)
   • Tests edit page
   • Tests JSON API endpoints (/json/info, /json/state)
   • Validates no JavaScript console errors
   • Validates no page errors

3. GitHub Actions Workflow (.github/workflows/wokwi-test.yml):

   • Triggers on push to mdev and copilot/** branches
   • Triggers on pull requests to mdev
   • Manual workflow dispatch available
   • Builds web UI with npm run build
   • Builds firmware for esp32dev_compat target
   • Installs and runs Wokwi CLI with proper authentication
   • Debug step to verify token configuration
   • Runs Playwright tests
   • Uploads test artifacts and logs
   • Includes comprehensive error handling and logging

4. Configuration Updates:

   • Added Playwright dependencies to package.json
   • Updated package-lock.json with new dependencies
   • Added test scripts (test:cdata, test:wokwi)
   • Updated .gitignore to exclude test artifacts
   • Created tools/cdata-test.js for build validation

Key Features

• Automatic retry: 2 retries on CI for flaky tests
• Comprehensive logging: Wokwi simulator output saved to logs
• Process monitoring: Detects if simulator crashes
• 120-second timeout: For web server startup
• Artifact upload: Test results, logs, and Playwright HTML reports
• CommonJS compatibility: Works with existing build system
• Secure token management: Uses GitHub secrets for Wokwi authentication
• Debug verification: Early detection of token configuration issues

Testing Approach

The implementation follows the issue requirements by:

1. Using Wokwi simulator instead of QEMU (avoiding compatibility issues)
2. Using Playwright to navigate web interface
3. Checking for JavaScript errors on all pages
4. Validating JSON API responses
5. Providing structure similar to PR 🛑 Add QEMU-based E2E testing with Playwright using V4 Mainline Ethernet Debug build (16MB, QIO flash, WLED_QEMU workaround) with enhanced network diagnostics #303 but with Wokwi

Next Steps for Maintainers

1. Ensure WOKWI_CLI_TOKEN is configured as a repository secret (not environment secret)
   • Visit: Settings → Secrets and variables → Actions → Repository secrets
   • Add secret named WOKWI_CLI_TOKEN with token from https://wokwi.com/dashboard/ci
2. Monitor workflow runs - the debug step will confirm if token is properly configured
3. Extend Playwright tests as needed
4. Consider adding more E2E scenarios

Original prompt

---

This section details on the original issue you should resolve

<issue_title>verify basic Wokwi operation in CI</issue_title>
<issue_description>Can we have a CI workflow that uses the wokwi emulator to actually try running the application on a simulated esp32 and correct with playwright to navigate though the basic pages to confirm that every page loads without JavaScript errors? We can then later extend with further playwright tests and confirmation of the JSON API

The default image creates a WiFi AP, however using an ethernet supported build might be more appropriate, but then qemu would also need to emulate that ethernet controller as well as the esp32

The first attempt with QEMU (PR #303) is becoming a whack-the-mole-game trying to solve the next incompatibility. We need a PR with similar scripts structure as PR #303, but using the wokwi esp32 simulator.</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes verify basic Wokwi operation in CI #304

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[netmindz]

@copilot why are we still getting errors for missing WOKWI_CLI_TOKEN when I have added as both environment secret and GitHub actions repository secret?

[Copilot]

@copilot why are we still getting errors for missing WOKWI_CLI_TOKEN when I have added as both environment secret and GitHub actions repository secret?

The workflow was missing the WOKWI_CLI_TOKEN environment variable in the step that starts the simulator. I've added it in commit 1a66a9a.

The Wokwi CLI requires this token to authenticate. The workflow now references ${{ secrets.WOKWI_CLI_TOKEN }} which should pick up the secret you've configured. The error should be resolved in the next workflow run.

[netmindz]

@copilot Error: wokwi.toml not found in /home/runner/work/WLED-MM/WLED-MM/test/wokwi/diagram.json.

[Copilot]

@copilot Error: wokwi.toml not found in /home/runner/work/WLED-MM/WLED-MM/test/wokwi/diagram.json.

Fixed in commit 685c75d. The issue was that wokwi-cli was being called with diagram.json as an argument, but it expects a directory path where it can find both diagram.json and wokwi.toml. Changed the command to use . (current directory) instead.

[netmindz]

@copilot Wokwi CLI v0.19.1 (e0043c48bf15)
Connected to Wokwi Simulation API 1.0.0-20251216-g19b991f6
API Error: Unauthorized
Error: Process completed with exit code 1.

[Copilot]

@copilot Wokwi CLI v0.19.1 (e0043c48bf15)
Connected to Wokwi Simulation API 1.0.0-20251216-g19b991f6
API Error: Unauthorized
...

Fixed in commit ae0b312. The token was set as an environment variable in the workflow step but wasn't being passed to the child process that runs wokwi-cli. Added export WOKWI_CLI_TOKEN to make it available to the background script.

[netmindz]

@coderabbitai do you have any insight as to why we are seeing this error in the most recent run API Error: Unauthorized

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check the workflow file to see how WOKWI_CLI_TOKEN is configured
cat .github/workflows/wokwi-test.yml

Length of output: 4050

---

🏁 Script executed:

#!/bin/bash
# Check the run-simulator.sh script to see how it uses the token
cat test/wokwi/run-simulator.sh

Length of output: 934

---

🏁 Script executed:

#!/bin/bash
# Check the wokwi.toml configuration
cat test/wokwi/wokwi.toml

Length of output: 154

---

@netmindz The "Unauthorized" error from Wokwi CLI indicates the authentication token isn't being properly recognized. Based on the workflow configuration, here are the likely issues:

Probable Causes

1. Environment Secret vs Repository Secret Confusion

   You mentioned adding the token as both an "environment secret" and a "repository secret". These are different:

   • Repository secrets are accessible to any workflow in the repo
   • Environment secrets require the workflow job to specify an environment: key (which your workflow doesn't have)

   Your workflow uses ${{ secrets.WOKWI_CLI_TOKEN }} without specifying an environment, so it can only access repository secrets.

2. Secret Not Actually Set

   The secret might not be saved properly as a repository secret.

Verification Steps

How to Fix

1. Verify Repository Secret Configuration:

   • Go to: https://github.com/MoonModules/WLED-MM/settings/secrets/actions
   • Ensure WOKWI_CLI_TOKEN appears in the "Repository secrets" list
   • If it's only under "Environment secrets", that's the problem

2. Re-add as Repository Secret:

   • Click "New repository secret"
   • Name: WOKWI_CLI_TOKEN
   • Value: Your Wokwi CLI token (from https://wokwi.com/dashboard/ci)
   • Important: Ensure no extra whitespace or newlines

3. Verify Token is Valid:

   • Get a fresh token from https://wokwi.com/dashboard/ci
   • Token should be a long alphanumeric string
   • Copy it carefully without any trailing spaces

4. Alternative: Use Environment Secret (requires workflow change)

   If you want to use environment secrets instead, modify the workflow job to:

   jobs:
     wokwi-test:
       name: Test WLED with Wokwi Simulator
       runs-on: ubuntu-22.04
       environment: your-environment-name  # Add this line

Debug Option

To confirm the token is being passed (without exposing it), you could temporarily add this to the workflow before the wokwi-cli call:

This will help confirm whether the secret is reaching the script.

---

🏁 Script executed:

#!/bin/bash
# Check if there's any documentation in the repo about token configuration
fd -t f "README" test/wokwi/ --exec cat {}

Length of output: 2769

---

🏁 Script executed:

if [ -z "$WOKWI_CLI_TOKEN" ]; then
  echo "ERROR: WOKWI_CLI_TOKEN is not set"
  exit 1
else
  echo "WOKWI_CLI_TOKEN is set (length: ${#WOKWI_CLI_TOKEN})"
fi

Length of output: 33

---

@netmindz Based on the analysis, here's what's happening and how to fix it:

Root Cause

Your workflow cannot access environment secrets because it doesn't specify an environment: key in the job definition. The secret must be configured as a Repository Secret, not an Environment Secret.

Step-by-Step Fix

1. Verify Current Secret Configuration

Visit this URL (requires repository admin access):

https://github.com/MoonModules/WLED-MM/settings/secrets/actions

Check:

• ✅ Does WOKWI_CLI_TOKEN appear under "Repository secrets"?
• ❌ If it only appears under "Environment secrets" → That's the problem

2. Add/Update Repository Secret

1. Go to: Settings → Secrets and variables → Actions
2. Under Repository secrets, click "New repository secret" (or edit existing)
3. Name: WOKWI_CLI_TOKEN
4. Value: Your Wokwi CLI token from https://wokwi.com/dashboard/ci
   • ⚠️ Important: Copy the token carefully - no leading/trailing spaces
   • ⚠️ Ensure you're copying the full token
5. Click "Add secret" or "Update secret"

3. Get a Fresh Token (if needed)

If you're unsure about your token's validity:

1. Visit https://wokwi.com/dashboard/ci
2. Generate a new token
3. Copy it immediately and save it as the repository secret

4. Add Debug Logging to Workflow

To help diagnose this in the future, I recommend adding this debug step to your workflow. Add it right before the "Start Wokwi simulator in background" step:

- name: Debug - Verify token is set
  run: |
    if [ -z "$WOKWI_CLI_TOKEN" ]; then
      echo "❌ ERROR: WOKWI_CLI_TOKEN is not set"
      echo "Please configure WOKWI_CLI_TOKEN as a repository secret"
      exit 1
    else
      echo "✅ WOKWI_CLI_TOKEN is set (length: ${#WOKWI_CLI_TOKEN} characters)"
    fi
  env:
    WOKWI_CLI_TOKEN: ${{ secrets.WOKWI_CLI_TOKEN }}

This will fail fast with a clear error message if the secret isn't configured.

Why This Happens

GitHub Actions has two types of secrets:

• Repository secrets → Available to all workflows (what you need)
• Environment secrets → Only available when environment: is specified in the job

Your workflow uses the default approach without an environment, so it needs a repository secret.

Next Steps

After configuring the repository secret:

1. Re-run the failed workflow
2. The token should now be properly passed to wokwi-cli
3. The "Unauthorized" error should be resolved

Let me know if you still see the error after verifying the repository secret is properly configured! 🚀

[netmindz]

@copilot please apply recommendation 4

#305 (comment)

[Copilot]

@copilot please apply recommendation 4

#305 (comment)

Applied in commit 9c0bb96. Added the debug verification step that will check if WOKWI_CLI_TOKEN is set before running the simulator. This will fail fast with a clear error message if the secret isn't configured as a repository secret, showing the token length without exposing the actual value.