Improve coverage with Cypress E2E tests and split test setup by mode (Zigbee/Matter/Multiprotocol) by tbrkollar · Pull Request #1680 · project-chip/zap

tbrkollar · 2026-01-26T15:05:43Z

Improved overall code coverage by adding a large set of Cypress E2E tests (especially UI coverage).
Split the Cypress E2E setup by Zigbee, Matter, and Multiprotocol modes (separate specs + mode-specific fixtures).
Made coverage collection safer and more reliable by only enabling Cypress coverage hooks when coverage is explicitly turned on, and by combining Jest + Cypress coverage into one report with a simple threshold check.

#1428
#956

gemini-code-assist · 2026-01-26T15:06:26Z

Summary of Changes

Hello @tbrkollar, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a comprehensive overhaul of the project's end-to-end testing strategy. It dramatically expands the existing Cypress test suite to cover a wider array of UI components and application flows, ensuring greater stability and reliability. A key architectural change is the segregation of tests and fixtures into distinct modes (Zigbee, Matter, Multiprotocol), allowing for more targeted and efficient testing. Furthermore, improvements have been made to the test execution environment, such as safer coverage collection and better handling of unexpected runtime errors, contributing to a more robust development process.

Highlights

Cypress E2E Test Expansion: Significantly increased code coverage by adding a large suite of new Cypress E2E tests, particularly focusing on UI components and application functionality across various pages like endpoint management, preferences, and toolbar interactions.
Mode-Specific Test Setup: Refactored the Cypress E2E test infrastructure to support distinct testing modes for Zigbee, Matter, and a new Multiprotocol configuration. This includes dedicated spec patterns, fixture folders, and command-line scripts for each mode.
Improved Test Reliability and Coverage Collection: Enhanced the robustness of Cypress tests by conditionally loading the code coverage plugin only when explicitly enabled, and by adding uncaught exception handling to prevent test failures from minor application errors. This also sets the stage for combining Jest and Cypress coverage reports.
Enhanced Testability with Data Attributes: Added data-cy and data-test attributes to numerous UI elements across various Vue components (e.g., SettingsSidebar, ZCLToolbar, ZclClusterView, ZapConfig), making E2E tests more stable and resilient to UI changes.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request significantly expands the Cypress end-to-end test suite by introducing new tests for various UI components and application functionalities, including config loading scenarios, endpoint duplication, and comprehensive tests for attributes, clusters, and commands in Zigbee mode. It also adds extensive UI tests for the main layout, toolbar, extensions, notifications, options, and preferences pages, along with a dedicated 404 error page test. The cypress.config.js was updated to support dynamic test execution based on protocol mode (Zigbee, Matter, Multiprotocol) and conditional code coverage, and a new cypress/README.md was added to document the expanded test organization. Additionally, a goBackButton custom command was introduced, and data-cy attributes were added to various UI elements to improve test robustness. Review comments highlighted the need to enable skipped tests, fix a typo in cypress/README.md, improve selector robustness in several UI tests by using data-cy attributes, clarify a comment in tour.cy.js, and refactor repetitive endpoint cleanup logic into a custom command.

gemini-code-assist · 2026-01-26T15:13:42Z

  it(
-    'edit endpoint -> Overwrite',
+    'edit endpoint -> delete and add',
    { retries: { runMode: 2, openMode: 2 } },
    () => {
      cy.get('[data-test="edit-endpoint"]').last().click()

      cy.fixture('data').then((data) => {
        cy.get('[data-test="select-endpoint-input"]')
          .click()
-          .type(data.endpoint2.substring(0, 5), { force: true })
+          .type(data.endpoint3.substring(0, 5), { force: true })
        cy.wait(500)
-        cy.get('div').contains(data.endpoint2).click({ force: true })
+        cy.get('div').contains(data.endpoint3).click({ force: true })
      })
      cy.get('[data-test="endpoint-title"]').click() // it makes sure that the previous input field has been unselected
      cy.get('button').contains('Save').click()
      cy.wait(500)
-      // Test overwrite option
-      cy.get('button').contains('Overwrite').click()
+      // Test delete and add option
+      cy.get('button').contains('Delete and Add').click()
      // Verify the endpoint was updated successfully
      cy.fixture('data').then((data) => {
-        cy.get('aside').children().should('contain', data.endpoint2)
+        cy.get('aside').children().should('contain', data.endpoint3)
      })
+      cy.get('[data-test="delete-endpoint"]').each(() => {
+        cy.get('[data-test="delete-endpoint"]').last().click()
+        cy.wait(300)
+        cy.get('#delete_endpoint').click()
+      })
+      cy.wait(300)
+      cy.get('#delete_last_endpoint').click()
    }
  )
+  it('create a new endpoint', () => {
+    // Verify the endpoint was created successfully
+    cy.fixture('data').then((data) => {
+      cy.get('aside').children().should('contain', data.endpoint1)
+    })
+  })
  it(
-    'edit endpoint -> delete and add',
+    'edit endpoint -> Overwrite',
    { retries: { runMode: 2, openMode: 2 } },
    () => {
      cy.get('[data-test="edit-endpoint"]').last().click()

      cy.fixture('data').then((data) => {
        cy.get('[data-test="select-endpoint-input"]')
          .click()
-          .type(data.endpoint3.substring(0, 5), { force: true })
+          .type(data.endpoint2.substring(0, 5), { force: true })
        cy.wait(500)
-        cy.get('div').contains(data.endpoint3).click({ force: true })
+        cy.get('div').contains(data.endpoint2).click({ force: true })
      })
      cy.get('[data-test="endpoint-title"]').click() // it makes sure that the previous input field has been unselected
      cy.get('button').contains('Save').click()
      cy.wait(500)
-      // Test delete and add option
-      cy.get('button').contains('Delete and Add').click()
+      // Test overwrite option
+      cy.get('button').contains('Overwrite').click()
      // Verify the endpoint was updated successfully
      cy.fixture('data').then((data) => {
-        cy.get('aside').children().should('contain', data.endpoint3)
+        cy.get('aside').children().should('contain', data.endpoint2)
      })
    }
  )


The order of the tests edit endpoint -> delete and add and edit endpoint -> Overwrite has been swapped compared to the original file. While not strictly an error, maintaining a consistent or logical flow (e.g., create, then overwrite, then delete) can improve test readability and maintainability. Consider reordering them to match the original logical flow if possible.

gemini-code-assist · 2026-01-26T15:13:42Z

+        return false
+      }
+      // Don't prevent other errors from failing the test
+      return true


The comment // Don't prevent other errors from failing the test is slightly misleading. The return true in the else block means that if an uncaught exception not matching the undefined.map() condition occurs, Cypress will fail the test, which is generally desired behavior. The global Cypress.on('uncaught:exception', (err, runnable) => { return false }) already prevents all uncaught exceptions from failing tests. This specific beforeEach handler is more granular. Clarify the comment to reflect that only specific errors are ignored, and others will still cause a failure.

// Don't prevent other errors from being ignored by this specific handler // (they will still be caught by the global handler if present, or fail the test otherwise) return true

gemini-code-assist · 2026-01-26T15:13:43Z

+    cy.get('body').then(($body) => {
+      if ($body.find('[data-test="delete-endpoint"]').length > 0) {
+        // Delete all endpoints except the last one using each()
+        cy.get('[data-test="delete-endpoint"]').each(() => {
+          cy.get('[data-test="delete-endpoint"]').last().click({ force: true })
+          cy.wait(500)
+          cy.get('#delete_endpoint').should('exist').click({ force: true })
+          cy.wait(500)
+        })
+        // Delete the last endpoint (which shows different dialog)
+        cy.wait(300)
+        cy.get('[data-test="delete-endpoint"]').then(($remaining) => {
+          if ($remaining.length > 0) {
+            cy.get('[data-test="delete-endpoint"]')
+              .last()
+              .click({ force: true })
+            cy.wait(500)
+            cy.get('#delete_last_endpoint')
+              .should('exist')
+              .click({ force: true })
+            cy.wait(500)
+          }
+        })
+      }
+    })


The cleanup logic for deleting endpoints is quite verbose and is repeated across multiple new test files. To improve reusability and readability, consider refactoring this into a custom Cypress command, similar to cy.addEndpoint or cy.setZclProperties.

gemini-code-assist · 2026-01-26T15:13:43Z

+      cy.addEndpoint(data.endpoint4)
+
+      // Use search functionality
+      cy.get('input[placeholder*="Search"], input[type="search"]').then(


The selector cy.get('input[placeholder*="Search"], input[type="search"]') is generic and might become brittle if the UI changes. If a data-cy attribute is available for the search input, it would be more robust to use that, e.g., cy.dataCy('cluster-search-input').

gemini-code-assist · 2026-01-26T15:13:44Z

+    cy.get('body').then(($body) => {
+      if ($body.find('[data-test="delete-endpoint"]').length > 0) {
+        // Delete all endpoints except the last one using each()
+        cy.get('[data-test="delete-endpoint"]').each(() => {
+          cy.get('[data-test="delete-endpoint"]').last().click({ force: true })
+          cy.wait(500)
+          cy.get('#delete_endpoint').should('exist').click({ force: true })
+          cy.wait(500)
+        })
+        // Delete the last endpoint (which shows different dialog)
+        cy.wait(300)
+        cy.get('[data-test="delete-endpoint"]').then(($remaining) => {
+          if ($remaining.length > 0) {
+            cy.get('[data-test="delete-endpoint"]')
+              .last()
+              .click({ force: true })
+            cy.wait(500)
+            cy.get('#delete_last_endpoint')
+              .should('exist')
+              .click({ force: true })
+            cy.wait(500)
+          }
+        })
+      }
+    })


The cleanup logic for deleting endpoints is quite verbose and is repeated across multiple new test files. To improve reusability and readability, consider refactoring this into a custom Cypress command, similar to cy.addEndpoint or cy.setZclProperties.

gemini-code-assist · 2026-01-26T15:13:44Z

+
+      // Search for commands
+      cy.get('input[placeholder*="Search"], input[type="search"]').then(
+        ($inputs) => {


The selector cy.get('input[placeholder*="Search"], input[type="search"]') is generic and might become brittle if the UI changes. If a data-cy attribute is available for the search input, it would be more robust to use that, e.g., cy.dataCy('command-search-input').

gemini-code-assist · 2026-01-26T15:13:44Z

+    cy.get('body').then(($body) => {
+      if ($body.find('[data-test="delete-endpoint"]').length > 0) {
+        // Delete all endpoints except the last one using each()
+        cy.get('[data-test="delete-endpoint"]').each(() => {
+          cy.get('[data-test="delete-endpoint"]').last().click({ force: true })
+          cy.wait(500)
+          cy.get('#delete_endpoint').should('exist').click({ force: true })
+          cy.wait(500)
+        })
+        // Delete the last endpoint (which shows different dialog)
+        cy.wait(300)
+        cy.get('[data-test="delete-endpoint"]').then(($remaining) => {
+          if ($remaining.length > 0) {
+            cy.get('[data-test="delete-endpoint"]')
+              .last()
+              .click({ force: true })
+            cy.wait(500)
+            cy.get('#delete_last_endpoint')
+              .should('exist')
+              .click({ force: true })
+            cy.wait(500)
+          }
+        })
+      }
+    })


The cleanup logic for deleting endpoints is quite verbose and is repeated across multiple new test files. To improve reusability and readability, consider refactoring this into a custom Cypress command, similar to cy.addEndpoint or cy.setZclProperties.

paulr34

This looks great overall. For the typo fixes Gemini suggested, it would probably be best to use “Commit suggestion” so those get applied cleanly.

Since this PR is changing the Cypress tests themselves, it would also be good to run the app manually and verify the behavior. Even if the changes are correct, there’s some inherent risk when modifying tests, so a quick manual check helps confirm everything is behaving as expected.

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

brdandu

Changes look fine to me. The code review suggestions do make sense so please resolve them and lets get this merged in for better code coverage

brdandu · 2026-02-02T14:24:42Z

+  describe('Scenario 1: Zigbee + Matter templates pre-loaded', () => {
+    it('Should verify Zigbee config loads correctly when both templates are pre-loaded', () => {
+      // This test runs in multiprotocol mode where both templates are pre-loaded
+      if (Cypress.env('mode') === 'multiprotocol') {


Avoid using strings across the code. Better to use predefined macros or create your own

This test case will be in my next PR. It only works in multiprotocol mode, so I marked it as skipped.

I’ve pushed a commit for this.

brdandu · 2026-02-02T14:46:39Z

    })
  })
-
-  it('create a new endpoint', () => {


Why is this test removed?

it was duplicated

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Centralize zigbee/matter/multiprotocol mode values in cypress/support/mode.js and replace hard-coded Cypress.env('mode') string comparisons across specs and cypress.config.js with Cypress.Mode.*. Also fix Cypress zap-config spec warning assertions.

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

tbrkollar added 5 commits January 16, 2026 11:42

Improve code coverage with Cypress E2E tests

6259cc0

Modify data-cy on back button

7849a90

Add Multiprotocol test case

61fb03e

Add a lot of tests for ui

0536ea2

Add matter adn multiprotocol config

a7b14b6

tbrkollar self-assigned this Jan 26, 2026

tbrkollar requested review from brdandu, paulr34 and tecimovic as code owners January 26, 2026 15:05

gemini-code-assist Bot reviewed Jan 26, 2026

View reviewed changes

paulr34 approved these changes Jan 28, 2026

View reviewed changes

tbrkollar and others added 3 commits February 2, 2026 12:13

Update cypress/README.md

0ed51a5

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Update cypress/e2e/ui/zap-config.cy.js

f73bcb3

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Update cypress/e2e/ui/zap-config.cy.js

ddabda9

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

brdandu requested changes Feb 2, 2026

View reviewed changes

tbrkollar and others added 6 commits February 3, 2026 12:31

Update cypress/e2e/ui/preferences-devtools.cy.js

3311f8b

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Update cypress/e2e/ui/preferences-devtools.cy.js

1487233

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Update Cypress tests to use more robust selectors

3baffd3

Update cypress/e2e/ui/preferences.cy.js

a9538cf

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Update Cypress tests to use more robust selectors on preference page

f4a4f34

tbrkollar requested a review from brdandu February 3, 2026 13:26

brdandu approved these changes Feb 4, 2026

View reviewed changes

tbrkollar merged commit 854958f into project-chip:master Feb 4, 2026
14 checks passed

This was linked to issues Feb 5, 2026

Update Cypress Matter fixtures for new data #956

Closed

Different Cypress tests for Zigbee and Matter #1428

Closed

Conversation

tbrkollar commented Jan 26, 2026

Uh oh!

gemini-code-assist Bot commented Jan 26, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist Bot Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

paulr34 left a comment

Choose a reason for hiding this comment

Uh oh!

brdandu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

brdandu Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

tbrkollar Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

tbrkollar Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

brdandu Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

tbrkollar Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants