Skip to content

Conversation

@asolntsev
Copy link
Contributor

@asolntsev asolntsev commented Jan 23, 2026

User description

Description

  1. Run Gradle tests as well on GA
  2. Upload test reports, so that developers can downloads them during next 2 weeks and investigate the reason of failure.

Types of changes

  • Change to the site (I have double-checked the Netlify deployment, and my changes look good)
  • Code example added (and I also added the example to all translated languages)
  • Improved translation
  • Added new translation (and I also added a notice to each document missing translation)

Checklist

  • I have read the contributing document.
  • [] I have used hugo to render the site/docs locally and I am sure it works.

PR Type

Enhancement


Description

  • Add Gradle test execution to Java examples CI pipeline

  • Upload test reports as artifacts for 14 days on failure

  • Update test file paths to write reports to standard directories


Diagram Walkthrough

flowchart LR
  A["Java CI Workflow"] --> B["Run Maven Tests"]
  A --> C["Run Gradle Tests"]
  B --> D["Test Execution"]
  C --> D
  D --> E{"Build Failed?"}
  E -->|Yes| F["Upload Test Reports"]
  F --> G["14-day Retention"]
Loading

File Walkthrough

Relevant files
Enhancement
java-examples.yml
Add Gradle tests and test report uploads                                 

.github/workflows/java-examples.yml

  • Add ./gradlew test command to stable release test job
  • Add ./gradlew with version parameter to nightly Linux/macOS test job
  • Add ./gradlew with version parameter to nightly Windows test job
  • Add new upload-artifact step to capture test reports on failure with
    14-day retention
+13/-0   
Configuration changes
SavingTest.java
Update PDF output path for test reports                                   

examples/java/src/test/java/dev/selenium/interactions/SavingTest.java

  • Update PDF output path from root directory to target/surefire-reports/
    directory
+1/-1     
LoggingTest.java
Update log file path for test reports                                       

examples/java/src/test/java/dev/selenium/troubleshooting/LoggingTest.java

  • Update log file path from root directory to target/surefire-reports/
    directory
  • Ensures log files are written to standard Maven test report location
+2/-2     

@netlify
Copy link

netlify bot commented Jan 23, 2026

👷 Deploy request for selenium-dev pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit 7fb68b6

@qodo-code-review
Copy link
Contributor

qodo-code-review bot commented Jan 23, 2026

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
Artifact data exposure

Description: Uploading Maven/Gradle test reports and logs as GitHub Actions artifacts on failure (e.g.,
examples/java/target/surefire-reports, examples/java/build/reports,
examples/java/build/test-results) may unintentionally expose sensitive information such as
environment details, URLs, stack traces, or secrets accidentally printed during tests, so
the artifact contents should be reviewed/sanitized before upload or access restricted.
java-examples.yml [111-120]

Referred Code
- name: Upload test report
  uses: actions/upload-artifact@v6
  if: failure()
  with:
    name: test-report-${{matrix.os}}-${{matrix.release}}
    retention-days: 14
    path: |
      examples/java/target/surefire-reports
      examples/java/build/reports
      examples/java/build/test-results
Ticket Compliance
🎫 No ticket provided
  • Create ticket/issue
Codebase Duplication Compliance
Codebase context is not defined

Follow the guide to enable codebase context checks.

Custom Compliance
🟢
Generic: Comprehensive Audit Trails

Objective: To create a detailed and reliable record of critical system actions for security analysis
and compliance.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Meaningful Naming and Self-Documenting Code

Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Error Handling

Objective: To prevent the leakage of sensitive system information through error messages while
providing sufficient detail for internal debugging.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Security-First Input Validation and Data Handling

Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent
vulnerabilities

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Robust Error Handling and Edge Case Management

Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation

Status:
Missing directory guard: The new write path assumes target/surefire-reports exists and can fail with an IOException
if the directory is absent or unwritable in some CI/test runners.

Referred Code
    String content = ((RemoteWebDriver) driver).print(new PrintOptions()).getContent();
    byte[] bytes = Base64.getDecoder().decode(content);
    Files.write(Paths.get("target", "surefire-reports", "selenium.pdf"), bytes);
}

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Logging Practices

Objective: To ensure logs are useful for debugging and auditing without exposing sensitive
information like PII, PHI, or cardholder data.

Status:
Potential sensitive logs: Writing detailed Selenium/WebDriver logs to target/surefire-reports/selenium.xml may
capture environment URLs, capabilities, or other sensitive context depending on runtime
configuration and should be reviewed/redacted if needed.

Referred Code
Handler handler = new FileHandler("target/surefire-reports/selenium.xml");
logger.addHandler(handler);

Logger.getLogger(RemoteWebDriver.class.getName()).setLevel(Level.FINEST);
Logger.getLogger(SeleniumManager.class.getName()).setLevel(Level.SEVERE);

Logger localLogger = Logger.getLogger(this.getClass().getName());
localLogger.warning("this is a warning");
localLogger.info("this is useful information");
localLogger.fine("this is detailed debug information");

byte[] bytes = Files.readAllBytes(Paths.get("target/surefire-reports/selenium.xml"));
String fileContent = new String(bytes);

Learn more about managing compliance generic rules or creating your own custom rules

  • Update
Compliance status legend 🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

@qodo-code-review
Copy link
Contributor

qodo-code-review bot commented Jan 23, 2026

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Fix Windows Gradle wrapper call

On Windows runners, use .\gradlew.bat instead of ./gradlew to correctly invoke
the Gradle wrapper from PowerShell.

.github/workflows/java-examples.yml [109]

-./gradlew "-Dselenium.version=$new_version" test --tests 'dev.selenium.*UsingSeleniumTest'
+.\gradlew.bat "-Dselenium.version=$new_version" test --tests 'dev.selenium.*UsingSeleniumTest'
  • Apply / Chat
Suggestion importance[1-10]: 9

__

Why: The suggestion correctly identifies that the ./gradlew command will fail on Windows runners, which use PowerShell. The proposed change to .\\gradlew.bat is essential for the Windows CI job to execute the Gradle tests successfully.

High
Ensure parent directories exist before writing

Programmatically create the parent directories for the output PDF file before
writing to it. This prevents a NoSuchFileException when running tests with build
tools other than Maven, such as Gradle.

examples/java/src/test/java/dev/selenium/interactions/SavingTest.java [21-23]

 String content = ((RemoteWebDriver) driver).print(new PrintOptions()).getContent();
 byte[] bytes = Base64.getDecoder().decode(content);
-Files.write(Paths.get("target", "surefire-reports", "selenium.pdf"), bytes);
+java.nio.file.Path pdf = Paths.get("target", "surefire-reports", "selenium.pdf");
+Files.createDirectories(pdf.getParent());
+Files.write(pdf, bytes);
  • Apply / Chat
Suggestion importance[1-10]: 8

__

Why: The suggestion correctly identifies that the hardcoded path target/surefire-reports will cause test failures when run with Gradle, which is being added in this PR. The proposed fix to create directories before writing is accurate and necessary for the PR's changes to work correctly.

Medium
Create log directory before file handler

Create the target/surefire-reports directory before instantiating FileHandler to
prevent IOException when running tests with build tools that do not create this
directory by default.

examples/java/src/test/java/dev/selenium/troubleshooting/LoggingTest.java [37]

+Files.createDirectories(Paths.get("target", "surefire-reports"));
 Handler handler = new FileHandler("target/surefire-reports/selenium.xml");
  • Apply / Chat
Suggestion importance[1-10]: 8

__

Why: The suggestion correctly identifies that the hardcoded path target/surefire-reports will cause test failures when run with Gradle, which is being added in this PR. The proposed fix to create the directory before initializing the FileHandler is correct and necessary.

Medium
Fix race condition in logging test

To prevent a race condition, ensure all log messages are written before
assertions by flushing and closing the FileHandler. Use a try-finally block to
guarantee the handler is also removed, preventing side effects in other tests.

examples/java/src/test/java/dev/selenium/troubleshooting/LoggingTest.java [30-54]

 public void logging() throws IOException {
     Logger logger = Logger.getLogger("");
     logger.setLevel(Level.FINE);
     Arrays.stream(logger.getHandlers()).forEach(handler -> {
         handler.setLevel(Level.FINE);
     });
 
     Handler handler = new FileHandler("target/surefire-reports/selenium.xml");
-    logger.addHandler(handler);
+    try {
+        logger.addHandler(handler);
 
-    Logger.getLogger(RemoteWebDriver.class.getName()).setLevel(Level.FINEST);
-    Logger.getLogger(SeleniumManager.class.getName()).setLevel(Level.SEVERE);
+        Logger.getLogger(RemoteWebDriver.class.getName()).setLevel(Level.FINEST);
+        Logger.getLogger(SeleniumManager.class.getName()).setLevel(Level.SEVERE);
 
-    Logger localLogger = Logger.getLogger(this.getClass().getName());
-    localLogger.warning("this is a warning");
-    localLogger.info("this is useful information");
-    localLogger.fine("this is detailed debug information");
+        Logger localLogger = Logger.getLogger(this.getClass().getName());
+        localLogger.warning("this is a warning");
+        localLogger.info("this is useful information");
+        localLogger.fine("this is detailed debug information");
 
-    byte[] bytes = Files.readAllBytes(Paths.get("target/surefire-reports/selenium.xml"));
-    String fileContent = new String(bytes);
+        handler.flush();
+        handler.close();
 
-    Assertions.assertTrue(fileContent.contains("this is a warning"));
-    Assertions.assertTrue(fileContent.contains("this is useful information"));
-    Assertions.assertTrue(fileContent.contains("this is detailed debug information"));
+        byte[] bytes = Files.readAllBytes(Paths.get("target/surefire-reports/selenium.xml"));
+        String fileContent = new String(bytes);
+
+        Assertions.assertTrue(fileContent.contains("this is a warning"));
+        Assertions.assertTrue(fileContent.contains("this is useful information"));
+        Assertions.assertTrue(fileContent.contains("this is detailed debug information"));
+    } finally {
+        logger.removeHandler(handler);
+    }
 }

[To ensure code accuracy, apply this suggestion manually]

Suggestion importance[1-10]: 7

__

Why: The suggestion correctly identifies a potential race condition that could lead to flaky tests and proposes a robust solution by flushing and closing the FileHandler and cleaning it up in a finally block. This significantly improves test reliability.

Medium
  • Update

until now, we executed only Maven tests. As a result, build.gradle was outdated.
@asolntsev asolntsev force-pushed the ci/java-upload-test-reports branch from 245b2f4 to 236a728 Compare January 23, 2026 12:54
Copy link
Member

@diemol diemol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to see them in a different job.

(probably not needed to run gradle tests with nightly selenium)
@asolntsev
Copy link
Contributor Author

I would prefer to see them in a different job.

done

@asolntsev asolntsev requested a review from diemol January 23, 2026 14:21
@harsha509 harsha509 merged commit e7f4460 into SeleniumHQ:trunk Jan 26, 2026
7 checks passed
@asolntsev asolntsev deleted the ci/java-upload-test-reports branch January 26, 2026 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants