Add a robust E2E testing suite for Kmesh Locality Load Balancing.

[sarafarajnasardi]

Review

What type of PR is this?
/kind enhancement

---

What this PR does / why we need it

This PR introduces a robust End-to-End (E2E) testing suite for Kmesh Locality Load Balancing.

As Kmesh moves toward feature parity with Istio’s advanced traffic management, it is critical to ensure that traffic correctly respects administrative locality boundaries (region, zone, and subzone) to achieve optimal performance and high availability.

---

Technical Implementation Details

1. Dynamic Multi-Zone Topology Simulation

• Automatically detects available nodes in the cluster.
• Simulates a multi-zone environment by dynamically applying:
  • topology.kubernetes.io/region
  • topology.kubernetes.io/zone
  • topology.kubernetes.io/subzone
• Enables validation of locality-aware routing even on single-cluster environments (e.g., Kind).

2. Advanced Pod Scheduling & Taint Handling

• Introduces a patchDeployment utility using kubectl patch to dynamically apply:
  • nodeSelector
  • tolerations
• Ensures client and server workloads are pinned to specific nodes to accurately verify intra-zone and cross-zone traffic behavior.
• Explicitly handles control-plane node taints, allowing the test to run on standard Kind clusters without manual intervention.

3. Verification Scenarios

• Locality Priority
  Verifies that a client in region1/zone1 sends traffic exclusively to the server subset in the same zone when localityLbSetting is enabled in the DestinationRule.

• Failover Logic
  Simulates a zone-level outage by scaling the local server subset to zero and verifies that Kmesh seamlessly fails over to a remote subset in zone2.

4. Environmental Stability & Reliability

• Uses t.Cleanup to restore node labels to their original state after test execution.
• Employs optimized timeout and retry logic via the Istio integration framework’s retry package to handle asynchronous resource readiness.

---

Which issue(s) this PR fixes

Fixes #1146

---

Special notes for your reviewer

• The test suite leverages istio.io/istio/pkg/test/framework and echo components to maintain consistency with existing Kmesh integration tests.
• Service definitions were aligned to port 5000 (standard echo instance port), while retaining port 80 as the external entry point to ensure compatibility with Istio’s internal routing logic.

---

Does this PR introduce a user-facing change?

NONE

---

release-note

Add refined End-to-End tests for Locality Load Balancing and Failover, including support for region/zone/subzone topology simulation.

[kmesh-bot]

Welcome @sarafarajnasardi! It looks like this is your first PR to kmesh-net/kmesh 🎉

[gemini-code-assist]

Summary of Changes

Hello @sarafarajnasardi, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a comprehensive End-to-End testing suite for Kmesh's Locality Load Balancing feature. The suite is designed to validate Kmesh's ability to respect administrative locality boundaries (region, zone, subzone) for traffic routing, ensuring optimal performance and high availability. It includes dynamic topology simulation, advanced pod scheduling, and verifies both locality priority and failover mechanisms.

Highlights

• Dynamic Multi-Zone Topology Simulation: The test suite dynamically labels Kubernetes nodes with topology.kubernetes.io/region, zone, and subzone to simulate a multi-zone environment, enabling locality-aware routing validation even on single-cluster setups like Kind.
• Advanced Pod Scheduling & Taint Handling: A patchDeployment utility is introduced to apply nodeSelector and tolerations to client and server workloads, ensuring they are pinned to specific nodes for accurate verification of intra-zone and cross-zone traffic behavior. It also explicitly handles control-plane node taints.
• Locality Priority Verification: The test confirms that a client in a specific zone (e.g., zone1) exclusively sends traffic to a server subset within the same zone when localityLbSetting is enabled in the DestinationRule.
• Failover Logic Validation: The suite simulates a zone-level outage by scaling down local server subsets and verifies that Kmesh correctly fails over traffic to a remote server subset in another zone.
• Environmental Stability & Reliability: Node labels applied for topology simulation are automatically restored to their original state using t.Cleanup after test execution, ensuring a clean test environment. Optimized timeout and retry logic is also employed.
• Integration Framework Consistency: The tests leverage istio.io/istio/pkg/test/framework and echo components, maintaining consistency with existing Kmesh integration tests, and align service definitions to port 5000 internally with port 80 as the external entry point.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• test/e2e/locality_lb_test.go
  • Added a new E2E test file (locality_lb_test.go) to validate Kmesh's locality-aware load balancing and failover capabilities.
  • Implemented dynamic node labeling to simulate multi-zone Kubernetes topologies for testing.
  • Introduced helper functions (setNodeLabel, removeNodeLabel, patchDeployment) for managing node labels and deployment affinities/tolerations.
  • Developed two main test scenarios: "locality priority" to ensure traffic stays within the local zone, and "failover" to verify traffic redirection upon local zone outage.
  • Integrated with Istio's test framework components (echo, retry) for robust and asynchronous testing.

Activity

• A new End-to-End test suite for Kmesh Locality Load Balancing has been added.
• The pull request addresses and fixes issue Add e2e for locality aware lb  #1146.
• The change is categorized as an enhancement (/kind enhancement).
• The release note highlights the addition of refined E2E tests for Locality Load Balancing and Failover, including support for region/zone/subzone topology simulation.
• No specific review comments or further activity details are provided in the context.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

---

Nodes labeled with care, Traffic flows to local zone, Failover, then peace.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive End-to-End test suite for Kmesh Locality Load Balancing, which is a valuable addition for ensuring feature correctness. The test implementation is well-structured and covers key scenarios like locality-aware routing and failover. My feedback focuses on improving the implementation by replacing shell executions of kubectl with direct Kubernetes API calls. This will enhance the tests' robustness, security, and maintainability by removing external dependencies and adhering to best practices for Kubernetes client interaction in Go.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 39.59%. Comparing base (a98ad40) to head (2ff702c).
⚠️ Report is 32 commits behind head on main.
see 3 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8e1228a...2ff702c. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[hzxuzhonghu]

I think you donot need to create subset here

[hzxuzhonghu]

IC, you created two deployment

[kmesh-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hzxuzhonghu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• test/OWNERS [hzxuzhonghu]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment