Skip to content

[CNV-63822] Test plan out-out of role aggregation #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rlobillo wants to merge 3 commits into
base: main
Choose a base branch
from
Open

[CNV-63822] Test plan out-out of role aggregation #28

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f5eb6dc
[CNV-63822] Test plan out-out of role aggregation
rlobillo Feb 6, 2026
6128ab1
Address comments and aligning to the newer version of the STP template
rlobillo Feb 26, 2026
7663f49
Address second round of comments and update status
rlobillo Mar 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
173 changes: 173 additions & 0 deletions stps/sig-iuo/CNV-63822-role-aggregation-opt-out.md
View file
Open in desktop
Copy link
Copy Markdown
Contributor

@rnetser rnetser Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you think having this nesting under stps/sig-iuo/CNV-63822-role-aggregation-opt-out is needed or just have the stp under iuo with the feature name?

Copy link
Copy Markdown
Author

@rlobillo rlobillo Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved.

Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
# Openshift-virtualization-tests Test plan

## **Role Aggregation Opt-Out - Quality Engineering Plan**

### **Metadata & Tracking**

| Field | Details |
|:-----------------------|:--------------------------------------------------------|
| **Enhancement(s)** | [Kubevirt VEP](https://github.com/kubevirt/enhancements/issues/160) |
| **Feature in Jira** | [CNV-50792](https://issues.redhat.com/browse/CNV-50792) |
| **Jira Tracking** | [CNV-63822](https://issues.redhat.com/browse/CNV-63822) |
| **QE Owner(s)** | Ramon Lobillo (@rlobillo), Alex Barker (@albarker-rh) |
| **Owning SIG** | sig-iuo (Install, Upgrade, Operators) |
| **Participating SIGs** | sig-ui |
| **Current Status** | Draft |

---

### **I. Motivation and Requirements Review (QE Review Guidelines)**

#### **1. Requirement & User Story Review Checklist**

| Check | Done | Details/Notes | Comments |
|:---------------------------------------|:-----|:--------------------------------------------------------------------------------|:------------------------------------------------------|
| **Review Requirements** | [x] | As a cluster-admin I want to decide which users will have access to the virtualization in the cluster. Not all project-admins should have this access but only the eligible ones | Per CNV-50792 feature request |
| **Understand Value** | [x] | A cluster-admin wants to control which users has access to view/create/edit openshift virtualization resources on a given namespace | Per CNV-50792 feature request |
| **Customer Use Cases** | [x] | * multi-tenant clusters|different namespaces are used to allow different workloads and prevent unallowed usage of workload that the tenant is not eligible to us|
| | | * Resources usage control|cluster admin wants to get a request to allow a specific user to create VMs and Storage|
Comment on lines +27 to +28
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Escape pipe characters in the “Customer Use Cases” table cell.

Line 28 and Line 29 use raw | inside a markdown table cell, which breaks column parsing. Use \| or split into <br> bullets to keep the table render intact.

🛠️ Proposed fix 
-| **Customer Use Cases**                 | [x]  | * multi-tenant clusters|different namespaces are used to allow different workloads and prevent unallowed usage of workload that the tenant is not eligible to us|
-|                                        |      | * Resources usage control|cluster admin wants to get a request to allow a specific user to create VMs and Storage|
+| **Customer Use Cases**                 | [x]  | - multi-tenant clusters: different namespaces are used to allow different workloads and prevent unauthorized usage |
+|                                        |      | - resources usage control: cluster admin can approve requests for specific users to create VMs and storage |
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
| **Customer Use Cases** | [x] | * multi-tenant clusters|different namespaces are used to allow different workloads and prevent unallowed usage of workload that the tenant is not eligible to us|
| | | * Resources usage control|cluster admin wants to get a request to allow a specific user to create VMs and Storage|
| **Customer Use Cases** | [x] | - multi-tenant clusters: different namespaces are used to allow different workloads and prevent unauthorized usage |
| | | - resources usage control: cluster admin can approve requests for specific users to create VMs and storage |
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@stps/sig-iuo/CNV-63822-role-aggregation-opt-out/stp.md` around lines 28 - 29,
The markdown table under the "Customer Use Cases" cell contains raw pipe
characters that break table parsing; edit the table cell in stp.md (the row
beginning with "**Customer Use Cases**") and either escape each raw pipe as "\|"
or replace the inline pipes with HTML line breaks (e.g., use "<br>" and bullets)
so the two bullets "multi-tenant clusters|different namespaces..." and
"Resources usage control|cluster admin..." render as intended and do not split
columns.

| **Testability** | [ ] | Blocked until HCO API modification is available; need to confirm field name and API | Cannot implement tests without actual implementation |
| **Acceptance Criteria** | [x] | (1) Config disables aggregation, (2) Users blocked without RoleBinding, (3) RoleBinding grants access | Defined in CNV-63822 epic |
| **Non-Functional Requirements (NFRs)** | [x] | Security (RBAC hardening), Backward Compatibility (default unchanged) | Upgrade and docs coverage required |

#### **2. Technology and Design Review**

| Check | Done | Details/Notes | Comments |
|:---------------------------------|:-----|:--------------------------------------------------------------------------------|:------------------------------------------------------|
| **Developer Handoff/QE Kickoff** | [x] |||
| **Technology Challenges** | [x] | N/A ||
| **Test Environment Needs** | [x] | Standard OCP + CNV cluster with HTPasswd IdP for unprivileged user testing | No special hardware required |
| **API Extensions** | [ ] | hco spec field TBD; | Cannot finalize until feature is completely implemented |
| **Topology Considerations** | [x] | Feature is cluster-scoped (KubeVirt CR level), topology-independent | Works on all topologies (standard, SNO, compact) |



### **II. Software Test Plan (STP)**

#### **1. Scope of Testing**
Copy link
Copy Markdown

@hmeir hmeir Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please align with the new STP template https://github.com/RedHatQE/openshift-virtualization-tests-design-docs/blob/main/stps/stp-template/stp.md

In general for the document

Copy link
Copy Markdown
Contributor

@rnetser rnetser Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Copy Markdown
Author

@rlobillo rlobillo Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.


**Testing Goals**
- [P0] Verify opt-out role aggregation can be enabled via hyperconvergeds.hco.kubevirt.io config
- [P0] Unprivileged users cannot access kubevirt resources without explicit RoleBinding when feature is enabled
- [P0] Explicit RoleBindings (admin, edit, view) grant access correctly
- [P0] Verify opt-out role aggregation can be disabled via hyperconvergeds.hco.kubevirt.io config

**Backward compatibility Goals**

- [P0] Default behavior (role aggregation enabled) remains unchanged
- [P0] Default behaviour is preserved across CNV z-stream upgrades

**Out of Scope:**

| Out-of-Scope Item | Rationale | PM/ Lead Agreement |
|:----------------------------------------------------------|:------------------------------------------------------------------------------|:-------------------|
| Testing OpenShift RBAC infrastructure itself | OCP responsibility | [ ] TBD |
| Testing all rules within kubevirt.io roles | kubevirt.io:{admin,edit,view} clusterroles contains rules that are not affected by this feature | [ ] TBD |
| External IdP compatibility (LDAP, Active Directory) | RBAC is IdP-agnostic; HTPasswd testing validates core logic | [ ] TBD |
| Multi-tenant cluster scale testing (100+ users) | RBAC overhead negligible; functional correctness sufficient at smaller scale | [ ] TBD |
| Testing kubevirt.io:migrate role aggregation | Already covered on tier2 regression testing: [test_migration_rights.py](https://github.com/RedHatQE/openshift-virtualization-tests/blob/main/tests/virt/cluster/migration_and_maintenance/rbac_hardening/test_migration_rights.py) | [ ] TBD |


#### **2. Test Strategy**

| Item | Description | Applicable (Y/N or N/A) | Comments |
|:-------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------|:---------|
| Functional Testing | Validates that the feature works according to specified requirements and user stories | Y | Core focus: verify RBAC opt-out behaviour |
| Automation Testing | Ensures test cases are automated for continuous integration and regression coverage | Y | |
| Performance Testing | Validates feature performance meets requirements (latency, throughput, resource usage) | N/A | |
| Security Testing | Verifies security requirements, RBAC, authentication, authorization, and vulnerability scanning | Y | Feature is a security enhancement |
| Usability Testing | Validates user experience, UI/UX consistency, and accessibility requirements. Does the feature require UI? If so, ensure the UI aligns with the requirements | Y | [CNV-80935](https://issues.redhat.com/browse/CNV-80935) |
| Compatibility Testing | Ensures feature works across supported platforms, versions, and configurations | Y | default behaviour will not change |
| Regression Testing | Verifies that new changes do not break existing functionality | Y | |
| Upgrade Testing | Validates upgrade paths from previous versions, data migration, and configuration preservation | Y | |
| Backward Compatibility Testing | Ensures feature maintains compatibility with previous API versions and configurations | Y | |
| Dependencies | Dependent on deliverables from other components/products? Identify what is tested by which team. | N | |
| Cross Integrations | Does the feature affect other features/require testing by other components? Identify what is tested by which team. | Y | UI |
| Monitoring | Does the feature require metrics and/or alerts? | N | |
| Cloud Testing | Does the feature require multi-cloud platform testing? Consider cloud-specific features. | N | |


#### **3. Test Environment**

| Environment Component | Configuration | Specification Examples |
|:----------------------------------------------|:-------------------------------|:-------------------------------------------------------------------|
| **Cluster Topology** | Standard or SNO | Feature works on all topologies; multi-node preferred |
| **OCP & OpenShift Virtualization Version(s)** | OCP 4.22 with CNV 4.22 | Target version where feature introduced |
| **CPU Virtualization** | N/A | Not relevant for RBAC testing |
| **Compute Resources** | Standard cluster resources | Minimum per worker: 4 vCPUs, 16GB RAM |
| **Special Hardware** | N/A | No special hardware required |
| **Storage** | Any RWX storage class | ocs-storagecluster-ceph-rbd-virtualization |
| **Network** | Default (OVN-Kubernetes) | No special network requirements |
| **Required Operators** | OpenShift Virtualization | Standard CNV installation |
| **Platform** | Any supported platform | |
| **Special Configurations** | HTPasswd identity provider | REQUIRED: Must have HTPasswd IdP with unprivileged user |

#### **3.1. Testing Tools & Frameworks**

| Category | Tools/Frameworks |
|:-------------------|:-----------------|
| **Test Framework** | |
| **CI/CD** | |
| **Other Tools** | |
Comment thread
coderabbitai[bot] marked this conversation as resolved.

#### **4. Entry Criteria**

- [X] KubeVirt PR #16350 **merged**
- [ ] HCO downstream implementation **complete** (field integrated into HCO CR)
- [ ] Requirements and design documents approved
- [ ] Developer Handoff/QE Kickoff meeting completed

#### **5. Risks**

| Risk Category | Specific Risk for This Feature | Mitigation Strategy | Status |
|:---------------------|:--------------------------------------------------------|:--------------------------------------------------------|:-----------|
Copy link
Copy Markdown
Contributor

@rnetser rnetser Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about ui development and functioanlity - is there a risk? iiuc they were not in the loop

Copy link
Copy Markdown
Author

@rlobillo rlobillo Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, added.

| IU adaptations | The feature description mentions IU changes that are still pending to concrete | Discussion started and will be tracked with IU team | [x] Active |
| Test Coverage | Cannot exhaustively test all RBAC role combinations | Test critical paths (all 4 roles); focus on acceptance criteria | [ ] |
Copy link
Copy Markdown
Contributor

@rnetser rnetser Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what will not be tested? get an ack from lead/PM

Copy link
Copy Markdown
Author

@rlobillo rlobillo Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are not testing all the actions that a role allows, just checking that role itself is correctly aggregated/not aggregated through a meaningful operation that the role allows.

@orenc1 is that ok from your view?

Copy link
Copy Markdown
Contributor

@rnetser rnetser Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i may be missing something here - what actions are there for an edit role beside editing + cannot delete for example?

Copy link
Copy Markdown
Author

@rlobillo rlobillo Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I don't get exactly what you mean. Could you please elaborate?

| Dependencies | HCO downstream implementation | Track progress and coordinate with HCO team | [x] Active |
| Untestable Aspects | Limited to HTPasswd; cannot test LDAP/AD/OAuth | RBAC logic is IdP-agnostic; HTPasswd validation sufficient | [ ] |


#### **8. Known Limitations**
Copy link
Copy Markdown
Contributor

@rnetser rnetser Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do not forget that you need to get an ack from pm/lead on all limitations

Copy link
Copy Markdown
Author

@rlobillo rlobillo Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack


No limitations.

---

### **III. Test Scenarios & Traceability**

| Requirement ID | Requirement Summary | Test Scenario(s) | Test Type(s) | Priority |
Comment thread
rnetser marked this conversation as resolved.
Copy link
Copy Markdown
Contributor

@rnetser rnetser Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about virtctl options like create and expose? can a user without view permissions run virtclt fslist , virtctl credentials etc? what about storage options like virtctl vmexport?
what about other resources in the ns? anything that a user can exploit?
what about view-edit? can a user have edit but not view? what will happen in the ui?
edit - "start/stop VM" - i assume a user with edit permissions can run any action on the vm (e.g pause/ reset etc), right? should it be tested?
how does the config affect vm migration?
based on the feature - you need to test console access for view role
based on the use cases hidden in the feature (this is why i mentioned the epic is not groomed) - you need to support giving access to specific vms in a ns to user1 and to other vms to user2 ("kubevirt.io:view is automatically aggregated to the view CR, the user would have access to all the consoles in the project")

Copy link
Copy Markdown
Author

@rlobillo rlobillo Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am replying on the email thread to address this concern.

Copy link
Copy Markdown
Contributor

@rnetser rnetser Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for visibility - i am not famialir with the roles enough to answer what is needed; please check what is needed

Copy link
Copy Markdown
Author

@rlobillo rlobillo Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please refer to the list of tests below.

|:-------------------------|:-----------------------------------------------------|:------------------------------------------------------------------------|:-----------------|:---------|
| KubeVirt PR #16350 | `RoleAggregationStrategy config should keep aggregate labels when RoleAggregationStrategy is nil` | | tier1 automation | P0 |
| KubeVirt PR #16350 | `RoleAggregationStrategy configuration should keep aggregate labels when RoleAggregationStrategy is AggregateToDefault` | | tier1 automation | P0 |
| KubeVirt PR #16350 | `RoleAggregationStrategy configuration should create ClusterRole without aggregate labels when RoleAggregationStrategy is Manual` | | tier1 auto | P0 |
| KubeVirt PR #16350 | `RoleAggregationStrategy configuration should remove aggregate labels from existing ClusterRole when strategy changes to Manual` | | tier1 auto | P0 |
| CNV-63822 | As an admin, I can enable the feature via config in hyperconverged CR | Verify config persists once enabled | tier2 automation | P0 |
| | As an admin, I can enable the feature so an unprivileged user with admin role on a namespace cannot perform kubevirt.io:admin actions | Verify user gets ForbiddenError | tier2 automation | P0 |
| | As an admin, I can enable the feature so an unprivileged user with edit role on a namespace cannot perform kubevirt.io:edit actions | Verify user gets ForbiddenError | tier2 automation | P0 |
| | As an admin, I can enable the feature so an unprivileged user with view role on a namespace cannot perform kubevirt.io:view actions | Verify user getse ForbiddenError | tier2 automation | P0 |
| | As an admin, I can specifically add kubevirt.io:admin permissions to an unprivileged user in a namespace when feature is enabled| Verify user can perform action | tier2 automation | P0 |
| | As an admin, I can specifically add kubevirt.io:edit permissions to an unprivileged user in a namespace when feature is enabled| Verify user can perform action | tier2 automation | P0 |
| | As an admin, I can specifically add kubevirt.io:view permissions to an unprivileged user in a namespace when feature is enabled| Verify user can perform action | tier2 automation | P0 |
| | As an admin, I can disable the feature via config in hyperconverged CR | Verify config persists once disabled and unprivileged user with admin role in a namespace can perform kubevirt:admin action | tier2 automation | P0 |
Comment thread
coderabbitai[bot] marked this conversation as resolved.

---

### **IV. Sign-off and Approval**

This Software Test Plan requires approval from the following stakeholders:

* **Reviewers:**
- [QE Lead / @rnester]
- [sig-iuo representative / @orenc1 @hmeir @OhadRevah @albarker-rh]
- [sig-ui representative / @gouyang]

* **Approvers:**
- [QE Manager / @kmajcher-rh @fabiand]
- [Product Manager / Ronen Sde-Or]

**Review Status:**
- [X] Draft complete
- [ ] QE team reviewed
- [ ] Dev/Arch reviewed
- [ ] PM approved
- [ ] Ready for implementation