diff --git a/stps/sig-storage/storage_multiarch_stp.md b/stps/sig-storage/storage_multiarch_stp.md new file mode 100644 index 0000000..f586c67 --- /dev/null +++ b/stps/sig-storage/storage_multiarch_stp.md @@ -0,0 +1,247 @@ +# Openshift-virtualization-tests Test plan + +## CDI support for heterogeneous multi-arch clusters - Quality Engineering Plan + +### **Metadata & Tracking** + +| Field | Details | +|:---------------------- |:-------------------------------------------------------------------------------------------------------------------------------- | +| **Enhancement(s)** | [dic-on-heterogeneous-cluster](https://github.com/kubevirt/enhancements/tree/main/veps/sig-storage/dic-on-heterogeneous-cluster) | +| **Feature in Jira** | [VIRTSTRAT-494](https://issues.redhat.com/browse/VIRTSTRAT-494) | +| **Jira Tracking** | [CNV-73892](https://issues.redhat.com/browse/CNV-73892) | +| **QE Owner(s)** | Yan Du | +| **Owning SIG** | sig-storage | +| **Participating SIGs** | sig-infra, sig-storage, sig-virt | +| **Current Status** | Draft | + +--- + +**Document Conventions:** +- **CDI**: Containerized Data Importer - responsible for importing VM disk images into Kubernetes +- **DIC**: DataImportCron - automates periodic import of VM images +- **SSP**: Scheduling, Scale and Performance Operator - manages VM templates and common resources +- **HCO**: Hyperconverged Cluster Operator - orchestrates OpenShift Virtualization deployment +- **IUO**: Integration/Update Operator - handles feature gate management and upgrades +- **VEP**: Virtualization Enhancement Proposal - design document for new features +- **Pull Method**: Mechanism for importing images (Pod-based or Node-based) + +### **I. Motivation and Requirements Review (QE Review Guidelines)** + +This section documents the mandatory QE review process. The goal is to understand the feature's value, +technology, and testability before formal test planning. + +#### **1. Requirement & User Story Review Checklist** + + + +| Check | Done | Details/Notes | Comments | +|:-------------------------------------- |:---- |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |:-------- | +| **Review Requirements** | [x] | Reviewed the relevant requirements | | +| **Understand Value** | [x] | CDI changes to support arch-specific registry imports for both pull modes | | +| **Customer Use Cases** | [x] | VM creation with architecture-specific DataSources | | +| **Testability** | [x] | Requirements are testable and unambiguous | | +| **Acceptance Criteria** | [x] | When importing an image from a [OCI Image Index](https://specs.opencontainers.org/image-spec/image-index/), I want to optionally specify a `platform` field to influence which image variant is selected from the multi-platform manifest. | | +| **Non-Functional Requirements (NFRs)** | [x] | NFRs (Performance, Usability, Monitoring, Scalability) are out of scope for this test plan | | + +#### **2. Technology and Design Review** + + + +| Check | Done | Details/Notes | Comments | +|:-------------------------------- |:---- |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |:-------- | +| **Developer Handoff/QE Kickoff** | [x] | Discuss within team for the feature | | +| **Technology Challenges** | [x] | Can use HA cluster, but should be verified on Multiarch cluster which is available only for 12 hours | | +| **Test Environment Needs** | [x] | MultiArch cluster | | +| **API Extensions** | [x] | **HCO**: `status.nodeInfo` (controlPlaneArchitectures, workloadsArchitectures), `status.dataImportCronTemplates` (originalSupportedArchitectures, conditions)
**SSP**: `enableMultipleArchitectures`, `cluster` fields (workloadArchitectures, controlPlaneArchitectures)
**CDI**: `platform.architecture` field in `DataVolumeSourceRegistry`, arch-specific `DataSource` (`-`), legacy `DataSource` redirects to arch-specific one | | + +### **II. Software Test Plan (STP)** + +This STP serves as the **overall roadmap for testing**, detailing the scope, approach, resources, and schedule. + +#### **1. Scope of Testing** + + + +**Testing Goals** + + + + + +**Functional Goals**: + +- **[P0]** Verify multi-arch image matching architecture with pull method Pod + +- **[P0]** Verify multi-arch image matching architecture with pull method Node + +- **[P0]** Validate DataSource pointing to another DataSource + +**Out of Scope (Testing Scope Exclusions)** + +| Non-Goal | Rationale | PM/ Lead Agreement | +|:------------------------------ |:-------------------------------------------------------------------------------------------------- |:------------------ | +| Performance Testing | Out of scope for this test plan | [ ] Name/Date | +| Security Testing | Out of scope for this test plan | [ ] Name/Date | +| Usability testing | Should be done by UI team | [ ] Name/Date | +| Backward Compatibility Testing | VM creation using architecture-specific DataSources and legacy DataSource covered by SSP test plan | [ ] Name/Date | + +#### **2. Test Strategy** + + + + | Item | Description | Applicable (Y/N or N/A) | Comments | + |:------------------------------ |:------------------------------------------------------------------------------------------------------------------------------------------------------------ |:----------------------- |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | Functional Testing | Validates that the new storage api | Y | | + | Automation Testing | Ensures test cases are automated | Y | | + | Performance Testing | Validates feature performance meets requirements (latency, throughput, resource usage) | N/A | | + | Security Testing | Verifies security requirements, RBAC, authentication, authorization, and vulnerability scanning | N/A | | + | Usability Testing | Validates user experience, UI/UX consistency, and accessibility requirements. Does the feature require UI? If so, ensure the UI aligns with the requirements | N/A | Should be done by UI team | + | Compatibility Testing | Ensures feature works across supported platforms, versions, and configurations | N/A | | + | Regression Testing | Verifies that new api changes do not break existing functionality | Y | | + | Upgrade Testing | Validates upgrade paths from previous versions, data migration, and configuration preservation | Y | | + | Backward Compatibility Testing | Ensures feature maintains compatibility with previous API versions and configurations | Y | | + | Dependencies | Dependent on deliverables from other components/products? Identify what is tested by which team. | N/A | | + | Cross Integrations | Does the feature affect other features/require testing by other components? Identify what is tested by which team. | Y | **IUO**: HCO node architecture tracking (`status.nodeInfo`), FG activation/propagation, new metrics & alerts, upgrade
**SSP**: Templates creation & utilization, new SSP API (`enableMultipleArchitectures`, `cluster` fields)
**Storage**: CDI-importer architecture selection, legacy `DataSource` backward compatibility, new CDI `platform` API
**Virt**: VM scheduling to correct architecture nodes, VM migration between same-arch nodes, upgrade | + | Monitoring | Does the feature require metrics and/or alerts? | N/A | | + | Cloud Testing | Does the feature require multi-cloud platform testing? Consider cloud-specific features. | N/A | not related to cloud | + +#### **3. Test Environment** + + + + | Environment Component | Configuration | Comments | + |:--------------------------------------------- |:------------------------ |:------------------------------------------------------------- | + | **Cluster Topology** | MultiArch cluster | 3 control-plane and 3-4 worker nodes | + | **OCP & OpenShift Virtualization Version(s)** | OCP 4.21, CNV-4.21 | OCP 4.21 and OpenShift Virtualization 4.21 | + | **CPU Virtualization** | Multi-arch cluster | 3 amd64 control-plane, 2 amd64 workers, and 1-2 arm64 workers | + | **Compute Resources** | N/A | No special compute requirements | + | **Special Hardware** | N/A | No special hardware required | + | **Storage** | io2-csi storage class | AWS EBS io2 CSI driver | + | **Network** | OVN-Kubernetes (default) | No special network requirements | + | **Required Operators** | N/A | N/A | + | **Platform** | AWS | ARM64 workers available on AWS | + | **Special Configurations** | N/A | No special configurations required | + +#### **3.1. Testing Tools & Frameworks** + + + + | Category | Tools/Frameworks | + |:------------------ |:----------------- | + | **Test Framework** | MultiArch cluster | + | **CI/CD** | | + | **Other Tools** | | + +#### **4. Entry Criteria** + + The following conditions must be met before testing can begin: + +- [x] VEP [dic-on-heterogeneous-cluster](https://github.com/kubevirt/enhancements/tree/main/veps/sig-storage/dic-on-heterogeneous-cluster) is approved and merged + +- [x] CDI support arch-specific registry imports + +- [ ] Test environment (MultiArch cluster) can be set up and configured + +- [x] Feature gate enableMultiArchBootImageImport can be enabled + + + +#### **5. Risks** + + + + | Risk Category | Specific Risk for This Feature | Mitigation Strategy | Status | + |:-------------------- |:------------------------------------------------------- |:---------------------------------------------------------------- |:------ | + | Timeline/Schedule | N/A | No timeline risks identified for this feature | [ ] | + | Test Coverage | N/A | Standard test coverage applies, no special risks | [ ] | + | Test Environment | [CNV-76482](https://issues.redhat.com/browse/CNV-76482) | Tracked in CNV-76482; coordinate testing schedules early | [ ] | + | Untestable Aspects | N/A | All aspects are testable within multi-arch environment | [ ] | + | Resource Constraints | N/A | Standard resource requirements, no constraints | [ ] | + | Dependencies | N/A | No external dependencies blocking testing | [ ] | + | Known Bugs | [CNV-75762](https://issues.redhat.com/browse/CNV-75762) | Tracked in CNV-75762; monitor bug status for fixes | [ ] | + +#### **6. Known Limitations** + + + + None + +--- + +### **III. Test Scenarios & Traceability** + + + + + +| Requirement ID | Requirement Summary | Test Scenario(s) | Tier | Priority | +|:-------------- |:------------------------------------------------------------------------------- |:--------------------------------------------------------------------------------------------------------------------------------------------- |:------ |:-------- | +| TC-01 | Pull multi-arch image matching architecture with pull method Pod | Verify the import succeeded with spec.registry.pullMethod: pod and spec.registry.platform.architecture: arm64 | Tier 1 | P1 | +| TC-02 | Pull failed when multi-arch image with absent architecture with pull method Pod | Verify the import failed with spec.registry.pullMethod: pod and spec.registry.platform.architecture: absent | Tier 1 | P0 | +| TC-03 | node selector for multi-arch image architecture with pull method Node | Verify the import pod label has "Unschedulable" condition with spec.registry.pullMethod: node and spec.registry.platform.architecture: absent | Tier 1 | P0 | +| TC-04 | DataSource pointing to another DataSource | Verify the import succeeded when define the DataSource to another DataSource | Tier 1 | P1 | +| TC-05 | Cross-architecture VM cloning | Clone amd64 VM, verify clone uses amd64 DataSource. Attempt cross-arch clone, verify appropriate error | Tier 2 | P1 | + +--- + +### **IV. Sign-off and Approval** + +This Software Test Plan requires approval from the following stakeholders: + +* **Reviewers:** + - + - +* **Approvers:** + - + -