Skip to content

add: container test plugin for pytest#17372

Merged
reubeno merged 1 commit into
4.0from
sudipta/4.0/add-container-runtime-test-system
Jun 4, 2026
Merged

add: container test plugin for pytest#17372
reubeno merged 1 commit into
4.0from
sudipta/4.0/add-container-runtime-test-system

Conversation

@realsdx
Copy link
Copy Markdown
Contributor

@realsdx realsdx commented May 20, 2026

Add a pytest based validation framework AzureLinux's OCI container images, integrated with azldev image test

Test it locally by (make sure to install and start podman first):

azldev image build container-base && azldev image test container-base

Comment thread base/images/tests/README.md
Comment thread base/images/tests/cases/runtime/container-base/test_nginx/test_nginx.py Outdated
Comment thread base/images/tests/README.md Outdated
Comment thread base/images/tests/README.md
@realsdx realsdx changed the title [WIP] add: container test plugin for pytest add: container test plugin for pytest May 26, 2026
@realsdx realsdx marked this pull request as ready for review May 26, 2026 08:37
Copilot AI review requested due to automatic review settings May 26, 2026 08:37
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends Azure Linux’s image-test framework to support runtime validation of OCI container images (in addition to existing static/offline filesystem checks) and wires the new runtime suite into azldev image test.

Changes:

  • Added --image-ref support (mutually exclusive with --image-path) and new pytest markers for runtime/container and Dockerfile-based tests.
  • Introduced a python-on-whales/Podman-based container runtime helper plus session/function fixtures to run podman exec-style tests.
  • Restructured test layout into cases/static/ vs cases/runtime/ and added an images.toml runtime test-suite for container-base variants.

Reviewed changes

Copilot reviewed 10 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
base/images/tests/utils/pytest_plugin.py Adds --image-ref, new markers, and collection-time auto-marking for runtime/static test layout.
base/images/tests/utils/container_runtime.py New Podman/python-on-whales orchestration helpers (load/pull/build/run/exec/cleanup).
base/images/tests/conftest.py Adds runtime container fixtures (podman_client, running_container, container_exec, etc.) and supports --image-ref.
base/images/tests/README.md Updates documentation to reflect static vs runtime suites and new CLI options/fixtures.
base/images/tests/pyproject.toml Adds python-on-whales dependency for runtime container orchestration.
base/images/tests/cases/static/test_os_release.py New shared static validation for /etc/os-release.
base/images/tests/cases/static/test_packages.py New shared static RPM/package presence/absence checks (capability-gated).
base/images/tests/cases/static/vm-base/test_kernel.py Adds VM kernel/modules checks and Fedora 43 CONFIG_LSM assertion.
base/images/tests/cases/static/vm-base/test_partitions.py Adds VM partition layout checks (root + EFI).
base/images/tests/cases/static/container-base/test_container.py Adds container static validation (no kernel modules).
base/images/tests/cases/runtime/container-base/test_basic.py Adds basic runtime exec tests (shell + DNS).
base/images/tests/cases/runtime/container-base/test_nginx/* Adds Dockerfile-based runtime example for nginx validation.
base/images/images.toml Wires runtime-container-tests into container-base images and splits suite test-paths into cases/static/ and cases/runtime/.
Comments suppressed due to low confidence (1)

base/images/tests/utils/pytest_plugin.py:156

  • pytest_configure derives image_type from --image-type, --capabilities, or --image-path, but it doesn’t consider --image-ref. When running with --image-ref and no --capabilities/--image-type, image_type stays None and check_tools(when=None) ends up requiring all tools (VM + container), which breaks the intended registry-ref workflow. Treat --image-ref as implying image_type="container" during this early preflight derivation.
    image_type = config.getoption("--image-type", default=None)
    if image_type is None:
        caps = parse_capabilities(config.getoption("--capabilities", default=None))
        if caps:
            image_type = derive_image_type_from_capabilities(caps)

Comment thread base/images/tests/utils/container_runtime.py Outdated
Comment thread base/images/tests/utils/container_runtime.py
Comment thread base/images/tests/utils/container_runtime.py
Comment thread base/images/tests/cases/runtime/container-base/test_nginx/Dockerfile Outdated
Comment thread base/images/tests/cases/runtime/container-base/test_basic.py Outdated
Comment thread base/images/tests/utils/pytest_plugin.py
Comment thread base/images/tests/conftest.py
Comment thread base/images/tests/cases/runtime/container-base/test_nginx/test_nginx.py Outdated
Comment thread base/images/tests/utils/container_runtime.py
Comment thread base/images/tests/README.md
Comment thread base/images/tests/utils/pytest_plugin.py
Comment thread base/images/tests/utils/container_runtime.py
Comment thread base/images/tests/utils/container_runtime.py Outdated
Comment thread base/images/tests/pyproject.toml
@realsdx realsdx requested a review from reubeno June 3, 2026 13:42
Add Podman-backed pytest fixtures for resolving container images, creating per-test containers, and executing commands against them.

Move existing image tests under static cases and add runtime coverage for container-base, including nginx custom image validation. Document running the container test suite and register the runtime capability in image metadata.
@realsdx realsdx force-pushed the sudipta/4.0/add-container-runtime-test-system branch from 5565cb9 to 5cb2ca8 Compare June 4, 2026 15:23
@reubeno reubeno merged commit e851705 into 4.0 Jun 4, 2026
12 checks passed
@reubeno reubeno deleted the sudipta/4.0/add-container-runtime-test-system branch June 4, 2026 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants