test(e2e): add SC-01..SC-62 live-server suite + CI

Wire the Phase 5 E2E rollout for the PVE cell of pve-python — mirrors the
TypeScript reference at pve-ts/e2e/ in idiomatic Python. 22 tests pass
green against the proxmox-docker pve-test container, 1 skips when cgroup
v2 is unavailable, 1 xfails on a documented QemuCreateVmRequest.to_dict
generator gap that auto-promotes when fixed upstream.

- e2e/ skeleton with conftest fixtures (creds, pve token client, node,
  session cleanup) and capability gates (kvm/cgroupv2/network) as
  importable markers
- e2e/helpers/ for credentials, fixtures, polling, ISO download
  (SHA-256 verified), and raw multipart upload + content listing
  (SDK upload helper can't carry bytes; diridx response mistypes inner
  items)
- e2e/test_*.py covers version (SC-01), auth (SC-10..14), authz
  (SC-20..22), CRUD (SC-30..34), ISO upload (SC-35), errors
  (SC-40..42), types (SC-50..52, including int64 verification), and
  VM/CT lifecycle (SC-60..62)
- pyproject.toml gains [project.optional-dependencies] test group
  (pytest, pytest-timeout, requests for the multipart fallback)
- docker-compose.yml at repo root for local runs against
  ghcr.io/client-api/proxmox-docker/pve-test:9.2
- .github/workflows/e2e.yml runs client-api/proxmox-docker-action@v1
  and pytest -v with the action's PROXMOX_* env exports
- .openapi-generator-ignore preserves e2e/, the workflow, and
  docker-compose.yml across regeneration
- tests/e2e/README.md redirects to the new layout

Author: bencurio

.github/workflows/e2e.yml

@@ -0,0 +1,44 @@
+name: e2e
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  pve:
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.13'
+          cache: pip
+
+      - name: Install package + test deps
+        run: pip install -e .[test]
+
+      - name: Authenticate to GHCR
+        uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Start PVE test container
+        id: proxmox
+        uses: client-api/proxmox-docker-action@v1
+        with:
+          product: pve
+          tag: '9.2'
+          enable-kvm: auto
+
+      - name: Run E2E tests
+        run: pytest e2e/ -v --tb=short
+        env:
+          PROXMOX_INSECURE: '1'
+          PROXMOX_KVM_AVAILABLE: ${{ steps.proxmox.outputs.kvm-available }}
+          PROXMOX_CGROUPV2_AVAILABLE: ${{ steps.proxmox.outputs.cgroupv2-available }}

.openapi-generator-ignore

@@ -1,23 +1,21 @@
 # OpenAPI Generator Ignore
 # Generated by openapi-generator https://github.com/openapitools/openapi-generator
-
+#
 # Use this file to prevent files from being overwritten by the generator.
-# The patterns follow closely to .gitignore or .dockerignore.
+# The patterns follow .gitignore syntax. Pipeline-tooling in pve-openapi
+# (scripts/sdk-sync.ts) also reads this file before deciding what to delete.
 
-# As an example, the C# client generator defines ApiClient.cs.
-# You can make changes and tell OpenAPI Generator to ignore just this file by uncommenting the following line:
-#ApiClient.cs
+# Hand-written E2E suite — never regenerate.
+e2e/
+e2e/**
 
-# You can match any string of characters against a directory, file or extension with a single asterisk (*):
-#foo/*/qux
-# The above matches foo/bar/qux and foo/baz/qux, but not foo/bar/baz/qux
+# CI workflow we own; generator only manages ci.yml + publish.yml.
+.github/workflows/e2e.yml
 
-# You can recursively match patterns against a directory, file or extension with a double asterisk (**):
-#foo/**/qux
-# This matches foo/bar/qux, foo/baz/qux, and foo/bar/baz/qux
+# Local docker harness for the E2E suite.
+docker-compose.yml
 
-# You can also negate patterns with an exclamation (!).
-# For example, you can ignore all files in a docs folder with the file extension .md:
-#docs/*.md
-# Then explicitly reverse the ignore rule for a single file:
-#!docs/README.md
+# Note: pyproject.toml is overwritten by every sync. The
+# [project.optional-dependencies] block must be re-added via the upstream
+# Mustache template (templates/_shared/pyproject.toml.mustache in pve-openapi)
+# — tracked as a follow-up; not ignored here.

docker-compose.yml

@@ -0,0 +1,21 @@
+services:
+  pve-test:
+    image: ghcr.io/client-api/proxmox-docker/pve-test:9.2
+    container_name: pve-test
+    privileged: true
+    devices:
+      - /dev/fuse
+      - /dev/kvm
+    tmpfs:
+      - /tmp
+      - /run
+      - /run/lock
+    ports:
+      - "8006:8006"
+    healthcheck:
+      # /version requires auth on PVE 9.x — accept any HTTP response (200/401) as "API is up".
+      test: ["CMD-SHELL", "curl -sk -o /dev/null -w '%{http_code}' https://localhost:8006/api2/json/version | grep -qE '^(200|401)$'"]
+      interval: 5s
+      timeout: 5s
+      retries: 24
+      start_period: 10s

e2e/README.md

@@ -0,0 +1,114 @@
+# E2E tests for `clientapi_pve`
+
+Live-server pytest suite. Runs against a real Proxmox VE instance — by default the
+`ghcr.io/client-api/proxmox-docker/pve-test` container, either spun up locally
+via `docker compose up -d` or in CI via
+[`client-api/proxmox-docker-action@v1`](https://github.com/client-api/proxmox-docker-action).
+
+## Quick start (local)
+
+```bash
+docker compose up -d
+sleep 20  # wait for healthcheck
+
+export PROXMOX_URL=https://localhost:8006
+export PROXMOX_USER=root@pam
+export PROXMOX_PASSWORD=proxmox123
+# Read the joined token header from the container:
+export PROXMOX_TOKEN_HEADER_VALUE="$(docker exec pve-test cat /run/credentials.json | jq -r .token_header_value)"
+export PROXMOX_INSECURE=1
+export PROXMOX_KVM_AVAILABLE=$(test -e /dev/kvm && echo true || echo false)
+export PROXMOX_CGROUPV2_AVAILABLE=$(test -d /sys/fs/cgroup/cgroup.controllers && echo true || echo false)
+
+pip install -e .[test]
+pytest e2e/ -v
+```
+
+## Environment contract
+
+| Var | Source | Notes |
+|---|---|---|
+| `PROXMOX_URL` | `proxmox-docker-action` | e.g. `https://localhost:8006` (no `/api2/json`) |
+| `PROXMOX_USER` | action | typically `root@pam` |
+| `PROXMOX_PASSWORD` | action | password ticket auth (SC-10..11, SC-14) |
+| `PROXMOX_TOKEN_HEADER_VALUE` | action | whole `PVEAPIToken=root@pam!test=<uuid>` string |
+| `PROXMOX_TOKEN_VALUE` | action | UUID half only (rarely needed) |
+| `PROXMOX_INSECURE` | local | set to `1` to skip TLS verification (self-signed dev cert) |
+| `PROXMOX_KVM_AVAILABLE` | action output | `true`/`false`; gates SC-60, SC-62 |
+| `PROXMOX_CGROUPV2_AVAILABLE` | action output | `true`/`false`; gates SC-61 |
+| `PROXMOX_NO_NETWORK` | manual | `1` to skip network-egress tests (SC-35, SC-62) |
+
+> Never reconstruct `PROXMOX_TOKEN_HEADER_VALUE` by hand — the Perl family
+> (PVE, PMG) joins with `=`, the Rust family (PBS, PDM) with `:`. The
+> container pre-joins it correctly.
+
+## Capability gates
+
+Tests that need kernel features import the marker symbol, never an inline env check:
+
+```python
+from e2e.conftest import requires_kvm
+
+@requires_kvm
+def test_vm_lifecycle(pve, node):
+    ...
+```
+
+When the gate is closed the test is **skipped**, not failed.
+
+## Fixture convention
+
+All entities created during the suite are named `e2e-…` (users, storages,
+ACLs) and live in the VM-ID range `101..199`. `cleanup_e2e()` runs at session
+start and end. VM 100 (`tiny-test`) and CT 200 (`tiny-ct`) are pre-seeded by
+the container and are never touched.
+
+## Scenario index (SC-01 … SC-62)
+
+| File | Scenarios |
+|---|---|
+| `test_version.py` | SC-01 |
+| `test_auth.py` | SC-10 … SC-14 |
+| `test_authz.py` | SC-20 … SC-22 |
+| `test_crud.py` | SC-30 … SC-34 |
+| `test_iso_upload.py` | SC-35 |
+| `test_errors.py` | SC-40 … SC-42 |
+| `test_types.py` | SC-50 … SC-52 |
+| `test_vm_lifecycle.py` | SC-60 (kvm-gated) |
+| `test_ct_lifecycle.py` | SC-61 (cgroupv2-gated) |
+| `test_vm_cdrom.py` | SC-62 (kvm+network-gated) |
+
+## Known SDK gaps surfaced by this suite
+
+These are real generator gaps the live tests uncovered; each has a focused
+workaround in `e2e/helpers/` so the rest of the suite stays clean, and the
+SC-NN that exercises it stays in the suite so it will auto-promote when the
+upstream template is fixed.
+
+1. **Ticket cookie format.** `Configuration.auth_settings` emits
+   `Cookie: <ticket>` for `PVEAuthCookie`. PVE expects
+   `Cookie: PVEAuthCookie=<ticket>`. Workaround in
+   `e2e/helpers/clients.py::ticket_client` pre-joins the prefix.
+2. **`NodesStorageApi.upload` carries no file body.** The OpenAPI spec models
+   the upload field as `tmpfilename` (a server-side path reference), so the
+   generated SDK can only send metadata. Workaround: raw multipart POST in
+   `e2e/helpers/upload.py::upload_iso`.
+3. **`NodesStorageDiridxResponse.data` types inner items as
+   `AccessGetAccessResponseDataInner`.** The generator picked the wrong inner
+   model, so `volid` deserializes as empty. Workaround:
+   `e2e/helpers/upload.py::list_storage_content` parses the raw JSON.
+4. **`QemuCreateVmRequest.to_dict()` references undefined indexed-family
+   fields** (`ide0`, `net0`, …). The model exposes collapsed `ides`/`nets`
+   maps but the serializer reaches for individual numbered properties.
+   SC-62 is marked `xfail` until the template is fixed.
+5. **POST endpoints with optional request models drop the body entirely.**
+   `vm_start` / `vm_stop` (etc.) send no body when no request model is
+   supplied; PVE returns 500 "malformed JSON". Tests always pass an empty
+   request model (`QemuVmStartRequest()`) instead of relying on the default.
+
+## Downstream cells
+
+`pbs-py`, `pmg-py`, `pdm-py` follow the same shape — copy `e2e/`, the
+pyproject test group, `docker-compose.yml`, and `.github/workflows/e2e.yml`,
+then swap product-specific bits (image tag, port, token separator). PMG skips
+SC-12/13 (no API tokens). PDM uses `/api2/extjs` in raw HTTP paths.

e2e/__init__.py

@@ -0,0 +1,64 @@
+"""Shared pytest fixtures and capability gates for the E2E suite."""
+from __future__ import annotations
+
+from typing import Iterator
+
+import pytest
+
+from clientapi_pve import Configuration, Pve
+from e2e.helpers.capability_gate import (
+    cgroupv2_available,
+    kvm_available,
+    network_available,
+)
+from e2e.helpers.credentials import Credentials, MissingCredentialError
+from e2e.helpers.fixtures import cleanup_e2e, first_node
+
+# Declared once here so test files import a symbol rather than re-checking env vars.
+requires_kvm = pytest.mark.skipif(not kvm_available(), reason="KVM not available")
+requires_cgroupv2 = pytest.mark.skipif(
+    not cgroupv2_available(), reason="cgroup v2 not available"
+)
+requires_network = pytest.mark.skipif(
+    not network_available(), reason="network not available (PROXMOX_NO_NETWORK=1)"
+)
+
+
+@pytest.fixture(scope="session")
+def creds() -> Credentials:
+    try:
+        return Credentials.from_env()
+    except MissingCredentialError as exc:
+        pytest.skip(str(exc))
+
+
+def _token_client(creds: Credentials) -> Pve:
+    cfg = Configuration(host=f"{creds.url}/api2/json")
+    cfg.verify_ssl = not creds.insecure
+    cfg.api_key["PVEApiToken"] = creds.token_header_value
+    return Pve(cfg)
+
+
+@pytest.fixture(scope="session")
+def pve(creds: Credentials) -> Pve:
+    """Default client: API-token auth (no CSRF dance)."""
+    return _token_client(creds)
+
+
+@pytest.fixture(scope="session")
+def node(pve: Pve) -> str:
+    return first_node(pve)
+
+
+@pytest.fixture(scope="session", autouse=True)
+def _session_cleanup(creds: Credentials, pve: Pve) -> Iterator[None]:
+    """Wipe any e2e-* leftovers before and after the suite."""
+    try:
+        node = first_node(pve)
+    except Exception:
+        node = ""
+    if node:
+        cleanup_e2e(pve, node)
+    yield
+    if node:
+        cleanup_e2e(pve, node)

e2e/conftest.py

@@ -0,0 +1,27 @@
+"""Capability gates for kernel features the test container may or may not have.
+
+The proxmox-docker-action exposes `kvm-available` and `cgroupv2-available` as
+step outputs; the workflow mirrors them into env vars. Locally, set them by hand.
+"""
+from __future__ import annotations
+
+import os
+
+
+def _truthy(name: str) -> bool:
+    return os.environ.get(name, "").lower() in ("1", "true", "yes")
+
+
+def kvm_available() -> bool:
+    """True when /dev/kvm is usable inside the container (PVE can boot VMs)."""
+    return _truthy("PROXMOX_KVM_AVAILABLE")
+
+
+def cgroupv2_available() -> bool:
+    """True when the host exposes cgroup v2 unified hierarchy (LXC requires it)."""
+    return _truthy("PROXMOX_CGROUPV2_AVAILABLE")
+
+
+def network_available() -> bool:
+    """True unless PROXMOX_NO_NETWORK is set (air-gapped runners)."""
+    return os.environ.get("PROXMOX_NO_NETWORK", "") != "1"

e2e/helpers/__init__.py

@@ -0,0 +1,70 @@
+"""Client factories for the two auth modes PVE supports.
+
+Token clients use the Authorization header and don't need a CSRF dance.
+Ticket clients require a paired ticket cookie + CSRFPreventionToken header
+for non-GET requests; SDK already wires both when both api_key entries are set.
+"""
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+from clientapi_pve import Configuration, Pve
+
+if TYPE_CHECKING:
+    from e2e.helpers.credentials import Credentials
+
+
+def token_client(creds: "Credentials") -> Pve:
+    cfg = Configuration(host=f"{creds.url}/api2/json")
+    cfg.verify_ssl = not creds.insecure
+    cfg.api_key["PVEApiToken"] = creds.token_header_value
+    return Pve(cfg)
+
+
+def ticket_client(
+    creds: "Credentials",
+    *,
+    ticket: str,
+    csrf: str | None = None,
+) -> Pve:
+    """Build a Pve client authenticated with a ticket cookie and (optionally) a CSRF header.
+
+    Omit `csrf` to deliberately simulate the SC-14 "missing CSRF header on write" case.
+    """
+    cfg = Configuration(host=f"{creds.url}/api2/json")
+    cfg.verify_ssl = not creds.insecure
+    # SDK bug workaround: the auth_settings cookie path emits `Cookie: <value>`
+    # but PVE expects `Cookie: PVEAuthCookie=<value>`. Pre-join here.
+    cfg.api_key["PVEAuthCookie"] = f"PVEAuthCookie={ticket}"
+    if csrf is not None:
+        cfg.api_key["CSRFPreventionToken"] = csrf
+    return Pve(cfg)
+
+
+def issue_ticket(creds: "Credentials", *, password: str | None = None) -> Pve:
+    """Log in with username+password and return a Pve client wired for ticket auth.
+
+    Raises ApiException(401) when the password is wrong (SC-11).
+    """
+    from clientapi_pve.models.access_ticket_create_ticket_request import (
+        AccessTicketCreateTicketRequest,
+    )
+
+    anon = Configuration(host=f"{creds.url}/api2/json")
+    anon.verify_ssl = not creds.insecure
+    bootstrap = Pve(anon)
+
+    response = bootstrap.accessTicket.create_ticket(
+        AccessTicketCreateTicketRequest(
+            username=creds.user,
+            password=password if password is not None else creds.password,
+        )
+    )
+    data = response.data
+    if data is None or not data.ticket:
+        raise RuntimeError(f"ticket login returned no ticket: {response!r}")
+    return ticket_client(
+        creds,
+        ticket=data.ticket,
+        csrf=data.csrf_prevention_token,
+    )