From 5ac623bf0e1ed3f34a06bc86481115a268c46e77 Mon Sep 17 00:00:00 2001 From: Marcus Burghardt Date: Thu, 21 May 2026 11:55:12 +0200 Subject: [PATCH 1/2] docs: add OpenSpec proposal for dev testing environment Add devcontainer-based testing environment proposal with design, specs, and tasks. This change will provide maintainers with a one-command Fedora environment for interactive CLI testing during PR reviews using GitHub Codespaces, DevPod, or VS Code Dev Containers. Assisted-by: OpenCode (claude-opus-4-6) Signed-off-by: Marcus Burghardt --- .../dev-testing-environment/.openspec.yaml | 3 + .../changes/dev-testing-environment/design.md | 207 ++++++++++++++++++ .../dev-testing-environment/proposal.md | 49 +++++ .../specs/dev-testing-environment/spec.md | 150 +++++++++++++ .../changes/dev-testing-environment/tasks.md | 83 +++++++ 5 files changed, 492 insertions(+) create mode 100644 openspec/changes/dev-testing-environment/.openspec.yaml create mode 100644 openspec/changes/dev-testing-environment/design.md create mode 100644 openspec/changes/dev-testing-environment/proposal.md create mode 100644 openspec/changes/dev-testing-environment/specs/dev-testing-environment/spec.md create mode 100644 openspec/changes/dev-testing-environment/tasks.md diff --git a/openspec/changes/dev-testing-environment/.openspec.yaml b/openspec/changes/dev-testing-environment/.openspec.yaml new file mode 100644 index 00000000..d39be0bd --- /dev/null +++ b/openspec/changes/dev-testing-environment/.openspec.yaml @@ -0,0 +1,3 @@ +schema: spec-driven +created: 2026-05-21 +status: specifying diff --git a/openspec/changes/dev-testing-environment/design.md b/openspec/changes/dev-testing-environment/design.md new file mode 100644 index 00000000..c3d92c07 --- /dev/null +++ b/openspec/changes/dev-testing-environment/design.md @@ -0,0 +1,207 @@ +## Context + +Maintainers reviewing PRs that affect CLI UX (command output, formatting, +error messages) need to manually test `complyctl` commands in a realistic +environment. Today, this requires cloning three repositories (complyctl, +complytime-providers, complytime-demos), setting up Vagrant with libvirt, +running Ansible playbooks, and SSH-ing into a Fedora VM. This friction +discourages thorough manual testing. + +The cross-repo integration test infrastructure already provides a proven +automated setup: building both repos, installing snappy/ampel, starting +a mock OCI registry with Gemara test content, and running `complyctl get`, +`generate`, and `scan` commands. This design adapts that same setup for +interactive human use via the devcontainer standard. + +## Goals / Non-Goals + +**Goals:** + +- Provide a one-command path from "reviewing a PR" to "testing CLI commands + in a Fedora environment." +- Support GitHub Codespaces, DevPod, and VS Code Dev Containers through + the open devcontainer.json standard. +- Reuse existing test infrastructure (mock OCI registry, Gemara test + fixtures, workspace configuration) rather than creating new test content. +- Document the workflow clearly for maintainers and contributors who may + not be familiar with devcontainers. + +**Non-Goals:** + +- Full system-level OpenSCAP scanning (container constraints are accepted). +- Replacing the complytime-demos Vagrant setup (it remains available for + comprehensive testing). +- Publishing or maintaining a pre-built container image in a registry. + Both repos build from a Containerfile at startup time to avoid registry + maintenance overhead. +- Dedicated onboarding workflows for external contributors (the + environment and documentation are accessible to contributors, but the + primary testing path and examples target maintainer PR review + scenarios). + +## Decisions + +### D1: Use the devcontainer.json open standard + +**Decision**: Use the devcontainer.json specification (containers.dev) rather +than a custom Makefile target, Vagrant, or proprietary tooling. + +**Alternatives considered**: + +- *Makefile target with raw podman/docker*: Simpler but no IDE integration, + no Codespaces support, no standard ecosystem. +- *Vagrant + libvirt*: Real VM but heavy setup, already exists in + complytime-demos, doesn't integrate with GitHub PRs. +- *Packit + Testing Farm only*: Good for automated RPM testing but designed + for automated test runs, not interactive sessions. + +**Rationale**: devcontainer.json is vendor-neutral and supported by GitHub +Codespaces (one-click from PR), DevPod (open-source local/cloud), and +VS Code Dev Containers. One configuration file, multiple runtimes. +Maintainers choose their preferred tool. + +### D2: Fedora base image from Fedora's public registry + +**Decision**: Use `registry.fedoraproject.org/fedora:43` as the base image. + +**Rationale**: OpenSCAP provider requires `openscap-scanner` and +`scap-security-guide`, which are Fedora/RHEL packages. Fedora's official +container registry is free, public, and doesn't require authentication. + +### D3: No custom container registry + +**Decision**: Build the Containerfile at devcontainer startup time. Do not +publish or maintain a pre-built image. + +**Alternatives considered**: + +- *Publish to ghcr.io*: Faster startup in complytime-providers (pull vs + build), but adds a CI workflow, image versioning, and another thing to + maintain. The Containerfile is small (Fedora + dnf install), so build + time is acceptable. + +**Rationale**: Avoids the complexity of maintaining a container image +pipeline. The Containerfile only installs a few system packages; the +heavier work (Go builds, tool installs) happens in the postCreateCommand +regardless. + +### D4: Reuse cross-repo test infrastructure for workspace setup + +**Decision**: The devcontainer post-create script reuses the same test +fixtures and mock OCI registry from `tests/cross-repo/testdata/` and +`cmd/mock-oci-registry/`. + +**Rationale**: The cross-repo integration test already proves this setup +works. The mock registry embeds Gemara catalogs and policies, provides +an OCI-compliant endpoint, and requires no external services. Reusing it +means zero new test content to create or maintain. + +### D5: Install snappy and ampel via go install + +**Decision**: Install external tools using `go install` rather than +downloading pre-built binaries or using GitHub Actions. + +**Alternatives considered**: + +- *carabiner-dev/actions/install*: Only works in GitHub Actions, not in + a local devcontainer. +- *Download release binaries*: Platform-specific, requires curl/wget and + architecture detection. + +**Rationale**: `go install` works identically in CI and local containers. +Go is already installed in the container. The install paths are documented +in the ampel-provider README and the quickstart spec. + +### D6: Pin tool versions, clone providers from main + +**Decision**: Install snappy and ampel at pinned version tags (e.g., +`@v0.2.4`, `@v1.2.1`) rather than `@latest`. Clone +`complytime-providers` from `main` (unversioned) and build from source. + +**Alternatives considered**: + +- *`@latest` for all tools*: Simpler but produces non-reproducible + environments. Two developers on different days get different versions. + The constitution prohibits floating tags for container images, and the + same principle applies to tool installs. +- *Download release binaries for providers*: Providers have not been + released to a package registry yet. Building from source is the only + available path. +- *Pin providers to a commit SHA*: Would provide reproducibility but + creates a maintenance burden. The providers repo is actively developed + and the devcontainer should track `main` for forward compatibility. + +**Rationale**: Pinned versions for snappy/ampel provide reproducibility +and match CI's approach (the cross-repo workflow uses pinned action +SHAs). Cloning providers from `main` is an accepted trade-off: providers +are same-org, under maintainer trust, and the devcontainer's value is +testing the latest code. The cloned commit SHA is logged for +auditability. + +### D7: Canonical source in complyctl, thin consumer in complytime-providers + +**Decision**: The full devcontainer configuration (Containerfile, +post-create script, documentation) lives in complyctl. complytime-providers +adds its own `.devcontainer/` with a similar Containerfile and a mirrored +post-create script that builds providers locally and clones complyctl +from main. + +**Rationale**: Mirrors the established pattern for cross-repo integration +tests: the canonical script lives in complyctl, complytime-providers +consumes it. complyctl is the user-facing CLI and the natural home for +the testing environment definition. + +## Risks / Trade-offs + +- **[Container limitations for OpenSCAP]** → Accepted. OpenSCAP system + scans are limited in containers. The devcontainer targets CLI UX testing + using the Ampel provider with the mock registry, which works fully in + a container. Full OpenSCAP testing remains available via complytime-demos. + +- **[Containerfile duplication across repos]** → Minimal. The Containerfile + is ~10 lines of dnf install. Each repo's post-create logic is different + (which binary to build from local source vs. clone from main), so some + divergence is unavoidable and acceptable. + +- **[Build time at startup]** → Building Go binaries and installing tools + via `go install` adds 2-5 minutes to devcontainer startup on a 4-core + machine. The devcontainer.json SHOULD specify `hostRequirements` with + a minimum of 4 cores and 16GB RAM. The default 2-core Codespace may + be slow or hit memory limits during Go builds. Codespaces prebuild + could reduce startup time in the future if needed. + +- **[GITHUB_TOKEN requirement]** → snappy requires a GitHub token to query + the GitHub API for branch protection rules. Codespaces can store this as + a secret; DevPod and local VS Code require manual export. Documentation + must make this clear. + +- **[mock-oci-registry as background process]** → The post-create script + starts the mock registry in the background. If it crashes, `complyctl get` + will fail with a connection error. The script includes a readiness check. + Restarting it manually is documented. Note: Codespace suspend/resume + kills background processes. After resuming, the mock registry must be + restarted manually. Documentation must cover this. + +- **[complytime-providers@main clone without commit pinning]** → + Accepted trade-off. The devcontainer clones `main` of a same-org repo + under maintainer trust. Unlike CI (which uses `actions/checkout` with + SHA pinning), the devcontainer prioritizes testing the latest provider + code. The post-create script logs the cloned commit SHA for + auditability. If `main` is broken, the post-create script fails with + a clear error identifying the upstream dependency issue. This differs + from CI's pinned approach because the devcontainer's purpose is + interactive testing of the latest code, not reproducible builds. + +- **[GITHUB_TOKEN least-privilege]** → The post-create script follows + the same least-privilege pattern as `cross_repo_integration_test.sh`: + captures the token, unsets it from the environment, and only passes + it to `complyctl` subprocesses that require it. This prevents `go + install`, `git clone`, and `make build` from inheriting the token + unnecessarily. + +- **[Fedora base image tag pinning]** → The Containerfile uses + `fedora:43` (a specific version tag, not `latest`). While digest + pinning would provide full reproducibility, it creates a maintenance + burden of tracking Fedora image updates. Tag-level pinning is an + intentional trade-off for a dev-only environment that benefits from + receiving security updates automatically. diff --git a/openspec/changes/dev-testing-environment/proposal.md b/openspec/changes/dev-testing-environment/proposal.md new file mode 100644 index 00000000..db5b4e14 --- /dev/null +++ b/openspec/changes/dev-testing-environment/proposal.md @@ -0,0 +1,49 @@ +## Why + +Manual testing of CLI UX changes requires maintainers to set up a complex +multi-repository environment (complyctl + complytime-providers + complytime-demos) +with Vagrant, Ansible, and libvirt. This friction discourages thorough manual +testing during PR reviews, especially for CLI output and user experience changes +that automated tests cannot fully validate. As the project receives more external +contributions, maintainers need a one-command path to a ready-to-test Fedora +environment for any PR. + +## What Changes + +- Add a `.devcontainer/` configuration to complyctl providing a Fedora-based + development and testing environment with all dependencies pre-installed. +- The devcontainer uses the open `devcontainer.json` standard, compatible with + GitHub Codespaces, DevPod, and VS Code Dev Containers. +- A `postCreateCommand` script automates the full setup: builds complyctl and + mock-oci-registry from the local source, installs snappy and ampel via + `go install` at pinned version tags, clones and builds + complytime-providers from `main`, copies provider binaries to the + discovery path, and configures a test workspace with Gemara content + via the mock OCI registry. +- Add documentation in `docs/` explaining how maintainers and contributors + use the devcontainer for PR review testing and interactive CLI exploration, + covering Codespaces, DevPod, and local VS Code workflows, with a practical + command reference and troubleshooting section. +- Update `README.md` with a reference to the new documentation. + +## Capabilities + +### New Capabilities + +- `dev-testing-environment`: Devcontainer configuration and setup automation + providing a one-command Fedora environment for interactive CLI testing + during PR reviews. + +## Impact + +- **New files**: `.devcontainer/Containerfile`, `.devcontainer/devcontainer.json`, + `.devcontainer/scripts/post-create.sh`, `docs/dev-testing-environment.md` +- **Modified files**: `README.md` (add link to new docs) +- **Dependencies**: Uses `registry.fedoraproject.org/fedora:43` as the base + container image. Installs `openscap-scanner`, `scap-security-guide`, `curl`, + and `jq` via dnf. Installs `snappy` (v0.2.4) and `ampel` (v1.2.1) via + `go install` at pinned versions from `carabiner-dev`. Clones + `complytime-providers` from GitHub at build time. +- **No changes to existing code**: This is purely additive infrastructure. +- **Downstream**: complytime-providers will add its own thin devcontainer + configuration in a follow-up change, consuming the same pattern. diff --git a/openspec/changes/dev-testing-environment/specs/dev-testing-environment/spec.md b/openspec/changes/dev-testing-environment/specs/dev-testing-environment/spec.md new file mode 100644 index 00000000..cd0a529f --- /dev/null +++ b/openspec/changes/dev-testing-environment/specs/dev-testing-environment/spec.md @@ -0,0 +1,150 @@ +## ADDED Requirements + +### FR-001: Devcontainer configuration provides a Fedora testing environment + +The repository MUST include a `.devcontainer/` directory with a +`devcontainer.json` and `Containerfile` that together define a Fedora-based +development environment. The Containerfile MUST use +`registry.fedoraproject.org/fedora:43` as the base image and MUST install +`openscap-scanner`, `scap-security-guide`, `golang`, `git`, `make`, +`curl`, and `jq` via dnf. + +#### Scenario: Devcontainer configuration is present and valid + +- **GIVEN** the repository contains `.devcontainer/devcontainer.json` + and `.devcontainer/Containerfile` +- **WHEN** a maintainer or contributor opens the repository in a + devcontainer-compatible tool (GitHub Codespaces, DevPod, or VS Code + Dev Containers) +- **THEN** the tool MUST detect `.devcontainer/devcontainer.json` and + build the environment from the Containerfile + +#### Scenario: Container is based on Fedora with required packages + +- **GIVEN** the Containerfile has been built successfully +- **WHEN** the devcontainer finishes building +- **THEN** the running environment MUST be Fedora-based with + `openscap-scanner` and `scap-security-guide` available as installed + packages + +### FR-002: Post-create setup produces a ready-to-test environment + +The devcontainer MUST define a post-create command (script) that +automatically builds complyctl from the local source, installs snappy +and ampel via `go install` at pinned version tags, clones and builds +complytime-providers from main, copies provider binaries to the +discovery path (`~/.complytime/providers/`), configures a test +workspace with Gemara content, and starts the mock OCI registry in the +background. The post-create script MUST exit with code 0 on success +and non-zero with a descriptive error message if any setup step fails. +The script MUST use strict shell error handling (`set -euo pipefail`). +The script MUST NOT pass `GITHUB_TOKEN` to subprocesses that do not +require it (least-privilege: capture, unset, selectively inject). + +#### Scenario: All binaries are built and available + +- **GIVEN** the devcontainer has been built from the Containerfile +- **WHEN** the devcontainer post-create command completes successfully +- **THEN** `complyctl` and `mock-oci-registry` MUST be available in + `./bin/` (from `make build`), `snappy` and `ampel` MUST be available + in `$GOPATH/bin`, and `complyctl-provider-ampel` MUST be present in + `~/.complytime/providers/` + +#### Scenario: Test workspace is configured + +- **GIVEN** the post-create command has completed successfully +- **WHEN** the devcontainer post-create command completes +- **THEN** a `complytime.yaml` workspace configuration MUST exist in + the test workspace directory (`~/test-workspace/`), pointing to the + mock OCI registry on localhost + +#### Scenario: Mock OCI registry is running + +- **GIVEN** the post-create command has completed successfully +- **WHEN** the devcontainer post-create command completes +- **THEN** the mock OCI registry MUST be running and serving Gemara + catalogs and policies on the default port (8765) + +#### Scenario: Post-create script fails gracefully on setup errors + +- **GIVEN** a setup step fails (e.g., `go install`, `git clone`, + `make build`) +- **WHEN** the post-create script encounters the failure +- **THEN** the script MUST report which step failed and exit with a + non-zero status code + +#### Scenario: Post-create succeeds without GITHUB_TOKEN + +- **GIVEN** the environment does NOT have `GITHUB_TOKEN` set +- **WHEN** the devcontainer post-create command runs +- **THEN** the script MUST complete successfully (exit code 0) and + emit a warning that `GITHUB_TOKEN` is required for `complyctl scan` + +#### Scenario: complyctl get and generate work end-to-end + +- **GIVEN** the post-create command has completed successfully +- **WHEN** a maintainer or contributor runs `complyctl get` followed by + `complyctl generate --policy-id test-ampel-bp` in the test workspace +- **THEN** both commands MUST complete successfully with expected output + +#### Scenario: complyctl scan works when GITHUB_TOKEN is configured + +- **GIVEN** the environment has `GITHUB_TOKEN` set with read access to + public repositories +- **WHEN** a maintainer or contributor runs + `complyctl scan --policy-id test-ampel-bp` in the test workspace +- **THEN** the command MUST complete successfully and produce scan + results + +#### Scenario: complyctl scan fails gracefully without GITHUB_TOKEN + +- **GIVEN** the environment does NOT have `GITHUB_TOKEN` set +- **WHEN** a maintainer or contributor runs + `complyctl scan --policy-id test-ampel-bp` in the test workspace +- **THEN** the command MUST exit with a non-zero code and output a + message indicating the token is required + +### FR-003: Documentation explains workflows for maintainers and contributors + +The repository MUST include documentation in `docs/` that explains how +to use the devcontainer for PR review testing and interactive CLI +exploration. The documentation MUST cover GitHub Codespaces, DevPod, +and VS Code Dev Containers workflows. The documentation MUST explain +the GITHUB_TOKEN requirement for scan commands that use snappy. The +documentation MUST include a practical command reference showing the +commands available in the environment, their expected behavior, and +what success looks like. + +#### Scenario: Documentation is discoverable from README + +- **GIVEN** the repository README contains a Documentation section +- **WHEN** a maintainer or contributor reads the repository README +- **THEN** they MUST find a link or reference to the dev testing + environment documentation + +#### Scenario: Documentation covers all supported tools + +- **GIVEN** the dev testing environment documentation exists +- **WHEN** a maintainer or contributor reads the documentation +- **THEN** they MUST find instructions for using the devcontainer with + GitHub Codespaces, DevPod, and VS Code Dev Containers + +#### Scenario: Documentation includes a practical command reference + +- **GIVEN** the devcontainer setup has completed +- **WHEN** a maintainer or contributor opens the dev testing environment + documentation +- **THEN** they MUST find a command reference section listing the + available `complyctl` commands (`get`, `generate`, `scan`), the + expected behavior of each, and troubleshooting steps for common + issues (mock registry restart, GITHUB_TOKEN not set, OpenSCAP + container limitations) + +#### Scenario: GITHUB_TOKEN setup is documented + +- **GIVEN** a maintainer or contributor wants to run `complyctl scan` + in the devcontainer +- **WHEN** they read the dev testing environment documentation +- **THEN** the documentation MUST explain how to configure GITHUB_TOKEN + for each supported tool (Codespaces secrets, DevPod env, manual + export) diff --git a/openspec/changes/dev-testing-environment/tasks.md b/openspec/changes/dev-testing-environment/tasks.md new file mode 100644 index 00000000..0a07df8c --- /dev/null +++ b/openspec/changes/dev-testing-environment/tasks.md @@ -0,0 +1,83 @@ +## 1. Containerfile + +- [ ] 1.1 Create `.devcontainer/Containerfile` using + `registry.fedoraproject.org/fedora:43` as base image with dnf install of + `openscap-scanner`, `scap-security-guide`, `golang`, `git`, `make`, + `curl`, and `jq` + +## 2. Post-create setup script + +- [ ] 2.1 Create `.devcontainer/scripts/post-create.sh` with `set -euo + pipefail` and executable permissions (`chmod +x`). The script builds + complyctl and mock-oci-registry from local source (`make build`). + Each step MUST report what it is doing and exit with a descriptive + error on failure +- [ ] 2.2 Add `go install` of snappy + (`github.com/carabiner-dev/snappy@v0.2.4`) and ampel + (`github.com/carabiner-dev/ampel/cmd/ampel@v1.2.1`) to the script. + Versions MUST be pinned (not `@latest`). Document pinned versions in + a comment block in the script for easy updates +- [ ] 2.3 Add clone of `complytime-providers@main`, build providers, and + copy `complyctl-provider-ampel` (only -- not openscap, which has + limited functionality in containers) to `~/.complytime/providers/`. + Log the cloned commit SHA for auditability. If the clone fails + (network issue, broken main), the script MUST exit with a clear + error identifying the failure as an upstream dependency issue +- [ ] 2.4 Add workspace setup: copy `tests/cross-repo/testdata/complytime.yaml` + and granular policies to the test workspace directory (`~/test-workspace/`) +- [ ] 2.5 Add mock OCI registry startup in background with readiness check + (mirror the registry startup and `curl` readiness-poll pattern from + `tests/cross-repo/cross_repo_integration_test.sh`) +- [ ] 2.6 Add `GITHUB_TOKEN` least-privilege handling: capture the token + into an internal variable, unset `GITHUB_TOKEN` from the environment + so that `go install`, `git clone`, and `make build` do not inherit it. + Only pass the token to `complyctl` subprocesses that need it (mirror + the pattern from `cross_repo_integration_test.sh`). If the token is + not set, emit a warning that `complyctl scan` requires it, but do + not fail the script + +## 3. Devcontainer configuration + +- [ ] 3.1 Create `.devcontainer/devcontainer.json` referencing the + Containerfile and setting `postCreateCommand` to run the post-create + script + +## 4. Documentation + +- [ ] 4.1 Create `docs/dev-testing-environment.md` with sections covering: + how to open from a PR in GitHub Codespaces, how to use with DevPod + (CLI and desktop), how to use with VS Code Dev Containers extension, + GITHUB_TOKEN configuration for each tool, what the environment + provides (binaries, test content, mock registry), a practical command + reference listing the available `complyctl` commands (`get`, `generate`, + `scan`) with expected behavior and what success looks like, and + troubleshooting steps for common issues (mock registry restart, + GITHUB_TOKEN not set, OpenSCAP container limitations, Codespace + suspend/resume killing background processes) +- [ ] 4.2 Update `README.md` to add a link to the dev testing environment + documentation in the existing docs section +- [ ] 4.3 Update `AGENTS.md` project structure to include `.devcontainer/` + with its sub-entries (Containerfile, devcontainer.json, + scripts/post-create.sh) + +## 5. CI Smoke Test + +- [ ] 5.1 Add a `make test-devcontainer` target that builds the + Containerfile (`podman build .devcontainer/`) to verify the image + definition is valid. This provides automated regression protection + for the Containerfile without requiring the full post-create setup + in CI + +## 6. Verification (manual) + +- [ ] 6.1 Verify the Containerfile builds: `podman build .devcontainer/` + exits 0 +- [ ] 6.2 Verify the post-create script completes: all binaries on PATH + (`command -v complyctl snappy ampel`), mock registry responds at + `localhost:8765/v2/`, `complytime.yaml` exists in `~/test-workspace/` +- [ ] 6.3 Verify CLI pipeline: `complyctl get` outputs + `Synchronization completed.`, `complyctl generate --policy-id + test-ampel-bp` outputs `Generation completed.`, `complyctl scan + --policy-id test-ampel-bp` (with GITHUB_TOKEN set) produces scan + results + From 456281e24845641121be822054b32dc2983048a3 Mon Sep 17 00:00:00 2001 From: Marcus Burghardt Date: Fri, 29 May 2026 08:19:01 +0200 Subject: [PATCH 2/2] feat: add devcontainer testing environment for PR reviews Add .devcontainer/ configuration providing a Fedora 43-based testing environment for interactive CLI testing during PR reviews. New files: - .devcontainer/Containerfile: Fedora 43 base with required packages - .devcontainer/devcontainer.json: devcontainer standard config - .devcontainer/scripts/post-create.sh: automated setup script - docs/dev-testing-environment.md: maintainer/contributor docs Modified files: - README.md: add link to dev testing environment docs - AGENTS.md: add .devcontainer/ to project structure - Makefile: add test-devcontainer smoke test target The post-create script builds complyctl and mock-oci-registry, installs snappy (v0.2.4) and ampel (v1.2.1) at pinned versions, clones and builds complytime-providers from main, configures a test workspace with Gemara content, and starts the mock OCI registry. GITHUB_TOKEN is handled with least-privilege scoping. Assisted-by: OpenCode (claude-opus-4-6) Signed-off-by: Marcus Burghardt --- .devcontainer/Containerfile | 24 ++ .devcontainer/devcontainer.json | 15 + .devcontainer/scripts/post-create.sh | 168 ++++++++++ AGENTS.md | 10 + CHANGELOG.md | 4 + Makefile | 5 + README.md | 1 + docs/TESTING_ENVIRONMENT.md | 288 ++++++++++++++++++ .../dev-testing-environment/.openspec.yaml | 2 +- .../changes/dev-testing-environment/design.md | 43 ++- .../dev-testing-environment/proposal.md | 8 +- .../specs/dev-testing-environment/spec.md | 29 +- .../changes/dev-testing-environment/tasks.md | 41 +-- 13 files changed, 609 insertions(+), 29 deletions(-) create mode 100644 .devcontainer/Containerfile create mode 100644 .devcontainer/devcontainer.json create mode 100755 .devcontainer/scripts/post-create.sh create mode 100644 docs/TESTING_ENVIRONMENT.md diff --git a/.devcontainer/Containerfile b/.devcontainer/Containerfile new file mode 100644 index 00000000..5d04ce92 --- /dev/null +++ b/.devcontainer/Containerfile @@ -0,0 +1,24 @@ +FROM registry.fedoraproject.org/fedora:43 + +# hadolint ignore=DL3041 +RUN dnf install -y \ + openscap-scanner \ + scap-security-guide \ + golang \ + git \ + make \ + curl \ + jq \ + tree \ + vim-enhanced \ + && dnf clean all + +# Create non-root user for container security best practice. +# devcontainer.json sets remoteUser=root for DevPod/Codespaces +# compatibility, but the default container user is non-root. +RUN useradd -m -s /bin/bash devuser +USER devuser + +# Allow go install to fetch newer toolchains when dependencies +# require a Go version newer than the system-packaged one. +ENV GOTOOLCHAIN=auto diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json new file mode 100644 index 00000000..1ae7e96d --- /dev/null +++ b/.devcontainer/devcontainer.json @@ -0,0 +1,15 @@ +{ + "name": "complyctl", + "build": { + "dockerfile": "Containerfile" + }, + "remoteUser": "root", + "runArgs": [ + "--security-opt", "label=disable" + ], + "postCreateCommand": "bash .devcontainer/scripts/post-create.sh", + "hostRequirements": { + "cpus": 4, + "memory": "16gb" + } +} diff --git a/.devcontainer/scripts/post-create.sh b/.devcontainer/scripts/post-create.sh new file mode 100755 index 00000000..25716f19 --- /dev/null +++ b/.devcontainer/scripts/post-create.sh @@ -0,0 +1,168 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: Apache-2.0 + +set -euo pipefail + +# --------------------------------------------------------------------------- +# PATH setup — ensure built binaries and Go-installed tools are available +# --------------------------------------------------------------------------- +export PATH="./bin:${GOPATH:-$(go env GOPATH)}/bin:${PATH}" + +# --------------------------------------------------------------------------- +# GITHUB_TOKEN least-privilege: capture and unset from environment +# Providers clone may need the token, but nothing else should see it. +# --------------------------------------------------------------------------- +_GITHUB_TOKEN="${GITHUB_TOKEN:-}" +unset GITHUB_TOKEN + +if [[ -z "${_GITHUB_TOKEN}" ]]; then + echo "WARNING: GITHUB_TOKEN was not set when the container started." + echo " complyctl scan requires a GitHub token to query branch" + echo " protection rules via snappy. get and generate work without it." + echo " Set it in your shell: export GITHUB_TOKEN=" +else + echo "NOTE: GITHUB_TOKEN was unset from the environment during setup" + echo " (least-privilege). Re-export it for scan commands:" + echo " export GITHUB_TOKEN=" +fi + +# --------------------------------------------------------------------------- +# Step 1: Build complyctl and mock-oci-registry +# make build compiles all cmd/ packages (complyctl, mock-oci-registry, etc.) +# --------------------------------------------------------------------------- +echo ">>> Building complyctl and mock-oci-registry..." +make build +echo " Build complete. Binaries in ./bin/" + +# --------------------------------------------------------------------------- +# Step 2: Install snappy, ampel, and conftest +# +# Pinned versions — update these when upgrading: +# snappy v0.2.4 https://github.com/carabiner-dev/snappy +# ampel v1.2.1 https://github.com/carabiner-dev/ampel +# conftest v0.68.2 https://github.com/open-policy-agent/conftest +# --------------------------------------------------------------------------- +echo ">>> Installing snappy, ampel, and conftest..." +go install github.com/carabiner-dev/snappy@v0.2.4 +go install github.com/carabiner-dev/ampel/cmd/ampel@v1.2.1 +go install github.com/open-policy-agent/conftest@v0.68.2 +echo " snappy, ampel, and conftest installed." + +# --------------------------------------------------------------------------- +# Step 3: Clone and build complytime-providers (all providers) +# --------------------------------------------------------------------------- +echo ">>> Cloning complytime-providers..." +PROVIDERS_TMP="$(mktemp -d)" +trap 'rm -rf "${PROVIDERS_TMP}"' EXIT + +# D6: Intentionally unpinned — tracks main for latest provider code. +# The commit SHA is logged below for auditability. +if ! git clone --depth 1 \ + https://github.com/complytime/complytime-providers.git \ + "${PROVIDERS_TMP}/complytime-providers"; then + echo "FATAL: Failed to clone complytime-providers." + echo " This is an upstream dependency required for the dev environment." + exit 1 +fi + +PROVIDERS_SHA="$(git -C "${PROVIDERS_TMP}/complytime-providers" \ + rev-parse HEAD)" +echo " Cloned complytime-providers at ${PROVIDERS_SHA}" + +echo ">>> Building complytime-providers..." +make -C "${PROVIDERS_TMP}/complytime-providers" build + +echo ">>> Installing provider binaries..." +mkdir -p "${HOME}/.complytime/providers" +for provider in ampel openscap opa; do + binary="complyctl-provider-${provider}" + src="${PROVIDERS_TMP}/complytime-providers/bin/${binary}" + if [[ -f "${src}" ]]; then + cp "${src}" "${HOME}/.complytime/providers/" + echo " Installed ${binary}" + else + echo " WARNING: ${binary} not found in build output, skipping." + fi +done + +# --------------------------------------------------------------------------- +# Step 4: Workspace setup — test-workspace with config and policies +# --------------------------------------------------------------------------- +echo ">>> Setting up test workspace..." +mkdir -p "${HOME}/test-workspace/.complytime/ampel/granular-policies" + +cp tests/cross-repo/testdata/complytime.yaml \ + "${HOME}/test-workspace/" + +cp tests/cross-repo/testdata/granular-policies/block-force-push.json \ + "${HOME}/test-workspace/.complytime/ampel/granular-policies/" + +echo " Test workspace ready at ~/test-workspace/" + +# --------------------------------------------------------------------------- +# Step 5: Start mock OCI registry +# --------------------------------------------------------------------------- +if curl -sf http://localhost:8765/v2/ > /dev/null 2>&1; then + echo ">>> Mock OCI registry already running on port 8765." +else + echo ">>> Starting mock OCI registry..." + ./bin/mock-oci-registry & + REGISTRY_PID=$! + +RETRIES=0 +MAX_RETRIES=30 +until curl -sf http://localhost:8765/v2/ > /dev/null 2>&1; do + RETRIES=$((RETRIES + 1)) + if [[ ${RETRIES} -ge ${MAX_RETRIES} ]]; then + echo "FATAL: Mock OCI registry failed to start after ${MAX_RETRIES} retries." + exit 1 + fi + sleep 0.5 +done + + echo " Mock OCI registry running (PID: ${REGISTRY_PID}, port: 8765)" +fi + +# --------------------------------------------------------------------------- +# Step 6: Record build commit for auto-rebuild detection +# --------------------------------------------------------------------------- +git rev-parse HEAD > ./bin/.build-commit + +# --------------------------------------------------------------------------- +# Step 7: Persist PATH and auto-rebuild hook for interactive shells +# --------------------------------------------------------------------------- +REPO_ROOT="$(pwd)" +if ! grep -q "complyctl dev environment" "${HOME}/.bashrc" 2>/dev/null; then + cat >> "${HOME}/.bashrc" << 'BASHRC' + +# complyctl dev environment — added by post-create.sh +export PATH="REPO_ROOT_PLACEHOLDER/bin:${GOPATH:-$(go env GOPATH)}/bin:${PATH}" + +# Auto-rebuild complyctl when source has changed (e.g., after +# checking out a PR branch). Skip with: export COMPLYCTL_SKIP_REBUILD=1 +if [[ -z "${COMPLYCTL_SKIP_REBUILD:-}" ]]; then + _repo="REPO_ROOT_PLACEHOLDER" + _build_commit="" + if [[ -f "${_repo}/bin/.build-commit" ]]; then + _build_commit="$(cat "${_repo}/bin/.build-commit")" + fi + _head="$(git -C "${_repo}" rev-parse HEAD 2>/dev/null || true)" + if [[ -n "${_head}" && "${_head}" != "${_build_commit}" ]]; then + echo ">>> Source changed (${_build_commit:0:8}..${_head:0:8}), rebuilding complyctl..." + if make -C "${_repo}" build 2>&1; then + echo "${_head}" > "${_repo}/bin/.build-commit" + echo " Rebuild complete." + else + echo " WARNING: Rebuild failed. Run 'make build' manually." + fi + fi + unset _repo _build_commit _head +fi +BASHRC + # Replace placeholder with actual repo root path + sed -i "s|REPO_ROOT_PLACEHOLDER|${REPO_ROOT}|g" "${HOME}/.bashrc" +fi + +echo ">>> Dev environment ready." +echo " Test workspace: ~/test-workspace/" +echo " Run: cd ~/test-workspace && complyctl get" diff --git a/AGENTS.md b/AGENTS.md index e1d42f36..799f8a33 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -46,6 +46,10 @@ make test-cross-repo PROVIDERS_BIN_DIR=/path/to/providers/bin # Behavioral assessment (EvaluationLog + SARIF reports) make test-behavioral + +# Devcontainer smoke test (verifies Containerfile builds) +make test-devcontainer +# → podman build -t complyctl-devcontainer-test .devcontainer/ ``` ### Lint & Format @@ -88,6 +92,11 @@ make crapload-check # check for CRAP regressions against baseline ## Project Structure ```text +.devcontainer/ # devcontainer config for testing environment +├── Containerfile # Fedora base image definition +├── devcontainer.json # devcontainer standard configuration +└── scripts/ + └── post-create.sh # setup automation script api/ # protobuf definitions (provider gRPC API) cmd/ ├── complyctl/ # CLI entrypoint (main.go) @@ -276,6 +285,7 @@ packages organized by domain responsibility. ## Recent Changes +- dev-testing-environment: Added `.devcontainer/` with Fedora-based devcontainer for interactive CLI testing; `docs/TESTING_ENVIRONMENT.md` documentation; `make test-devcontainer` CI smoke target; post-create script with GITHUB_TOKEN least-privilege handling - scan-error-exit-codes: `complyctl scan` exits non-zero on operational errors; `ScanResponse.errors` proto field added; `ScanResult`/`RouteScanResult()` in `pkg/provider/manager.go`; `FormatOperationalWarnings` in `internal/output/scan_summary.go`; `processScanOutput`/`checkOperationalErrors`/`reportOperationalWarnings` in `cmd/complyctl/cli/scan.go` - 005-bundle-resolver-alignment: Policy resolver supports both split-layer and Gemara bundle-format OCI artifacts; `internal/policy/loader.go` gained `LoadBundleFiles()`, `DetectManifestShape()`, `resolveManifest()`; `PolicyLoader` interface extended with bundle methods; `MockBundlePolicySource` added to `internal/cache/cachetest/` - 005-rpm-packaging-ci: Added Go 1.25 + go-rpm-macros, Packit, Testing Farm (TMT/FMF) diff --git a/CHANGELOG.md b/CHANGELOG.md index ec6c4268..41d37be5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,10 @@ ### Added +- Devcontainer configuration for interactive CLI testing during PR + reviews (`.devcontainer/`, `docs/TESTING_ENVIRONMENT.md`, + `make test-devcontainer`). Supports GitHub Codespaces, DevPod, and + VS Code Dev Containers. - Cross-repo integration test infrastructure validating the complyctl + Ampel provider pipeline end-to-end (`tests/cross-repo/`, `make test-cross-repo`). - CI workflow `ci_cross_repo_integration.yml` that builds complyctl from the PR diff --git a/Makefile b/Makefile index ab8312e9..bce565c6 100644 --- a/Makefile +++ b/Makefile @@ -57,6 +57,11 @@ endif timeout 120 ./tests/cross-repo/cross_repo_integration_test.sh .PHONY: test-cross-repo +test-devcontainer: ## verify devcontainer Containerfile builds + podman build -t complyctl-devcontainer-test .devcontainer/ + @echo "Containerfile builds successfully." +.PHONY: test-devcontainer + ##@ Compilation all: clean vendor test-unit build ## compile from scratch diff --git a/README.md b/README.md index baf45f57..211f9507 100644 --- a/README.md +++ b/README.md @@ -65,6 +65,7 @@ A lightweight compliance runtime that pulls [Gemara](https://gemara.openssf.org/ - [Quick Start](./docs/QUICK_START.md) - [Provider Guide](https://github.com/complytime/complytime-providers/blob/main/docs/provider-guide.md) - [E2E Testing](./tests/e2e/README.md) +- [Testing Environment](./docs/TESTING_ENVIRONMENT.md) ## CLI Commands diff --git a/docs/TESTING_ENVIRONMENT.md b/docs/TESTING_ENVIRONMENT.md new file mode 100644 index 00000000..dddc923b --- /dev/null +++ b/docs/TESTING_ENVIRONMENT.md @@ -0,0 +1,288 @@ +# Dev Testing Environment + +complyctl includes a Fedora-based devcontainer that provides a +one-command path to interactive CLI testing during PR reviews. +The environment comes pre-built with all binaries, a mock OCI +registry loaded with test content, and a ready-to-use workspace +so reviewers can immediately exercise `complyctl` commands +against realistic policy data. + +## Prerequisites + +You need one of the following tools installed to open the +devcontainer. If none are installed, pick the one that fits +your workflow: + +- **GitHub Codespaces** -- no local setup required; works + directly from a PR on GitHub. Best for quick PR review + testing. +- **DevPod** -- open-source, runs locally or on remote + providers. Install from + [devpod.sh/docs/getting-started/install](https://devpod.sh/docs/getting-started/install). + Requires a container runtime (`podman` or `docker`). +- **VS Code** with the + [Dev Containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) + extension. Requires a container runtime (`podman` or + `docker`). + +The devcontainer builds a Fedora image and runs a post-create +script that sets up all binaries and test content. This takes +2-5 minutes on first launch. The `~/test-workspace/` directory +and mock OCI registry are only available **inside** the running +devcontainer, not on your host machine. + +## GitHub Codespaces + +1. Navigate to the PR you want to test on GitHub. +2. Click **Code** > **Codespaces** > **Create codespace on + \**. +3. Codespaces auto-detects `.devcontainer/devcontainer.json` + and builds the environment. + +**GITHUB_TOKEN**: Set via **Settings > Codespaces > Secrets** +in your GitHub user settings, or configure it at the repository +level under **Settings > Secrets and variables > Codespaces**. + +**Note**: Codespace suspend/resume kills background processes. +The mock OCI registry will need to be restarted manually after +resuming -- see the Troubleshooting section below. + +## DevPod + +### First-time setup + +Configure the provider to auto-stop containers after 30 +minutes of inactivity. This only needs to be done once and +applies to all workspaces: + +```bash +# For podman (default on Fedora/RHEL) +devpod provider set-options podman \ + -o INACTIVITY_TIMEOUT=30m + +# For docker +devpod provider set-options docker \ + -o INACTIVITY_TIMEOUT=30m +``` + +To change the timeout later, re-run with a different value +(e.g., `1h`, `10m`). + +### CLI + +Create the workspace and connect: + +```bash +# From a remote repository (e.g., testing a PR) +devpod up github.com/complytime/complyctl --ide none \ + && devpod ssh complyctl + +# From a local branch (e.g., testing your own changes) +devpod up . --ide none && devpod ssh complyctl +``` + +### Testing a different branch or PR + +If you already have a workspace and check out a different +branch (e.g., a contributor's PR), the source code inside +the container updates automatically because DevPod +bind-mounts your local directory. + +The environment **auto-rebuilds** `complyctl` when it +detects the source has changed (different commit). Start +the workspace and connect -- the rebuild happens on login: + +```bash +git checkout pr-branch +devpod up . --ide none && devpod ssh complyctl +``` + +To skip the auto-rebuild (e.g., for a docs-only change): + +```bash +COMPLYCTL_SKIP_REBUILD=1 devpod ssh complyctl +``` + +To fully recreate the workspace from scratch: + +```bash +devpod up . --ide none --recreate && devpod ssh complyctl +``` + +### Resuming a stopped container + +If the container was stopped (by inactivity timeout or +`devpod stop`), use `devpod up` to restart it. Do **not** +use `devpod ssh` directly on a stopped workspace -- it may +fail to reconnect: + +```bash +devpod up . --ide none && devpod ssh complyctl +``` + +When done, exit the SSH session (`exit`). The container +stops automatically after the inactivity timeout, or stop +it immediately: + +```bash +devpod stop complyctl +``` + +### Desktop + +Open DevPod Desktop and add a new workspace from the GitHub +URL `https://github.com/complytime/complyctl`. + +**GITHUB_TOKEN**: Export the token before starting DevPod: + +```bash +export GITHUB_TOKEN= +devpod up github.com/complytime/complyctl --ide none +``` + +Alternatively, configure the environment variable through +DevPod's environment configuration settings. + +## VS Code Dev Containers + +1. Clone the repository locally. +2. Open the repository folder in VS Code. +3. When prompted, click **Reopen in Container**. + +You can also open the Command Palette (`Ctrl+Shift+P` / +`Cmd+Shift+P`) and select **Dev Containers: Reopen in +Container**. + +**GITHUB_TOKEN**: Either export the token before starting +VS Code: + +```bash +export GITHUB_TOKEN= +code /path/to/complyctl +``` + +Or set it in the integrated terminal after the container opens: + +```bash +export GITHUB_TOKEN= +``` + +## What the Environment Provides + +### Binaries + +| Binary | Location | +|--------|----------| +| `complyctl` | `./bin/` | +| `mock-oci-registry` | `./bin/` | +| `snappy` | `$GOPATH/bin` | +| `ampel` | `$GOPATH/bin` | +| `complyctl-provider-ampel` | `~/.complytime/providers/` | +| `complyctl-provider-openscap` | `~/.complytime/providers/` | +| `complyctl-provider-opa` | `~/.complytime/providers/` | + +### Test Content + +- A mock OCI registry running on `localhost:8765`, loaded with + Gemara test catalogs and policies. +- A test workspace at `~/test-workspace/` with a + `complytime.yaml` pre-configured to point at the mock + registry. + +### System Packages + +- `openscap-scanner` +- `scap-security-guide` + +**Note**: OpenSCAP has limited functionality in containers. +See [Troubleshooting > OpenSCAP limitations](#openscap-limitations) +for details and the recommended testing path. + +## Command Reference + +These are the primary commands for testing `complyctl` inside +the devcontainer: + +```bash +cd ~/test-workspace + +# Fetch policies from the mock registry +complyctl get +# Expected: "Synchronization completed." + +# Generate a policy bundle for the ampel provider +complyctl generate --policy-id test-ampel-bp +# Expected: "Generation completed." + +# Run a scan (requires GITHUB_TOKEN) +GITHUB_TOKEN= complyctl scan \ + --policy-id test-ampel-bp +# Expected: Scan results with requirement status +``` + +## Troubleshooting + +### Mock registry not running + +If `complyctl get` fails to connect, the mock registry may not +be running. Start it manually and verify: + +```bash +./bin/mock-oci-registry & +curl -sf http://localhost:8765/v2/ +``` + +A successful response confirms the registry is available. + +### GITHUB_TOKEN not set + +`complyctl scan` will fail if `GITHUB_TOKEN` is not set. +Export it in your shell: + +```bash +export GITHUB_TOKEN= +``` + +### OpenSCAP limitations + +OpenSCAP system scans are limited inside containers due to +missing host-level access. Use the Ampel provider with the mock +registry for CLI testing. Full OpenSCAP testing is available via +[complytime-demos](https://github.com/complytime/complytime-demos). + +### File ownership changed after using DevPod (podman) + +When using DevPod with podman rootless, the container's user +namespace remaps your host UID to a different UID inside the +container. This can change file ownership on the host after +the workspace stops, causing git to refuse operations: + +``` +fatal: detected dubious ownership in repository at '/path/to/complyctl' +``` + +Fix by restoring ownership with `podman unshare`: + +```bash +podman unshare chown -R 0:0 /path/to/complyctl +``` + +This is inherent to podman rootless user namespace mapping +and does not affect files inside the running devcontainer. + +### After Codespace resume + +Background processes are killed when a Codespace is suspended. +After resuming, restart the mock registry manually: + +```bash +./bin/mock-oci-registry & +``` + +## See Also + +- [Quick Start](./QUICK_START.md) +- [E2E Testing](../tests/e2e/README.md) +- [Cross-Repo Integration Tests](../tests/cross-repo/) +- [complytime-demos](https://github.com/complytime/complytime-demos) + -- full OpenSCAP testing in a Fedora VM diff --git a/openspec/changes/dev-testing-environment/.openspec.yaml b/openspec/changes/dev-testing-environment/.openspec.yaml index d39be0bd..36ba5c29 100644 --- a/openspec/changes/dev-testing-environment/.openspec.yaml +++ b/openspec/changes/dev-testing-environment/.openspec.yaml @@ -1,3 +1,3 @@ schema: spec-driven created: 2026-05-21 -status: specifying +status: verifying diff --git a/openspec/changes/dev-testing-environment/design.md b/openspec/changes/dev-testing-environment/design.md index c3d92c07..74a33c5f 100644 --- a/openspec/changes/dev-testing-environment/design.md +++ b/openspec/changes/dev-testing-environment/design.md @@ -138,7 +138,41 @@ are same-org, under maintainer trust, and the devcontainer's value is testing the latest code. The cloned commit SHA is logged for auditability. -### D7: Canonical source in complyctl, thin consumer in complytime-providers +### D7: GOTOOLCHAIN=auto for Go version compatibility + +**Decision**: Set `ENV GOTOOLCHAIN=auto` in the Containerfile rather +than installing a specific Go version from source. + +**Rationale**: Fedora's packaged Go may lag behind what dependencies +require (e.g., Fedora 43 ships Go 1.25, but snappy requires Go 1.26). +`GOTOOLCHAIN=auto` lets `go install` download the required toolchain +automatically, avoiding the need to track Go releases in the +Containerfile. + +### D8: SELinux compatibility via runArgs + +**Decision**: Set `--security-opt label=disable` in `devcontainer.json` +`runArgs` to disable SELinux label enforcement for the container. + +**Rationale**: Podman rootless on SELinux-enforcing hosts (Fedora, RHEL) +blocks access to bind-mounted workspace files without the correct +SELinux context. Using `label=disable` is the standard devcontainer +approach for dev-only environments where the security boundary is the +developer's own workstation. + +### D9: Auto-rebuild on source change + +**Decision**: The post-create script installs a `.bashrc` hook that +compares the current HEAD commit against a stored build marker +(`./bin/.build-commit`). If they differ, `make build` runs +automatically on shell login. + +**Rationale**: The primary use case is testing PR changes. After +checking out a contributor's branch, the user should immediately have +working binaries without remembering to run `make build`. The opt-out +(`COMPLYCTL_SKIP_REBUILD=1`) covers docs-only or spec-only changes. + +### D10: Canonical source in complyctl, thin consumer in complytime-providers **Decision**: The full devcontainer configuration (Containerfile, post-create script, documentation) lives in complyctl. complytime-providers @@ -199,6 +233,13 @@ the testing environment definition. install`, `git clone`, and `make build` from inheriting the token unnecessarily. +- **[Podman rootless file ownership shifts]** → Running DevPod with + podman rootless causes user namespace UID mapping that changes file + ownership on the host. After stopping a devcontainer, files may + appear owned by a mapped UID (e.g., 165536). Fix with + `podman unshare chown -R 0:0 `. This is inherent to rootless + containers and does not affect the devcontainer's functionality. + - **[Fedora base image tag pinning]** → The Containerfile uses `fedora:43` (a specific version tag, not `latest`). While digest pinning would provide full reproducibility, it creates a maintenance diff --git a/openspec/changes/dev-testing-environment/proposal.md b/openspec/changes/dev-testing-environment/proposal.md index db5b4e14..701a63f0 100644 --- a/openspec/changes/dev-testing-environment/proposal.md +++ b/openspec/changes/dev-testing-environment/proposal.md @@ -37,13 +37,13 @@ environment for any PR. ## Impact - **New files**: `.devcontainer/Containerfile`, `.devcontainer/devcontainer.json`, - `.devcontainer/scripts/post-create.sh`, `docs/dev-testing-environment.md` + `.devcontainer/scripts/post-create.sh`, `docs/TESTING_ENVIRONMENT.md` - **Modified files**: `README.md` (add link to new docs) - **Dependencies**: Uses `registry.fedoraproject.org/fedora:43` as the base container image. Installs `openscap-scanner`, `scap-security-guide`, `curl`, - and `jq` via dnf. Installs `snappy` (v0.2.4) and `ampel` (v1.2.1) via - `go install` at pinned versions from `carabiner-dev`. Clones - `complytime-providers` from GitHub at build time. + `jq`, `tree`, and `vim-enhanced` via dnf. Installs `snappy` (v0.2.4), + `ampel` (v1.2.1), and `conftest` (v0.68.2) via `go install` at pinned + versions. Clones `complytime-providers` from GitHub at build time. - **No changes to existing code**: This is purely additive infrastructure. - **Downstream**: complytime-providers will add its own thin devcontainer configuration in a follow-up change, consuming the same pattern. diff --git a/openspec/changes/dev-testing-environment/specs/dev-testing-environment/spec.md b/openspec/changes/dev-testing-environment/specs/dev-testing-environment/spec.md index cd0a529f..37782f90 100644 --- a/openspec/changes/dev-testing-environment/specs/dev-testing-environment/spec.md +++ b/openspec/changes/dev-testing-environment/specs/dev-testing-environment/spec.md @@ -7,7 +7,9 @@ The repository MUST include a `.devcontainer/` directory with a development environment. The Containerfile MUST use `registry.fedoraproject.org/fedora:43` as the base image and MUST install `openscap-scanner`, `scap-security-guide`, `golang`, `git`, `make`, -`curl`, and `jq` via dnf. +`curl`, `jq`, `tree`, and `vim-enhanced` via dnf. The Containerfile +MUST set `ENV GOTOOLCHAIN=auto` so that `go install` can fetch newer +Go toolchains when dependencies require them. #### Scenario: Devcontainer configuration is present and valid @@ -30,8 +32,9 @@ development environment. The Containerfile MUST use ### FR-002: Post-create setup produces a ready-to-test environment The devcontainer MUST define a post-create command (script) that -automatically builds complyctl from the local source, installs snappy -and ampel via `go install` at pinned version tags, clones and builds +automatically builds complyctl from the local source, installs snappy, +ampel, and conftest via `go install` at pinned version tags, clones and +builds complytime-providers from main, copies provider binaries to the discovery path (`~/.complytime/providers/`), configures a test workspace with Gemara content, and starts the mock OCI registry in the @@ -46,8 +49,10 @@ require it (least-privilege: capture, unset, selectively inject). - **GIVEN** the devcontainer has been built from the Containerfile - **WHEN** the devcontainer post-create command completes successfully - **THEN** `complyctl` and `mock-oci-registry` MUST be available in - `./bin/` (from `make build`), `snappy` and `ampel` MUST be available - in `$GOPATH/bin`, and `complyctl-provider-ampel` MUST be present in + `./bin/` (from `make build`), `snappy`, `ampel`, and `conftest` MUST + be available in `$GOPATH/bin`, and `complyctl-provider-ampel`, + `complyctl-provider-openscap`, and + `complyctl-provider-opa` MUST be present in `~/.complytime/providers/` #### Scenario: Test workspace is configured @@ -96,6 +101,15 @@ require it (least-privilege: capture, unset, selectively inject). - **THEN** the command MUST complete successfully and produce scan results +#### Scenario: Auto-rebuild on source change + +- **GIVEN** the devcontainer is running and the user checks out a + different branch (e.g., a contributor's PR) +- **WHEN** the user opens a new shell session inside the container +- **THEN** the environment MUST detect the source change (different + HEAD commit) and automatically rebuild complyctl. The user MUST be + able to skip the auto-rebuild by setting `COMPLYCTL_SKIP_REBUILD=1` + #### Scenario: complyctl scan fails gracefully without GITHUB_TOKEN - **GIVEN** the environment does NOT have `GITHUB_TOKEN` set @@ -113,7 +127,10 @@ and VS Code Dev Containers workflows. The documentation MUST explain the GITHUB_TOKEN requirement for scan commands that use snappy. The documentation MUST include a practical command reference showing the commands available in the environment, their expected behavior, and -what success looks like. +what success looks like. The `devcontainer.json` MUST include +`--security-opt label=disable` in `runArgs` for SELinux compatibility +with podman rootless. The documentation MUST explain how to configure +DevPod inactivity timeout and how to resume a stopped container. #### Scenario: Documentation is discoverable from README diff --git a/openspec/changes/dev-testing-environment/tasks.md b/openspec/changes/dev-testing-environment/tasks.md index 0a07df8c..fcf1b79b 100644 --- a/openspec/changes/dev-testing-environment/tasks.md +++ b/openspec/changes/dev-testing-environment/tasks.md @@ -1,34 +1,36 @@ ## 1. Containerfile -- [ ] 1.1 Create `.devcontainer/Containerfile` using +- [x] 1.1 Create `.devcontainer/Containerfile` using `registry.fedoraproject.org/fedora:43` as base image with dnf install of `openscap-scanner`, `scap-security-guide`, `golang`, `git`, `make`, - `curl`, and `jq` + `curl`, `jq`, `tree`, and `vim-enhanced` ## 2. Post-create setup script -- [ ] 2.1 Create `.devcontainer/scripts/post-create.sh` with `set -euo +- [x] 2.1 Create `.devcontainer/scripts/post-create.sh` with `set -euo pipefail` and executable permissions (`chmod +x`). The script builds complyctl and mock-oci-registry from local source (`make build`). Each step MUST report what it is doing and exit with a descriptive error on failure -- [ ] 2.2 Add `go install` of snappy - (`github.com/carabiner-dev/snappy@v0.2.4`) and ampel - (`github.com/carabiner-dev/ampel/cmd/ampel@v1.2.1`) to the script. +- [x] 2.2 Add `go install` of snappy + (`github.com/carabiner-dev/snappy@v0.2.4`), ampel + (`github.com/carabiner-dev/ampel/cmd/ampel@v1.2.1`), and conftest + (`github.com/open-policy-agent/conftest@v0.68.2`) to the script. Versions MUST be pinned (not `@latest`). Document pinned versions in a comment block in the script for easy updates -- [ ] 2.3 Add clone of `complytime-providers@main`, build providers, and - copy `complyctl-provider-ampel` (only -- not openscap, which has - limited functionality in containers) to `~/.complytime/providers/`. +- [x] 2.3 Add clone of `complytime-providers@main`, build providers, and + copy all provider binaries (`complyctl-provider-ampel`, + `complyctl-provider-openscap`, `complyctl-provider-opa`) to + `~/.complytime/providers/`. Log the cloned commit SHA for auditability. If the clone fails (network issue, broken main), the script MUST exit with a clear error identifying the failure as an upstream dependency issue -- [ ] 2.4 Add workspace setup: copy `tests/cross-repo/testdata/complytime.yaml` +- [x] 2.4 Add workspace setup: copy `tests/cross-repo/testdata/complytime.yaml` and granular policies to the test workspace directory (`~/test-workspace/`) -- [ ] 2.5 Add mock OCI registry startup in background with readiness check +- [x] 2.5 Add mock OCI registry startup in background with readiness check (mirror the registry startup and `curl` readiness-poll pattern from `tests/cross-repo/cross_repo_integration_test.sh`) -- [ ] 2.6 Add `GITHUB_TOKEN` least-privilege handling: capture the token +- [x] 2.6 Add `GITHUB_TOKEN` least-privilege handling: capture the token into an internal variable, unset `GITHUB_TOKEN` from the environment so that `go install`, `git clone`, and `make build` do not inherit it. Only pass the token to `complyctl` subprocesses that need it (mirror @@ -36,15 +38,19 @@ not set, emit a warning that `complyctl scan` requires it, but do not fail the script +- [x] 2.7 Add auto-rebuild hook to `.bashrc`: compare HEAD against + `./bin/.build-commit` on shell login, rebuild if changed. Skip with + `COMPLYCTL_SKIP_REBUILD=1` + ## 3. Devcontainer configuration -- [ ] 3.1 Create `.devcontainer/devcontainer.json` referencing the +- [x] 3.1 Create `.devcontainer/devcontainer.json` referencing the Containerfile and setting `postCreateCommand` to run the post-create script ## 4. Documentation -- [ ] 4.1 Create `docs/dev-testing-environment.md` with sections covering: +- [x] 4.1 Create `docs/TESTING_ENVIRONMENT.md` with sections covering: how to open from a PR in GitHub Codespaces, how to use with DevPod (CLI and desktop), how to use with VS Code Dev Containers extension, GITHUB_TOKEN configuration for each tool, what the environment @@ -54,15 +60,15 @@ troubleshooting steps for common issues (mock registry restart, GITHUB_TOKEN not set, OpenSCAP container limitations, Codespace suspend/resume killing background processes) -- [ ] 4.2 Update `README.md` to add a link to the dev testing environment +- [x] 4.2 Update `README.md` to add a link to the dev testing environment documentation in the existing docs section -- [ ] 4.3 Update `AGENTS.md` project structure to include `.devcontainer/` +- [x] 4.3 Update `AGENTS.md` project structure to include `.devcontainer/` with its sub-entries (Containerfile, devcontainer.json, scripts/post-create.sh) ## 5. CI Smoke Test -- [ ] 5.1 Add a `make test-devcontainer` target that builds the +- [x] 5.1 Add a `make test-devcontainer` target that builds the Containerfile (`podman build .devcontainer/`) to verify the image definition is valid. This provides automated regression protection for the Containerfile without requiring the full post-create setup @@ -81,3 +87,4 @@ --policy-id test-ampel-bp` (with GITHUB_TOKEN set) produces scan results +