diff --git a/architecture/trusted-publishing-proposal.md b/architecture/trusted-publishing-proposal.md new file mode 100644 index 0000000000..805840e5ea --- /dev/null +++ b/architecture/trusted-publishing-proposal.md @@ -0,0 +1,140 @@ +### **Proposal: Modernizing the Release Pipeline with an Orphan Manifest Branch** + +#### **1. Objective** + +To re-architect the release process to enable **npm trusted publishing (OIDC)** by eliminating the dependency on `npm dist-tag` for client discovery. This new architecture will be more robust, eliminate race conditions, and provide powerful new features for both end-users and enterprise clients, such as version pinning and simplified asset mirroring. + +--- + +#### **2. The New Architecture** + +The core of this solution is to use a dedicated, **orphan Git branch** as the single source of truth for release metadata. This branch, named `release-manifests`, will have a separate history from `main` and will contain only JSON manifest files. + +This approach is clean, robust, and leverages a battle-tested pattern for separating source code from generated release assets. + +##### **Two-Tier Manifest System** + +For each successful release of a tool (e.g., `appmap`), the CI process will generate and publish two distinct manifest files to the `release-manifests` branch: + +1. **Versioned Manifest:** An immutable, permanent manifest named after the full version tag (e.g., `appmap-v1.2.3.json`). This creates a complete, browsable history of all release metadata. +2. **"Latest" Pointer Manifest:** A "floating" manifest (e.g., `appmap-latest.json`) which is an exact copy of the latest versioned manifest. This file acts as a stable pointer for clients to discover the most recent version. + +--- + +#### **3. Implementation Plan** + +##### **Part 1: CI/CD Workflow Modifications** + +1. **`release.yml` (JavaScript Publishing Workflow):** + * Extracted from current `main.yml` workflow for clarity and better permission scoping; trigerred on successful build on the main branch. + * This workflow will be configured for **trusted publishing**. + * **Action:** Configure the `release` job with an explicit `permissions` block for OIDC, and remove the `YARN_NPM_AUTH_TOKEN` environment variable. For example: + + ```yaml + permissions: + id-token: write + contents: read + ``` + * **Action:** Remove the `verifyConditionsCmd` from `.releaserc.js` that previously checked for `YARN_NPM_AUTH_TOKEN`. + +2. **`build-native.yml` & `build-native-scanner.yml` (Native Binaries Workflows):** + * The `finalize-release` job will be responsible for generating and publishing the manifests *after* all binaries have been successfully uploaded to the GitHub Release. + * **Action:** Remove steps that generate and upload `.sha256` files. The digest will now be verified via the manifest. + * **Action:** Remove the step that runs `yarn npm tag add ...`. + * **Action:** Add new steps to generate and publish the manifest files. (Note: `contents: write` permission required.) + + ```yaml + - name: Generate Release Manifests + id: generate_manifest + run: | + # This is a conceptual script. The actual implementation would query the + # GitHub API for the release assets and construct the JSON. + VERSION_TAG="${{ github.ref_name }}" # e.g., @appland/appmap-v1.2.3 + MANIFEST_DIR="./dist" + + # Extract the base name (e.g., appmap-v1.2.3) by stripping the @appland/ prefix + BASE_TAG="${VERSION_TAG#@appland/}" + + # Determine the tool name (e.g., appmap) from the base tag + TOOL_NAME="${BASE_TAG%%-v*}" + + LATEST_MANIFEST_FILENAME="${TOOL_NAME}-latest.json" + VERSIONED_MANIFEST_FILENAME="${BASE_TAG}.json" + + echo "Generating manifest for ${VERSION_TAG}" + MANIFEST_CONTENT=$(gh api repos/:owner/:repo/releases/tags/${VERSION_TAG} | jq '{tag_name, assets: [.assets[] | {name, browser_download_url, digest}]}') + + mkdir -p "${MANIFEST_DIR}" + echo "${MANIFEST_CONTENT}" > "${MANIFEST_DIR}/${VERSIONED_MANIFEST_FILENAME}" + cp "${MANIFEST_DIR}/${VERSIONED_MANIFEST_FILENAME}" "${MANIFEST_DIR}/${LATEST_MANIFEST_FILENAME}" + + - name: Publish Manifests to release-manifests branch + uses: peaceiris/actions-gh-pages@4f9cc6602d3f66b9c108549d475ec49e8ef4d45e # v4.0.0 + with: + github_token: ${{ secrets.GITHUB_TOKEN }} + publish_branch: release-manifests + publish_dir: ./dist + keep_files: true + user_name: 'appland-release' + user_email: 'release@app.land' + commit_message: 'Update manifest for ${{ github.ref_name }}' + ``` + +##### **Part 2: Client-Side Extension Modifications** + +The client logic becomes dramatically simpler and more powerful. + +1. **Standard Discovery:** + * To get the latest version, the client makes a single, permanent `GET` request: + `https://raw.githubusercontent.com/getappmap/appmap-js/release-manifests/appmap-latest.json` + +2. **Advanced Discovery (Version Pinning):** + * To get a specific version, the client can now construct a URL to its permanent manifest: + `https://raw.githubusercontent.com/getappmap/appmap-js/release-manifests/appmap-v1.2.3.json` + +3. **Verification:** + * The client will parse the manifest, download the appropriate asset from the `browser_download_url`, and verify its integrity using the SHA256 `digest`. + +--- + +#### **4. Benefits of This Architecture** + +* **Enables Trusted Publishing:** The dependency on `npm dist-tag` is completely removed. +* **Atomic and Race-Free:** Clients will only see a new "latest" manifest *after* all binaries for that release are confirmed to exist. +* **Robust History & Version Pinning:** The two-tier system provides both a stable pointer and a permanent, auditable record for every release, enabling powerful rollback and pinning scenarios. +* **Clean Separation:** The `main` branch history remains clean. Release metadata is logically isolated in its own branch. + +--- + +### **5. Advanced Capabilities Unlocked by This Design** + +#### **Enterprise Asset Mirroring** + +This design makes mirroring trivial and secure. + +* An enterprise client can override a single URL in the extension's settings to point to their own manifest mirror. +* Their mirroring process is simple: + 1. Download the official manifest from `.../appmap-latest.json`. + 2. Download all assets listed in the manifest, verifying each one against its `digest`. + 3. Upload the assets to their internal server. + 4. Host a modified `appmap-latest.json` manifest, updating the `browser_download_url` for each asset but **preserving the original `digest`**. This extends the chain of trust. + +#### **Minimal Manifest Structure** + +```json +{ + "tag_name": "@appland/appmap-v1.2.3", + "assets": [ + { + "name": "appmap-linux-x64", + "browser_download_url": "https://.../appmap-linux-x64", + "digest": "sha256:..." + } + ] +} +``` +* **`tag_name`:** The precise version string. +* **`assets`:** An array of release artifacts. + * **`name`:** The unique filename. + * **`browser_download_url`:** The direct download URL. + * **`digest`:** The SHA256 checksum, prefixed with `sha256:`. \ No newline at end of file