Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions workspaces/install-dynamic-plugins/.changeset/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Changesets

Hello and welcome! This folder has been automatically generated by `@changesets/cli`, a build tool that works
with multi-package repos, or single-package repos to help you version and publish your code. You can
find the full documentation for it [in our repository](https://github.com/changesets/changesets)

We have a quick list of common questions to get you started engaging with this project in
[our documentation](https://github.com/changesets/changesets/blob/main/docs/common-questions.md)
10 changes: 10 additions & 0 deletions workspaces/install-dynamic-plugins/.changeset/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"$schema": "https://unpkg.com/@changesets/config@3.0.0/schema.json",
"changelog": "@changesets/cli/changelog",
"commit": false,
"fixed": [],
"linked": [],
"access": "public",
"baseBranch": "main",
"updateInternalDependencies": "patch"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@red-hat-developer-hub/cli-module-install-dynamic-plugins': minor
---

Initial release. TypeScript/Node.js port of the RHDH init-container installer (originally Python; see [redhat-developer/rhdh#4574](https://github.com/redhat-developer/rhdh/pull/4574)), packaged as a Backstage CLI module. The `install` command is registered through `createCliModule` so the package is auto-discovered by `backstage-cli` when listed as a dependency. The package also ships a self-contained esbuild bundle as its `bin`, so direct `npx install-dynamic-plugins <dir>` invocations (and RHDH's init-container `COPY` of the `.cjs`) stay fast and don't require `@backstage/cli-node` at runtime. Env vars, on-disk layout, `plugin-hash` format, and tar/OCI security guards are byte-compatible with the previous Python implementation.
1 change: 1 addition & 0 deletions workspaces/install-dynamic-plugins/.eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
module.exports = require('../../.eslintrc.cjs');
54 changes: 54 additions & 0 deletions workspaces/install-dynamic-plugins/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# macOS
.DS_Store

# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
lerna-debug.log*

# Coverage directory generated when running tests with coverage
coverage

# Dependencies
node_modules/

# Yarn files
.pnp.*
.yarn/*
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/sdks
!.yarn/versions

# Node version directives
.nvmrc

# dotenv environment variables file
.env
.env.test

# Build output
dist
dist-types

# Temporary change files created by Vim
*.swp

# MkDocs build output
site

# Local configuration files
*.local.yaml

# Sensitive credentials
*-credentials.yaml

# vscode database functionality support files
*.session.sql

# E2E test reports
e2e-test-report/
6 changes: 6 additions & 0 deletions workspaces/install-dynamic-plugins/.prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
dist
dist-types
coverage
.vscode
.eslintrc.js
**/dist/install-dynamic-plugins.cjs
54 changes: 54 additions & 0 deletions workspaces/install-dynamic-plugins/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
{
"name": "@internal/install-dynamic-plugins",
"version": "0.0.0",
"private": true,
"engines": {
"node": "22 || 24"
},
"workspaces": {
"packages": [
"packages/*"
]
},
"scripts": {
"tsc": "tsc",
"tsc:full": "tsc --skipLibCheck true --incremental false",
"build:all": "backstage-cli repo build --all",
"build:api-reports": "yarn build:api-reports:only --tsc",
"build:api-reports:only": "backstage-repo-tools api-reports -o ae-wrong-input-file-type,ae-undocumented --validate-release-tags",
"build:knip-reports": "backstage-repo-tools knip-reports",
"clean": "backstage-cli repo clean",
"test": "backstage-cli repo test",
"test:all": "backstage-cli repo test --coverage",
"fix": "backstage-cli repo fix",
"lint": "backstage-cli repo lint --since origin/main",
"lint:all": "backstage-cli repo lint",
"prettier:check": "prettier --check .",
"prettier:fix": "prettier --write .",
"new": "backstage-cli new --scope @red-hat-developer-hub",
"postinstall": "cd ../../ && yarn install"
},
"prettier": "@spotify/prettier-config",
"lint-staged": {
"*.{js,jsx,ts,tsx,mjs,cjs}": [
"eslint --fix",
"prettier --write"
],
"*.{json,md}": [
"prettier --write"
]
},
"dependencies": {
"@backstage/cli": "^0.36.0",
"@backstage/cli-defaults": "0.1.0",
"@backstage/repo-tools": "^0.17.0",
"@changesets/cli": "^2.27.1",
"@jest/environment-jsdom-abstract": "^30.3.0",
"@spotify/prettier-config": "^15.0.0",
"jest": "^30.3.0",
"jsdom": "^27.1.0",
"prettier": "^3.4.2",
"typescript": "~5.8.0"
},
"packageManager": "yarn@4.12.0"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
module.exports = require('@backstage/cli/config/eslint-factory')(__dirname);
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
node_modules/
build/
coverage/
*.log
.yarn/
dist/
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
node_modules/
coverage/
build/
dist/
package-lock.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# cli-module-install-dynamic-plugins

Backstage CLI module that downloads, extracts, and configures RHDH dynamic plugins listed in a `dynamic-plugins.yaml` file.

This package replaces the previous Python implementation (`install-dynamic-plugins.py`) with a TypeScript/Node.js implementation. The runtime contract — input config, output `app-config.dynamic-plugins.yaml`, on-disk layout, hash-based change detection, lock file — is **unchanged**.

The package has two invocation paths, both running the same `installer.ts` pipeline:

- **`bin/install-dynamic-plugins` → fast-path** that loads `dist/installer.cjs.js` directly. Direct `npx install-dynamic-plugins` and any host that resolves the bin via `node_modules/.bin/...` hits this path — bypasses `@backstage/cli-node`'s `runCliModule` dispatch (~80 ms saved on cold start).
- **`main: dist/index.cjs.js` → `createCliModule(...)`**, exposed for `backstage-cli` discovery. When a host project lists this package as a dependency, `backstage-cli install <dynamic-plugins-root>` is registered automatically.

## Usage

### Direct (bundled bin)

```sh
npx @red-hat-developer-hub/cli-module-install-dynamic-plugins ./dynamic-plugins-root
```

Or install globally:

```sh
npm install -g @red-hat-developer-hub/cli-module-install-dynamic-plugins
install-dynamic-plugins ./dynamic-plugins-root
```

### Via `backstage-cli` discovery

When the package is a dependency of a project that uses `backstage-cli`, the `install` command is registered automatically:

```sh
backstage-cli install ./dynamic-plugins-root
```

Runtime requirements: Node.js 22 or 24, and `skopeo` on `PATH` for OCI plugin support. `npm` is also expected on `PATH` for NPM-sourced plugins.

## How RHDH consumes it

The init container invokes the wrapper `install-dynamic-plugins.sh /dynamic-plugins-root`, which delegates to the bin installed via `yarn install` from this package (see [redhat-developer/rhdh#4908](https://github.com/redhat-developer/rhdh/pull/4908)). Node.js is already present in the runtime image (it runs the Backstage backend), and `skopeo` is installed for OCI inspection — no new system packages are required.

## Architecture

```
src/
├── index.ts # createCliModule default export (backstage-cli discovery)
├── command.ts # loader for the `install` command (used by cli-module)
├── installer.ts # install pipeline + main() — the single source of truth
├── log.ts # uniform stdout logger
├── errors.ts # InstallException
├── types.ts # PluginSpec / Plugin / PluginMap / PullPolicy + constants
├── util.ts # shared helpers (fileExists, isInside, isPlainObject, tar filters)
├── run.ts # subprocess wrapper with structured errors
├── concurrency.ts # Semaphore + mapConcurrent + getWorkers()
├── which.ts # PATH lookup (no `which` dep)
├── skopeo.ts # Skopeo wrapper with promise-based inspect cache
├── image-resolver.ts # registry.access.redhat.com → quay.io fallback
├── image-cache.ts # OciImageCache — share OCI tarballs across plugins
├── tar-extract.ts # streaming OCI / NPM extraction with security checks
├── npm-key.ts # NPM package-spec parsing
├── oci-key.ts # OCI package-spec parsing + {{inherit}} + auto-path
├── integrity.ts # streaming SRI integrity verification
├── merger.ts # plugin merging + deep-merge with conflict detection
├── plugin-hash.ts # hash for change-detection ("already installed?")
├── installer-oci.ts # install one OCI plugin
├── installer-npm.ts # install one NPM (or local) plugin
├── catalog-index.ts # CATALOG_INDEX_IMAGE extraction
└── lock-file.ts # exclusive lock + SIGTERM cleanup
```

### Concurrency strategy (resource-conscious)

OCI plugin downloads are parallelized via `mapConcurrent`. NPM `npm pack` calls stay sequential because the upstream npm registry throttles parallel fetches.

The default worker count comes from `getWorkers()`:

```
Math.max(1, Math.min(Math.floor(availableParallelism() / 2), 6))
```

`availableParallelism()` honours cgroup CPU limits, so init containers in OpenShift won't try to use 16 workers on a 0.5 CPU pod. Override with `DYNAMIC_PLUGINS_WORKERS=<n>`.

### Memory budget

All tar extraction is streaming via `node-tar` — large layers never load into RAM. SHA verification streams chunks through `node:crypto`. A typical 10-plugin run sits around 20–80 MB peak RSS, comfortably below an init-container memory limit of 512 Mi.

### Security checks (parity with the previous Python script)

| Check | Source |
| --------------------------------------------------------------------- | ------------------------------------ |
| Path-traversal in plugin path (`..`, absolute paths) | `tar-extract.ts` |
| Per-entry size cap (zip bomb) — `MAX_ENTRY_SIZE`, default 20 MB | `tar-extract.ts`, `catalog-index.ts` |
| Symlink / hardlink target must stay inside destination | `tar-extract.ts` |
| Reject device files / FIFOs / unknown entry types | `tar-extract.ts` |
| `package/` prefix enforced for NPM tarballs | `tar-extract.ts` |
| SRI integrity verification (`sha256` / `sha384` / `sha512`) | `integrity.ts` |
| Registry fallback: `registry.access.redhat.com/rhdh` → `quay.io/rhdh` | `image-resolver.ts` |

## Environment variables

| Variable | Default | Purpose |
| --------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------ |
| `MAX_ENTRY_SIZE` | `20000000` | Per-entry byte limit when extracting tarballs |
| `SKIP_INTEGRITY_CHECK` | `false` | When `true`, skip the SRI integrity check for remote NPM packages |
| `CATALOG_INDEX_IMAGE` | _(unset)_ | OCI image to extract `dynamic-plugins.default.yaml` and catalog entities from |
| `CATALOG_ENTITIES_EXTRACT_DIR` | `$TMPDIR/extensions` | Where to extract `catalog-entities/` from the catalog-index image |
| `DYNAMIC_PLUGINS_WORKERS` | `auto` | Worker count override for parallel OCI downloads (`auto` uses `availableParallelism()/2`, capped at 6) |
| `DYNAMIC_PLUGINS_LOCK_TIMEOUT_MS` | `600000` (10 min) | Max time to wait for the lock file before aborting with an error |

## Development

From the workspace root:

```sh
yarn install
yarn tsc # type-check
yarn test # Jest unit tests (166 tests)
yarn workspace @red-hat-developer-hub/cli-module-install-dynamic-plugins build
```

`yarn build` runs `backstage-cli package build` and emits the unbundled `dist/*.cjs.js` + type declarations. The package is published as-is; no committed bundle.

## Compatibility notes

- The **input contract** matches the previous Python script exactly: same `dynamic-plugins.yaml` schema (`includes`, `plugins`, `package`, `pluginConfig`, `disabled`, `pullPolicy`, `forceDownload`, `integrity`).
- The **output contract** matches: same `app-config.dynamic-plugins.yaml`, same plugin directory layout, same `dynamic-plugin-config.hash` / `dynamic-plugin-image.hash` files.
- `{{inherit}}` semantics, OCI path auto-detection, registry fallback, integrity algorithms, lock-file behaviour are preserved.
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/usr/bin/env node
/*
* Copyright Red Hat, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

const path = require('node:path');
const fs = require('node:fs');

/* eslint-disable-next-line no-restricted-syntax */
const isLocal = fs.existsSync(path.resolve(__dirname, '../src'));

if (isLocal) {
require('@backstage/cli-node/config/nodeTransform.cjs');
}

const { runCliModule } = require('@backstage/cli-node');
const cliModule = require(isLocal ? '../src/index' : '..').default;
const pkg = require('../package.json');

runCliModule({ module: cliModule, name: pkg.name, version: pkg.version });
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
## CLI Report file for "@red-hat-developer-hub/cli-module-install-dynamic-plugins"

> Do not edit this file. It is a report generated by `yarn build:api-reports`

### `install-dynamic-plugins`

```
Usage: @red-hat-developer-hub/cli-module-install-dynamic-plugins [options] [command]

Options:
-V, --version
-h, --help

Commands:
help [command]
install
```

### `install-dynamic-plugins install`

```
Usage: @red-hat-developer-hub/cli-module-install-dynamic-plugins install [flags...] <dynamic-plugins-root>

Options:
-h, --help
```
Loading
Loading