Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
a2fc9dd
CCM-12869: add config -> DynamoDB publisher tooling + release-bundled…
m-houston Feb 27, 2026
fc3a7f4
Refactor Node.js setup in CI workflows and enhance dependency management
m-houston Feb 27, 2026
7b82e85
Ensure workspace packages are generated before bundling in release sc…
m-houston Feb 27, 2026
74fd573
Enhance bundling process in release script to resolve workspace packa…
m-houston Feb 27, 2026
79879b0
Use absolute path for alias in bundling to ensure correct resolution …
m-houston Feb 27, 2026
2d41988
Update dependencies in package.json and package-lock.json for improve…
m-houston Feb 27, 2026
ff73805
Improve debug logging and update bundling configuration for workspace…
m-houston Feb 27, 2026
c2f003c
Add ddb-publish bundling action and update release workflows
m-houston Feb 27, 2026
9234b39
Add workflow artifact upload and improve ddb-publish bundle resolution
m-houston Mar 3, 2026
b30ec6f
Refactor ddb-publish action to use action repository for artifact dow…
m-houston Mar 3, 2026
70ff42e
Enhance ddb-publish action to search multiple workflows for successfu…
m-houston Mar 3, 2026
daecddc
Enhance runPublisher to log detailed steps and summarize loaded records
m-houston Mar 3, 2026
1f2a4bd
Add integration tests for DynamoDB Local and support custom endpoint …
m-houston Mar 3, 2026
6d3216e
Remove redundant console log eslint disables in bundle-release.mjs
m-houston Mar 3, 2026
365fe60
Use testcontainers for ddb-local container management
m-houston Mar 3, 2026
288f9b2
Refactor pre-commit hooks to remove redundant local repo entries
m-houston Mar 3, 2026
b15b3ab
feat(ddb-publisher): document local DynamoDB usage and add sample con…
m-houston Mar 11, 2026
67e9305
feat(tests): enhance integration tests for DynamoDB Local and improve…
m-houston Mar 12, 2026
53ff4e0
refactor(tests): replace 'kind' fields with 'id' in config store vali…
m-houston Mar 12, 2026
eed3b67
feat(tests): add test for explicit AWS env values in local DynamoDB c…
m-houston Mar 12, 2026
423ba1d
refactor(publish): remove 'entity' and 'env' fields from records and …
m-houston Mar 12, 2026
5a82841
test(run): manage AWS environment variables during DynamoDB local tests
m-houston Mar 12, 2026
6054b00
chore(template): add note for code generated by a coding agent in PR …
m-houston Mar 12, 2026
2052c7a
feat(lint): add linting and type checking scripts; update package.jso…
m-houston Mar 13, 2026
07296d3
fix(bundle): use fileURLToPath for packageRoot resolution; update ver…
m-houston Mar 13, 2026
2f9126b
feat(package): add pretypecheck script and update dependencies
m-houston Mar 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
- [ ] I have added tests to cover my changes
- [ ] I have updated the documentation accordingly
- [ ] This PR is a result of pair or mob programming
- [ ] This PR includes code generated by a coding agent

---

Expand Down
65 changes: 65 additions & 0 deletions .github/actions/bundle-ddb-publish/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: "Bundle ddb-publish"
description: "Build and package the ddb-publish release bundle (ddb-publish-bundle.tgz) from the workspace sources."

inputs:
node-version:
description: "Node.js version to use"
required: true
run-typecheck:
description: "Run workspace typecheck before bundling"
required: false
default: "true"

outputs:
tarball-path:
description: "Path to the generated tarball"
value: ${{ steps.bundle.outputs.tarball_path }}

runs:
using: "composite"
steps:
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: "${{ inputs.node-version }}"
cache: npm

- name: Install dependencies
shell: bash
run: |
set -euo pipefail
npm ci

- name: Generate dependencies
shell: bash
run: |
set -euo pipefail
npm run generate-dependencies --workspaces --if-present

- name: Typecheck
if: inputs.run-typecheck == 'true'
shell: bash
run: |
set -euo pipefail
npm run typecheck --workspaces --if-present

- name: Build ddb-publish bundle
shell: bash
run: |
set -euo pipefail
npm run bundle:release --workspace @supplier-config/ddb-publisher

- name: Smoke-test bundle
shell: bash
run: |
set -euo pipefail
node packages/ddb-publisher/artifacts/ddb-publish/index.cjs --help > /dev/null

- name: Package tarball
id: bundle
shell: bash
run: |
set -euo pipefail
tarball="ddb-publish-bundle.tgz"
tar -czf "$tarball" -C packages/ddb-publisher/artifacts/ddb-publish .
echo "tarball_path=$tarball" >> "$GITHUB_OUTPUT"
46 changes: 46 additions & 0 deletions .github/workflows/release-ddb-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: Release ddb-publish bundle

on:
release:
types: [published]
workflow_dispatch:

permissions:
contents: write

jobs:
bundle-and-upload:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Read Node version
id: versions
shell: bash
run: |
set -euo pipefail
echo "node=$(grep '^nodejs\s' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT"

- name: Bundle ddb-publish
id: bundle
uses: ./.github/actions/bundle-ddb-publish
with:
node-version: "${{ steps.versions.outputs.node }}"
run-typecheck: "true"

- name: Upload release asset
if: github.event_name == 'release'
env:
GH_TOKEN: ${{ github.token }}
shell: bash
run: |
set -euo pipefail
gh release upload "${{ github.event.release.tag_name }}" "${{ steps.bundle.outputs.tarball-path }}" --clobber

- name: Upload bundle as workflow artifact
if: github.event_name != 'release'
uses: actions/upload-artifact@v4
with:
name: ddb-publish-bundle
path: "${{ steps.bundle.outputs.tarball-path }}"
16 changes: 16 additions & 0 deletions .github/workflows/stage-2-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -152,3 +152,19 @@ jobs:
sonar_organisation_key: "${{ vars.SONAR_ORGANISATION_KEY }}"
sonar_project_key: "${{ vars.SONAR_PROJECT_KEY }}"
sonar_token: "${{ secrets.SONAR_TOKEN }}"
test-integration:
name: "Integration tests"
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: "Checkout code"
uses: actions/checkout@v4
- name: "Repo setup"
run: |
npm ci
- name: "Generate dependencies"
run: |
npm run generate-dependencies --workspaces --if-present
- name: "Run integration tests"
run: |
npm run test:integration
22 changes: 22 additions & 0 deletions .github/workflows/stage-3-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,28 @@ jobs:
uses: ./.github/actions/build-docs
with:
version: "${{ inputs.version }}"

ddb-publish-bundle:
name: "Bundle ddb-publish CLI"
runs-on: ubuntu-latest
timeout-minutes: 3
steps:
- name: "Checkout code"
uses: actions/checkout@v4

- name: "Bundle ddb-publish"
id: bundle
uses: ./.github/actions/bundle-ddb-publish
with:
node-version: "${{ inputs.nodejs_version }}"
run-typecheck: "false"

- name: "Upload bundle as workflow artifact"
uses: actions/upload-artifact@v4
with:
name: ddb-publish-bundle
path: "${{ steps.bundle.outputs.tarball-path }}"

artefact-1:
name: "Artefact 1"
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion .tool-versions
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
act 0.2.64
gitleaks 8.24.0
jq 1.6
nodejs 22.11.0
nodejs 24.14.0
pre-commit 3.6.0
terraform 1.9.2
terraform-docs 0.19.0
Expand Down
54 changes: 49 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,13 @@ This is a monorepo containing:

- **packages/events** - Event schemas and domain models using Zod
- **packages/event-builder** - CloudEvent builders for domain objects
- **packages/file-store** - Loads on-disk config stores and validates records against the Zod schemas
- **packages/ddb-publisher** - CLI and release bundle for auditing and publishing config-store records into DynamoDB
- **lambdas/** - Lambda function implementations
- **infrastructure/** - Terraform infrastructure as code
- **docs/** - Documentation site
- **actions/ddb-publish** - Composite GitHub Action wrapper around the bundled DynamoDB publisher
- **tests/example-config-store** - Minimal end-to-end config-store fixture used by CLI and integration tests

## Event Builder Package

Expand Down Expand Up @@ -46,6 +50,8 @@ The event-builder package provides functions to build CloudEvents from domain ob

All functions accept domain objects and a starting counter, returning an array of CloudEvents.

The package also exports single-record builders (`buildLetterVariantEvent()`, `buildPackSpecificationEvent()`, `buildSupplierEvent()`, `buildVolumeGroupEvent()`, `buildSupplierAllocationEvent()`, `buildSupplierPackEvent()`), plus `configFromEnv()`, `buildEventSource()`, and `buildBaseEventEnvelope()` from `packages/event-builder/src/index.ts`.

### Testing

```bash
Expand Down Expand Up @@ -94,8 +100,25 @@ const description = $Postage.shape.deliveryDays.meta()?.description;

1. Update domain schema in `packages/events/src/domain/`
2. Update event builder if needed in `packages/event-builder/src/`
3. Update test files in `__tests__/` directories
4. Run tests: `npm test`
3. If the persisted JSON shape or entity folders change, update `packages/file-store/src/`, `packages/ddb-publisher/src/`, and sample records in `tests/example-config-store/`
4. Update test files in `packages/events/src/**/__tests__/`, `packages/event-builder/src/__tests__/`, `packages/file-store/src/__tests__/`, and `packages/ddb-publisher/src/__tests__/` / `src/__integration__/` as needed
5. Regenerate schema artefacts from `packages/events` when required: `npm run gen:jsonschema`, `npm run gen:asyncapi`, `npm run gen:erd`
6. Run tests: `npm run test:unit`

### Working with the Config Store and DDB Publisher

1. Persisted entity directories are fixed to: `volume-group`, `letter-variant`, `pack-specification`, `supplier`, `supplier-allocation`, `supplier-pack`
2. Use `loadConfigStore()` from `packages/file-store/src/loader/config-store-loader.ts` to read JSON records and `validateConfigStore()` from `packages/file-store/src/validation/config-store-validator.ts` to collect schema issues
3. Record filenames matter: `supplier/sup-1.json` must contain `"id": "sup-1"`, or validation fails
4. For local validation without AWS calls, run:

```bash
npm run cli --workspace @supplier-config/ddb-publisher -- \
--source tests/example-config-store \
--env draft \
--table supplier-config-draft \
--dry-run
```

### Adding Metadata to Schema Fields

Expand Down Expand Up @@ -139,16 +162,29 @@ Common markdown rules to follow:
# Install all dependencies
npm install

# Run tests for specific workspace
npm test --workspace=packages/events
npm test --workspace=packages/event-builder
# Run unit tests for all workspaces
npm run test:unit

# Run tests for specific workspaces
npm run test:unit --workspace @nhsdigital/nhs-notify-event-schemas-supplier-config
npm run test:unit --workspace @supplier-config/event-builder

# Run integration tests for the DynamoDB publisher
npm run test:integration --workspace @supplier-config/ddb-publisher

# Build all packages
npm run build --workspaces

# Lint all code
npm run lint

# Type check all workspaces
npm run typecheck

# Check/fix dependency version drift across workspaces
npm run deps:check
npm run deps:sync

# Check markdown formatting
./scripts/githooks/check-markdown-format.sh <file>

Expand All @@ -173,9 +209,17 @@ Check the error output for specific Zod validation failures. Common issues:
- Wrong data types (string vs number)
- Invalid date formats

### Local DynamoDB and Testcontainers

- `SUPPLIER_CONFIG_DDB_ENDPOINT_URL=http://localhost:8000` points `ddb-publisher` at DynamoDB Local; when the endpoint is localhost-based, the CLI supplies default fake AWS credentials if they are unset
- `packages/ddb-publisher/src/__integration__/publish-action.integration.test.ts` uses Testcontainers; if Docker startup fails under Colima, retry with `TESTCONTAINERS_RYUK_DISABLED=true` or `TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock`

## Agent Notes

- The project uses Zod 4.x for schema validation
- Metadata is stored in Zod's global registry via `.meta()`
- Test files should be updated whenever domain models change
- Always run markdown linter when editing `.md` files to catch formatting issues
- `tests/example-config-store/` is the shared fixture for local CLI runs and `ddb-publisher` integration tests
- Persisted entity directory names are enforced by `packages/file-store/src/types.ts` and `packages/file-store/src/loader/config-store-loader.ts`
- Root automation entry points are `npm run test:unit`, `npm run test:integration`, and `npm run typecheck`
Loading
Loading