feat: suppress feature specs covered by scenario-template ontology#332
Conversation
For every Invoke/Observe step in a scenario-template instantiation (EdgeLifecycle, EntityLifecycle, UpdatedFieldVisibleOnReadBack, StateTransitionVisibleAfterAction) the materializer now omits the per-endpoint feature spec for the operationId those steps target. The lifecycle spec is the canonical functional test for those operations; emitting a parallel feature spec is either strictly weaker (EntityLifecycle case) or structurally malformed (EdgeLifecycle case: a key-only prereq chain cannot encode the establish-before-revoke precondition). The behaviour is template-agnostic and coverage-driven, not hard-coded: - New module `materializer/src/coverage.ts` walks the on-disk template scenario JSONs under `generated/<config>/scenarios/templates/<Template>/` and collects every operationId bound to an Invoke/Observe step. Per template, the opt-out lives on the ABox as `ScenarioTemplate.suppressesFeatureTest` (defaults to `true`); flipping it to `false` lets a future non-functional template (smoke / chaos / load) emit a spec without claiming coverage of the operations it touches. - The orchestrator filters the `--all` feature loop by the resulting suppression set and writes a stable, sorted `coverage.json` artefact next to the suite so the suppression is diffable in PRs. L3 invariants updated: - Partition-cut #162 PR 4 now subtracts suppressed opIds. - awaitEventually #106 skips suppressed opIds (their reads are wrapped by the template emitter). - #305 Phase 5a getIncident chain-shape assertions move from `getIncident.feature.spec.ts` to `state-transitions/Incident.resolveIncident.lifecycle.spec.ts`; the emitted chain (deploys incident-script-task.bpmn, runs searchIncidents → extracts incidentKey → consumes via runtime binding) is unchanged. - New positive invariant: every suppressed opId has a coverage entry pointing at an emitted lifecycle spec that exists on disk. Catches the regression class where an empty/stale coverage map silently deletes the only spec for an operation. 73 feature specs are suppressed in the camunda-oca config (covered by the 26 lifecycle scenarios across the 4 active templates). Closes #331
There was a problem hiding this comment.
Pull request overview
Adds a coverage-derived suppression mechanism so per-endpoint Playwright *.feature.spec.ts files are not emitted for operations already exercised by emitted scenario-template lifecycle/state-transition specs, and writes a coverage.json artefact to make this suppression auditable and testable.
Changes:
- Introduces
materializer/src/coverage.tsto derive covered/suppressedoperationIds by scanning planner-emitted template scenario JSONs. - Updates the materializer
--allfeature emission loop to skip suppressed operations and to emit a stablecoverage.jsonalongside generated suites. - Extends the scenario-template ontology schema with
suppressesFeatureTestand updates L3 regression invariants to account for suppression and to assert suppressed ops are backed by an on-disk lifecycle spec.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| path-analyser/src/ontology/scenarioTemplateSchema.ts | Adds suppressesFeatureTest to the TS schema for scenario templates. |
| ontology/vocabulary/scenario-template.schema.json | Regenerates the JSON schema to include suppressesFeatureTest. |
| materializer/src/index.ts | Builds coverage before the --all feature loop, skips suppressed ops, and writes coverage.json. |
| materializer/src/coverage.ts | New coverage extractor that walks template scenario JSONs to derive suppression + coverage entries. |
| configs/camunda-oca/regression-invariants.test.ts | Adjusts invariants to subtract suppressed ops and adds a new invariant validating coverage entries point to existing lifecycle specs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Reviewer note: hardcoded template registry in
|
…nomy Adds tests/codegen/coverage.test.ts with four focused assertions on materializer/src/coverage.ts: 1. default (no ABox) treats every template as suppressing 2. suppressesFeatureTest: false honours the opt-out 3. only invoke/observe contribute; prereqChain is scaffolding 4. templates absent from templateOutputDirs are skipped Closes a guard gap surfaced during PR #332 review: the false-opt-out path was previously exercised only at the ABox level (no row has suppressesFeatureTest: false today) and a future rename of the schema field or the reader-side property access could silently invert the default. Test 2 fails red on either drift. Refs #331, PR #332.
Implements coverage-driven suppression of per-endpoint feature specs for operations already exercised by a scenario-template instantiation (EdgeLifecycle, EntityLifecycle, UpdatedFieldVisibleOnReadBack, StateTransitionVisibleAfterAction).
What
materializer/src/coverage.tswalks emitted template scenario JSONs and collects every operationId bound to anInvoke/Observestep.ScenarioTemplate.suppressesFeatureTeston the ABox (defaults totrue). Schema TS source + regenerated JSON Schema artefact updated in lockstep.--allfeature loop by the resulting suppression set and writes a stable, sortedcoverage.jsonalongside the suite so the suppression is diffable in PRs.Why
A scenario-template instantiation is the canonical functional test for the operationIds bound to its
Invoke/Observesteps. Emitting a parallel per-endpoint feature spec is either strictly weaker (EntityLifecycle case) or structurally malformed (EdgeLifecycle case: a key-only prereq chain cannot encode the establish-before-revoke precondition the feature spec needs). See #331 for the full motivation.The implementation is template-agnostic — adding a new
ScenarioTemplatetoconfigs/<config>/ontology/scenario-templates.json(and wiring its output dir intotemplateOutputDirs) extends suppression automatically.Numbers
In the camunda-oca config: 73 feature specs suppressed across 26 lifecycle scenarios (4 templates).
L3 invariants
Updated to reflect the intentional behaviour change:
getIncidentchain-shape assertions re-pointed fromgetIncident.feature.spec.ts→state-transitions/Incident.resolveIncident.lifecycle.spec.ts; the emitted chain is unchanged.Test layers touched
configs/camunda-oca/regression-invariants.test.ts(3 updated, 1 added).ontology/vocabulary/scenario-template.schema.json(mandatory pair with the TS source).Pre-push gate
npm run build:analyser,npm run build:emitter-sdk✅tsc --noEmitacross all 6 workspace tsconfigs ✅TEST_SEED=snapshot-baseline npm run testsuite:generate && npm run generate:request-validation✅npm test— 627 passed, 4 skipped, 0 failed ✅npm run lint— 0 errors ✅Closes #331